The growing importance of the Explainable Artificial Intelligence (XAI) field has resulted in the proposal of several methods for producing visual heatmaps of the classification decisions of deep learning models. However, visual explanations are not enough since different end-users have different backgrounds and preferences. Natural language explanations (NLEs) are inherently understandable by humans and, thus, can complement visual explanations. In the literature, the problem of generating NLEs is usually framed as traditional supervised image captioning, where the model learns to produce some human collected ground-truth explanations. In this talk, the audience is invited to navigate the state-of-the-art in image captioning and NLE generation, from the very first approaches with LSTMs to more recent Transformer-based architectures. The last part of the talk will encompass the speaker’s ongoing research on the topic, particularly focusing on the distinction between image captioning and NLE generation and on how we can go from one to the other without requiring human collected NLEs for training.
Isabel Rio-Torto is a second-year PhD candidate at Faculdade de Ciências da Universidade do Porto (FCUP) and a researcher at Instituto de Engenharia, Sistemas e Computadores, Tecnologia e Ciência (INESC TEC). Her PhD, “Self-explanatory computer-aided diagnosis with limited supervision”, supervised by professors Luís F. Teixeira and Jaime S. Cardoso, focuses on tackling the two main issues that prevent deep learning models from being deployed to the clinics: the need for large amounts of labelled data and lack of explainability. Currently she is working on the generation of Natural Language Explanations for Computer Vision applications. She is also an invited teaching assistant at Faculdade de Engenharia da Universidade do Porto (FEUP). Isabel previously studied Electrical and Computers Engineering at FEUP, where she developed her master thesis “Producing Decisions and Explanations: A Joint Approach Towards Explainable CNNs”, which won her the 1st place at 2020 Fraunhofer Portugal Challenge (MSc category) and the Best 2020 Master Thesis Award by Associação Portuguesa de Reconhecimento de Padrões (APRP).
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.AcceptRejectPrivacy Policy
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.