Exploring Label Structure and Spatial Attention for Fashion Images Classification

26 May 2020

Beatriz Ferreira ISR/IST

In order to make decisions, for instance when purchasing a product, people rely on rich and accurate descriptions, which entail multi-label retrieval processes. However, multi-label classification is challenged by high dimensional and complex feature spaces and its dependency on large and accurately annotated datasets. Deep learning approaches brought a definite breakthrough in performance across numerous machine learning problems, and image classification was, undoubtedly, one of the tasks where these approaches had greater repercussions. In this presentation we will focus on image classification of fashion images, using deep learning approaches to tackle the multi-class/multi-label problems in order to generate rich images descriptions. Fashion datasets are challenging because they include a vast amount of similarly looking images and they are annotated with a large diversity of attributes but with few labels per exemplar. To address the previous issues we explore domain knowledge to constrain the (otherwise completely data-driven) solutions. Specifically, we first show how to incorporate knowledge about annotations structure. Secondly, we use context and semantic localization to guide an attention mechanism that designs the feature space by focusing on visually meaningful regions. We show with thorough experimentation the performance gains achieved for both cases.


Download Slides


Exploring Label Structure and Spatial Attention for Fashion Images Classification banner Beatriz Quintino Ferreira is a PhD student of the NETSyS program, from the Signal and Image Processing Group at Instituto de Sistemas e Robótica. Her main research interests lie in the intersection of Computer Vision and Machine Learning. She is also an apologist of interpretable models, as she deems interpretability to be fundamental to the development of richer and more robust models, more easily comprehended by humans. She has been a PhD student intern at Farfetch and a visiting scholar at CMU. Some of her recent publications can be found on KDD and on ICCV workshops.