ML-Inception: understanding where and why models work (and don’t work)
21 June 2022
Carlos Soares FEUP
A subgroup discovery-based method has recently been proposed to understand the behavior of models in the (original) feature space. The subgroups identified represent areas of feature space where the model obtains better or worse predictive performance than on average. For instance, in the marketing domain, the approach extracts subgroups such as: for customers with higher income and who are younger, the random forest achieves higher accuracy than on average. Here, we propose the use of metalearning to analyze those subgroups on the metafeature space, where they are characterized in a domain-independent way, using statistical and information theoretic properties. We then use association rules to relate characteristics of the subgroups to improvement or degradation of the performance of models. For instance, in the same domain, the approach extracts rules such as: when the class entropy decreases and the mutual information increases in the subgroup data, the random forest achieves lower accuracy. We illustrate the approach with some empirical results.
Carlos Soares is an Associate Professor at the Faculty of Engineering of U. Porto, where he holds the positions of Subdirector of the Dep. of Informatics Engineering, Director of the Ph.D. programme on Informatics Engineering and Adjunct Director of the M.Sc. programme on Data Science and Engineering. Carlos teaches at the Porto Business School, where he is the co-Director of the executive programme on Business Intelligence & Analytics. He is also an External Advisor for Intelligent Systems at Fraunhofer Portugal AICOS, a researcher at LIACC and a collaborator at LIAAD-INESC TEC. The focus of his research is on metalearning/autoML but he has a general interest in Data Science. He has participated in 20+ national and international R&ID as well as consulting projects. Carlos regularly collaborates with companies, including Feedzai, Accenture and InovRetail. He has published/edited several books and 150+ papers in journals and conferences, (90+/125+ indexed by ISI/Scopus) and supervised 10+/50+ Ph.D./M.Sc. thesis. Recent participation in the organization of events, includes ECML PKDD 2015, IDA 2016 and Discovery Science 2021, as programme co-chair. In 2009, he was awarded the Scientific Merit and Excellence Award of the Portuguese AI Association.
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.AcceptRejectPrivacy Policy
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.