Slicing, Chatting, and Refining: A Concept-Based Approach for Machine Learning Model Validation with ConceptSlicer
Abstract
As machine learning (ML) gains wider adoption in real-world applications, the validation of ML models becomes fundamental for its productization, particularly in safety-critical applications. Recently, data slice finding has emerged as a popular method for validating ML models, but it requires additional metadata or cross-modal embeddings for the slices to be interpretable. We propose ConceptSlicer, an integrated workflow that facilitates the slicing of computer vision models using visual concepts. This approach breaks down the image dataset into interpretable visual concepts, serving as metadata in the slice finding process. Our system offers insights into model issues and enables a deeper understanding of computer vision models' strengths and weaknesses. We evaluate ConceptSlicer through interviews with eight domain experts and machine learning practitioners, and fine-tune the ML models based on their feedback. Our study also highlights varied attitudes towards large foundational models, encouraging contemplation of the challenges and opportunities presented by this technological advancement. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000670800Publication status
publishedExternal links
Book title
IUI '24: Proceedings of the 29th International Conference on Intelligent User InterfacesPages / Article No.
Publisher
Association for Computing MachineryEvent
Subject
Data Slicing; Data-Centric AI; Human-in-the-loopOrganisational unit
02219 - ETH AI Center / ETH AI Center
More
Show all metadata
ETH Bibliography
yes
Altmetrics