A new method for auditing artificial-intelligence models has exposed signals that simpler checks can misread, including a microscope artifact in skin images and sex in a binge-drinking prediction model. Called ICON, it estimates how much a deep network relies on each supplied concept while accounting for the other concepts and the outcome at the same time.
The work is an arXiv preprint, version 4, dated 15 October 2026. Its tests covered synthetic images, dermoscopic skin-image models and UK Biobank neuroimaging models.
A different way to measure what a model uses
Many explanation methods ask whether a single concept can be decoded from a model representation. ICON instead treats the concepts as a group. It first uses partial least squares to combine the concepts and the model outcome into mutually orthogonal latent concepts, then uses Type I sums of squares to allocate explained variance back to the strongest original contributor. Any variance not assigned to the supplied concepts remains unexplained.
That design is meant to reduce misleading univariate scores when concepts are correlated with one another or with the outcome, and the audit begins with a supplied concept set.
The synthetic test
In the ToyBrains benchmark, the researchers compared 90 trained models across six scenarios, five architectures and three random seeds. Each method used 500 random test samples and 30 random concepts.
The simulation used total variation distance to compare each method's estimated breakdown with the controlled ground truth. ICON had the lowest mean error at two high-dimensional internal layers: 0.34 and 0.22, compared with 0.52 and 0.39 for the best baseline methods. At the scalar output, the figures were 0.14 for partial R2 and 0.15 for ICON.
The method also held its error near 0.20 as the concept set grew from five to 100 concepts. Baseline errors rose to between 0.40 and 0.85 in that sweep, while ICON's error levelled off at about 250 auditing samples. These simulation experiments were descriptive rather than hypothesis-tested.
A microscope artifact becomes a warning signal
The skin-image test used 25,331 images from the ISIC 2019 training set. The researchers removed 795 images with a naturally occurring microscope-lens artifact, leaving 24,536 images annotated for nine artifact concepts.
The researchers then varied how strongly the artifact was built into model training. As that shortcut became stronger, ICON's relative importance assigned to the microscope rose from about 0% to 43% in the prediction logit. When the method audited the clean model without the artifact, it assigned less than 1% importance to the microscope, while a linear probe reached as high as 12%.
Taken together, those controlled runs show why joint auditing matters: the microscope signal rose with the imposed artifact but stayed below 1% when the clean model was audited.
Brain models reveal strong associations
Both UK Biobank neuroimaging tasks used 30 concept variables alongside the model outcome, giving ICON 31 variables to decompose. In the binge-drinking classification analysis, the method highlighted sex as an important concept. The model's reported accuracy was 60.0%, with a 0.5 percentage-point standard deviation, while a retrained model using a sex-balanced cohort reached 53.3% with the same reported spread.
The binge-drinking analysis included 6,255 participants in the natural-correlation cohort and 6,309 in the sex-balanced cohort. The reported correlation between the binge-drinking outcome and the sex concept was approximately 0.23. The lower accuracy after sex balancing shows the change in model performance in this comparison, but the association should not be read as a causal mechanism for binge drinking.
A separate brain-age model was trained with 10,000 healthy participants and audited on a fixed set of 5,000 participants drawn from a larger pool not used for training. The model reported a mean absolute error of 2.89 years and an R2 of 0.750. At the final layer, ICON attributed about 77% of the variance to age, while the listed non-age concepts together accounted for less than 1%.
About 22% of the scalar-output variance was unexplained by the 30 supplied concepts or the outcome. In tests designed to alter selected concept relationships, six of eight out-of-distribution comparisons were feasible; none showed a significant error difference, and the mean absolute change in error was at most 0.069 years. A conditional-independence test detected three small dependencies that matched ICON's three leading non-age concepts.
How the method performed across tasks
Across these settings, the results support ICON as an audit tool for deep representations when the concepts of interest are supplied. The synthetic comparison showed lower error at high-dimensional layers, the skin-image exercise tracked a controlled artifact, and the brain-age model separated age from the small contribution of the listed alternatives.
The authors' comparisons were not all designed as formal significance tests. The first two experiments were descriptive; the third used two-sided tests with Bonferroni correction. No sample-size calculation was performed, and cohort sizes were set by annotation availability and the ISIC release.
The work was funded by multiple Deutsche Forschungsgemeinschaft grants, with additional support from the Gemeinnützige Hertie Stiftung and the Cluster of Excellence “Machine Learning – New Perspectives for Science”.
Paper data and sources
Original title: ICON Decomposition: Multivariate Concept-Level Explanations of Deep Representations for Model Auditing
Authors: Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer et al.
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-26
DOI: Not available
Original paper · Full text