A Metacognitive Perspective on Anomaly Detection with Vision Foundation Models

Monitoring and deciding dissociate in anomaly detection: a Mahalanobis scorer monitors better, and an Extreme Value Theory threshold recovers its segmentation without anomalous data.

Abstract

Anomaly Detection (AD) in images identifies defective regions by scoring against a model of normality learned from defect-free data. Current state of the art methods encode images via a frozen Vision Foundation Model (VFM) backbone and score them by their Nearest-Neighbor (NN) distance to a memory bank of normal samples. However, due to the parallel evolution of VFM encoders and scoring mechanism, the need to disentangle their individual contributions has emerged, in order to guarantee the deployment of the best performing model in specific industrial production environments. In our work, we analyze AD with a metacognitive perspective, we freeze the backbone and swap the scorer, replacing NN distance with a Mahalanobis score over a Gaussian distribution. By doing that, we can evaluate two different metacognitive functions of the scoring mechanisms: monitoring how far an observation is from the learned model of normality via a threshold-free metric, and deciding when the difference justifies an action according to a threshold. We show that these two metacognitive functions can dissociate. We further notice that replacing a NN memory bank with a simpler Mahalanobis model improves pixel-level monitoring (+8.00 AU-PRO0.05 on MVTec AD 2) while degrading the segmentation performance after the thresholding step (−13.46 F1 ). Lastly, we show that a threshold derived by Extreme Value Theory (EVT) recovers that loss without using anomalous data: the F1 difference between the two scorers narrows from 13.46 to 0.61 points on MVTec AD 2, with consistent gains on MVTec AD and VisA.

Cite

@inproceedings{galiazzo2026metacognitive,
  title     = {A Metacognitive Perspective on Anomaly Detection with Vision Foundation Models},
  author    = {Galiazzo, Matteo and Onorato, Gabriele and Loi, Dario and Bastian, Nathaniel D. and Restuccia, Francesco},
  booktitle = {Proceedings of the AAAI Fall Symposium on Artificial Metacognition (METACOG)},
  series    = {AAAI Fall Symposium Series},
  address   = {Arlington, VA, USA},
  month     = {November},
  year      = {2026}
}

All publications