Publications

While Vision Foundation Models (VFMs) are increasingly used in several computer vision tasks, their internal representations remain substantially opaque. This mostly stems from polysemanticity, i.e., individual neurons encode mixtures of unrelated visual patterns. In stark contrast, human reasoning is inherently monosemantic, i.e., it naturally relies on distinct, isolated features to process information. Therefore, monosemanticity needs to be implemented within the model’s activation space to obtain interpretable concept representations. Existing methods mostly operate on convolutional networks, so when applied to VFMs their “crop-and-resize” approach distorts the model’s representations by disrupting global self-attention mechanisms and discarding spatial geometry. Furthermore, prior work requires downstream classification labels and is based on KL-divergence, which requires propagating gradients back from the classifier head, ultimately incurring excessive concept extraction time and making it hardly applicable to extract task-agnostic concepts. To overcome these issues, we propose PLACE, a label-free concept extraction framework that operates directly on the native patch-token activations of a VFM. We introduce a geometric Gram-matrix alignment loss that, jointly with reconstruction, upper-bounds the downstream KL divergence of any linear classifier, providing a label-free proxy for faithfulness, i.e., the notion that extracted concepts, although obtained without labels, are relevant to downstream classification tasks. Evaluations on image classification tasks (ImageNet and COCO) on DINOv2, ViT-MAE and ViT-B/16 show that PLACE ❶ extracts concepts about 20× faster than KL-supervised extraction and 11× faster than Sparse Autoencoders (SAEs) using our Gram alignment loss. Moreover, PLACE yields ❷ sparser concepts than SAEs in four of six backbone and dataset configurations (up to 0.678 vs. 0.276 Gini sparsity score), and ❸ concepts that align with those obtained under classifier supervision (mean matched cosine similarity of 0.923 on ViT-MAE). Finally, when a single concept bank is learned from unlabeled multi-class data, the Gram loss improves this alignment over reconstruction loss alone.

Anomaly detection in industrial visual inspection is challenged by the scarcity of defective samples and the computational demands of modern architectures. While recent methods have achieved strong performance through synthetic anomaly generation or pretrained feature extractors, these approaches introduce domain bias and may fail to capture the true distribution of real-world defects. Vision Transformer based methods offer powerful global context modeling but suffer from slow convergence and high data requirements that limit practical deployment. In this paper, we propose Caps-ViT, a reconstruction-based architecture by training solely on normal, real-world industrial imagery. Our approach eliminates dependence on artificial defect synthesis and external pretraining through three key contributions: (1) the integration of a DigitCaps layer for hierarchical feature encoding in the latent space, (2) a decaying noise schedule for the Gaussian Mixture Density Network that balances regularization with reconstruction precision across training stages, and (3) the adoption of Level Weighted SSIM loss to accelerate color and texture convergence. We evaluate Caps-ViT on five benchmark datasets comprising 39 product categories: MVTec AD, BTAD, KSDD, VisA, and the recently introduced MVTec AD 2. Training exclusively on normal authentic images, our approach surpasses the synthetic-free transformer-based VT-ADL, in 27 out of 31 categories on established benchmarks. On the challenging MVTec AD 2 dataset, Caps-ViT achieves 60.4% AU-PRO0.30, surpassing the embedding-based method EfficientAD (58.7%) by +1.7%. Latent space analysis reveals that performance correlates strongly with the geometric separability of normal and anomalous representations, providing interpretable insights into model behavior across different anomaly types.

Heterogeneous graphs, whose nodes and edges can belong to different types and feature spaces, arise in many real-world domains, including biology, recommendation, social networks, and computer systems. Existing heterogeneous graph neural networks typically handle this heterogeneity at the architectural level through relation-specific modules, meta-path machinery or type-aware attention, which often leads to increasingly specialised, parameter-heavy designs. In this work, we propose HetSheaf, a framework for learning heterogeneous graphs through cellular sheaves. Instead of encoding heterogeneity solely in the architecture, HetSheaf represents it directly in the underlying data structure by assigning type-aware local feature spaces and learning restriction maps conditioned on node features, node types, and edge types. To support graph-level prediction, we further introduce SheafPool, a universal stalk-space readout that aggregates node representations while being invariant to local changes of basis, thereby making graph classification with sheaf networks well-defined and achieving an F1 Score up to 42 percentage points higher than mean pooling. Across a diverse suite of benchmarks (node classification, link prediction and graph classification), HetSheaf consistently achieves up to 2 percentage points higher performance (up to 94.93% Macro F1 Score on node classification and up to 99.64% on link prediction) on the Heterogeneous Graph Benchmark (HGB) framework against homogeneous (GCN, GAT, GIN, GraphSAGE), heterogeneous (R-GCN, HAN, HGT) and type-agnostic sheaf baselines, while reducing the number of parameters by up to 10×.

In Test-Time Adaptation (TTA), when out-of-distribution samples interleave unpredictably with corrupted in-distribution samples, adaptation without corrective measures is prone to significant performance decrease. Existing protocols rely on unreliable factors (e.g., predictive entropy) which are unable to distinguish corrupted in-distribution data from semantically novel open-set samples. To address this issue, we formalize TTA through a two-level metacognitive framework that decouples task execution from control, facilitating a unified taxonomy that categorizes existing TTA protocols across three core axes, i.e., which knowledge is monitored, which part of the models are targeted for adaptation, and how adaptation occurs. We provide evidence that robust corrective measures in TTA require monitoring grounded in structured internal representations to reliably estimate model competence. Lastly, we frame monitoring as a resource-allocation problem, examine neuromorphic substrates for bounded self-regulation, and outline key open challenges.

Anomaly Detection (AD) in images identifies defective regions by scoring against a model of normality learned from defect-free data. Current state of the art methods encode images via a frozen Vision Foundation Model (VFM) backbone and score them by their Nearest-Neighbor (NN) distance to a memory bank of normal samples. However, due to the parallel evolution of VFM encoders and scoring mechanism, the need to disentangle their individual contributions has emerged, in order to guarantee the deployment of the best performing model in specific industrial production environments. In our work, we analyze AD with a metacognitive perspective, we freeze the backbone and swap the scorer, replacing NN distance with a Mahalanobis score over a Gaussian distribution. By doing that, we can evaluate two different metacognitive functions of the scoring mechanisms: monitoring how far an observation is from the learned model of normality via a threshold-free metric, and deciding when the difference justifies an action according to a threshold. We show that these two metacognitive functions can dissociate. We further notice that replacing a NN memory bank with a simpler Mahalanobis model improves pixel-level monitoring (+8.00 AU-PRO0.05 on MVTec AD 2) while degrading the segmentation performance after the thresholding step (−13.46 F1 ). Lastly, we show that a threshold derived by Extreme Value Theory (EVT) recovers that loss without using anomalous data: the F1 difference between the two scorers narrows from 13.46 to 0.61 points on MVTec AD 2, with consistent gains on MVTec AD and VisA.

While Sheaf Neural Networks (SNNs) enrich n-node m-edge diffusion of f-dimensional node features with d-dimensional node and edge stalks, they require computing a sparse nd x nd sheaf Laplacian, whose computation is prohibitive on edge devices. This paper addresses this issue with Implicit Sheaf Operators (ISOs), a type of sheaf topological operators that leverage the topological structure of cellular sheaves to decompose the global diffusion process into local operator-vector products. We mathematically prove that ISO eliminates the O(md^2 log(md^2)) sparse-coalescing term and reduces the space complexity by O(min(d, f)) with respect to traditional Neural Sheaf Diffusion (NSD). Our experiments indicate that ISO-based operators achieve latency reduction of up to 4.87× and save up to 3.92× peak GPU memory. On real graph classification data, ISO uses up to 1.24× less energy and 3.08× less on-device memory as measured on a Jetson Orin Nano. Finally, to bring ISO to practice, we introduce Sheaf Convolutional Networks (SCNs), an ISO-based SNN attaining up to 3.93% higher accuracy compared to NSD-based SNNs on node classification benchmarks.

The information bottleneck (IB) finds a compressed representation that preserves information relevant to a task. Traditional IB defines this compression–prediction tradeoff for a fixed joint input–output distribution. While robust IB variants account for perturbations in that distribution, they do not explicitly preserve the information needed to estimate the uncertainty at the output of the bottleneck. To address this fundamental limitation, we propose the credal information bottleneck (CIB), which for the first time represents uncertainty in the data-generating distribution and the data processing, and propagates it over the bottleneck, enabling its estimation and categorization at the bottleneck output. Importantly, CIB disentangles aleatoric, data-induced and processing-induced uncertainty. This knowledge about the sources of uncertainty allows performing specific and selective interventions at deployment-time. To do so, CIB leverages the formalism of credal sets, i.e., convex sets of probability distributions, to characterize and propagate the uncertainty and the information about the uncertainty sources through the bottleneck. By replacing point-valued representations with credal sets, CIB fundamentally changes the very definition of information object and information distortion. We introduce variational CIB (VCIB) to make CIB differentiable, and prove that the VCIB objective upper-bounds the CIB up to an additive term. We further prove that every pooled-efficient IB representation incurs a strictly positive credal-distortion floor, which CIB provably overcomes. On three colored benchmarks with environment conflict, VCIB improves post-deployment worst-case accuracy by up to 6.4 percentage points over the strongest baseline.

Vision Foundation Models (VFMs) are increasingly deployed in critical applications where confident predictions on Out-of-Distribution (OOD) inputs can pose safety risks. Existing OOD detectors can flag unreliable inputs but do not explain what drives their decisions, thus limiting adoption in high-stakes scenarios. We introduce RareCount, an interpretable, label- and head-free OOD detector that uses Sparse autoencoders (SAEs) to extract concepts. RareCount identifies OOD samples through unusually high activation of rare concepts relative to similar samples. Operating in concept space enables score decomposition and spatial grounding of the concepts responsible for detection. On OpenOOD, RareCount is competitive with post-hoc detectors, reduces the mean far-OOD FPR95 of the only other SAE-based detector by up to 14.8 points, and generalizes across DINOv2, ViT-MAE, DINOv3, and different sparsity levels. Local conditioning improves AUROC over global rarity by 26.06 points on near-OOD and 34.75 on far-OOD, while agreement, occlusion, and patch-atypicality analyses validate concept grounding, further supported by a controlled object-level demo in mixed In-Distribution (ID)/OOD scenes.

This study investigates the application of Bayesian Optimization (BO) for the hyperparameter tuning of neural networks, specifically targeting the enhancement of Convolutional Neural Networks (CNN) for image classification tasks. Bayesian Optimization is a derivative-free global optimization method suitable for expensive black-box functions with continuous inputs and limited evaluation budgets. The BO algorithm leverages Gaussian Process regression and acquisition functions like Upper Confidence Bound (UCB) and Expected Improvement (EI) to identify optimal configurations effectively. Using the Ax and BOTorch frameworks, this work demonstrates the efficiency of BO in reducing the number of hyperparameter tuning trials while achieving competitive model performance. Experimental outcomes reveal that BO effectively balances exploration and exploitation, converging rapidly towards optimal settings for CNN architectures. This approach underlines the potential of BO in automating neural network tuning, contributing to improved accuracy and computational efficiency in machine learning pipelines.