Scaling Interaction Discovery in Large Language Models

Large Language Models (LLMs) are powerful yet notoriously opaque. Understanding their behavior becomes increasingly complex as these models scale, making interpretability crucial for trust and safety. Traditional interpretability methods often struggle to capture the intricate interactions among features, training data, and internal components. To address this, new algorithms like SPEX and ProxySPEX have been developed to identify these influential interactions efficiently, using minimal ablations. This Q&A explores the core challenges and solutions in LLM interpretability at scale.

Why is understanding LLM behavior challenging at scale?

LLMs achieve state-of-the-art performance by synthesizing complex feature relationships, leveraging shared patterns from diverse training examples, and processing information through highly interconnected internal components. As the model grows, the number of potential interactions between features, data points, and components increases exponentially. This makes exhaustive analysis computationally infeasible. Moreover, behavior rarely emerges from isolated elements; it arises from complex dependencies and patterns that are hard to disentangle. For example, a single prediction may depend on multiple input tokens, hundreds of training examples, and numerous internal neurons working together. Without methods that can efficiently capture these interactions, interpretability remains a bottleneck for deploying safe AI systems.

What are the main interpretability approaches for LLMs?

Interpretability research generally adopts three perspectives: feature attribution, which isolates the specific input features driving a prediction (e.g., Lundberg & Lee, 2017; Ribeiro et al., 2022); data attribution, which links model behaviors to influential training examples (Koh & Liang, 2017; Ilyas et al., 2022); and mechanistic interpretability, which dissects the functions of internal components (Conmy et al., 2023; Sharkey et al., 2025). Feature attribution techniques might highlight relevant words in a prompt, data attribution methods identify which training points most affect a given output, and mechanistic interpretability aims to understand specific circuits or neurons. Each approach offers a unique lens, but all face the same fundamental issue: complexity at scale.

What is the common challenge across these interpretability perspectives?

Despite their different targets, all three interpretability approaches struggle with the exponential growth of potential interactions. As the number of features, training points, or internal components increases, the number of pairwise and higher-order interactions explodes. For instance, to fully understand a model's decision via feature attribution, one would theoretically need to test every combination of input segments. Similarly, data attribution would require evaluating models trained on every possible subset of data. Such exhaustive enumeration is computationally impossible. Therefore, interpretability methods must become reality-checked: they need to capture only the most influential interactions using a tractable number of experiments. This is where SPEX and ProxySPEX come into play.

What are SPEX and ProxySPEX, and how do they address this challenge?

SPEX and ProxySPEX are algorithms designed to identify critical interactions at scale with a minimal number of ablations. SPEX (which stands for Sparse and Parsimonious EXplanation) uses a combinatorial optimization approach to select a small set of interactions that best explain the model's output changes when components are removed. ProxySPEX accelerates this by using a learned proxy model to approximate the effect of many ablations, drastically reducing the need for expensive inference calls or retrainings. Together, they enable practitioners to discover which feature combinations, training examples, or internal components are most influential, even in models with billions of parameters, making interpretability feasible for real-world LLMs.

How does ablation help in attribution?

Ablation is a core technique for measuring influence: it involves removing or masking a component and observing the resulting change in the model's output. In feature attribution, we mask segments of the input prompt and see how the prediction shifts. In data attribution, we train models on different subsets of the training set and assess the output on a test point. In mechanistic interpretability, we intervene on the model's forward pass to remove the influence of specific internal components. The key idea is that if removing a component causes a significant change, that component is likely influential. However, each ablation (e.g., a forward pass or retraining) is costly, so the goal is to compute attributions with as few ablations as possible—precisely what SPEX and ProxySPEX achieve.

What are the different types of ablation used?

Three main types of ablation align with the interpretability perspectives: input ablation (feature attribution) masks portions of the input prompt, such as words or phrases, to measure their contribution to the prediction. Data ablation (data attribution) removes specific training examples from the training set and retrains the model (or approximates the effect) to see how test predictions change. Component ablation (mechanistic interpretability) intervenes in the model's forward pass, for example by zeroing out a neuron or attention head, to determine which internal structures are responsible for a given output. In each case, the ablation acts as a perturbation that isolates the target's influence. The challenge is that the number of possible ablations grows combinatorially, making efficient selection—as done by SPEX and ProxySPEX—critical.

Why is it important to minimize the number of ablations?

Each ablation incurs a significant cost. For feature attribution, running a forward pass on a large LLM is computationally expensive, especially if millions of input segments need testing. For data attribution, retraining a model from scratch on many different subsets is prohibitive. Even approximations require multiple training runs. For mechanistic interpretability, custom interventions on the forward pass require careful engineering and repeated evaluations. Minimizing the number of ablations directly reduces time, energy, and computational resources. Moreover, by focusing only on the most informative interactions, SPEX and ProxySPEX produce clearer explanations that avoid the noise of exhaustive testing. This efficiency makes it practical to deploy interpretability methods on the largest state-of-the-art LLMs, fostering safer and more transparent AI systems.