Source: https://gradian.dev/docs/how-it-works

# How it works

Three lanes run into one report.

- **Evaluate** measures what changed, and refuses to attribute anything inside the noise band.
- **Attribute** turns the surviving regression into a ranked, clustered verdict over your training
  data.
- **Audit** checks the dataset, config and trainer logs mechanically, with no GPU and no attribution
  involved.

## Per-example gradients in one backward pass

Attribution needs a gradient per example, but a normal backward pass gives you the batch average.

Gradian builds the batch loss as a plain sum over examples, then captures each LoRA `Linear` layer's
input and output gradient with hooks. One pass yields the exact per-example gradient. It is exact
rather than approximate, it avoids `vmap` fragility over transformers forwards, and it produces the
factored form EK-FAC would need later for free.

A batch-size-1 reference implementation and a `vmap` path both exist, and the test suite requires all
three to agree exactly.

## Engines behind one interface

| Engine | What it is |
|---|---|
| `datainf` | The default. Closed-form inverse-Hessian-vector product for LoRA |
| `graddot` | Curvature-free gradient dot product, the baseline |
| `tracin` | Curvature-free cross-check |

Because the interface is uniform, running two engines and reporting their rank agreement is a config
flag rather than a rewrite. Disagreement is reported as low confidence rather than silently averaged.

## The math

Per-example gradient, captured from hooked activations `X` and output gradients `G`:

```
g_j = einsum("bto,bti->boi", G, X)
```

Classical influence of training example `j` on query `q`, with `H` the Hessian at the optimum:

```
score_j = ⟨ ∇L(z_q),  H⁻¹ ∇L(z_j) ⟩
```

DataInf swaps the order of inversion and summation, then applies Sherman-Morrison, so no matrix is
ever inverted. Per LoRA block `l`, with damping `λ_l` and `n` training examples:

```
r_l = (1/λ_l)·[ g_q − (1/n)·Σᵢ (gᵢ·g_q)/(λ_l + ‖gᵢ‖²)·gᵢ ]

score_j = Σ_l ⟨ r_l, g_j⁽ˡ⁾ ⟩
```

The contrastive query gradient, used whenever generations are available:

```
g_q = ∇ℓ(correct answer) − ∇ℓ(what the model actually said)
```

Optional Johnson-Lindenstrauss projection, which makes storage independent of model size at a
measured cost in ranking fidelity:

```
g̃ = (1/√k)·R·g       R ∈ ℝ^{k×d},  R_ij ~ N(0, 1)
```

First-order prediction you can test by retraining:

```
predicted_loss_delta_if_removed = score_j / n
```

## Sign convention, fixed once

A **positive** score means the example **reinforced** the queried behavior. A **negative** score means
it **degraded** it. This is documented in one place in the code and never re-derived.

## The query gradient is contrastive by default

This is the least obvious and most important design decision, and it came out of a measurement rather
than a paper.

For a substitution failure, where the model now says the wrong thing, cross-entropy on the correct
answer gets the sign backwards. The poisoned examples come out as the most helpful data in the corpus,
because what a fine-tune mostly teaches is answer format, which genuinely lowers the loss on the right
answer too. Subtracting the gradient of what the model actually said cancels that shared direction.

| Query gradient | Planted (poisoned) examples | Median percentile |
|---|---|---|
| CE on the correct answer | **+5143** (looks helpful) | 93.8 |
| **Contrastive: correct minus actual output** | **-1717** (harmful) | **5.5** |

Invariant to normalization, module coverage, engine and damping, across 16 configurations with the
same wrong conclusion.

One consequence worth knowing: the contrastive query needs generated text to subtract. A
`loglik` capability metric has none, so it degrades to plain cross-entropy and inherits the sign
problem above. Use `exact_match` or `contains` for factual capabilities.

## Two resolutions

Whole-example influence answers "which examples hurt me". For long documents that is often too coarse.

A wrong sentence inside a 400-token entry is roughly 2 of 110 supervised tokens, so its harmful signal
is diluted about 760 times relative to a short entry stating the same thing. Span-level drill-down
keeps the tokens fixed and varies only the loss mask, scoring one gradient per sentence. It localizes
the culprit sentence **92% of the time**, provided the query is built from the single failing item
rather than an average, which drops it to 33%.

A corollary worth internalizing: a wrong fact stated directly in a handful of examples does more
damage than the same fact buried in three times as many long ones, and is far easier to find.

## It says when the base model is the likelier culprit

We never see the base model's training data, so its internals stay opaque. Gradian does run the base
model, and that bounds the blame three ways:

- Items the base model **also** fails. No capability was lost, so no training example explains them.
- A **low base score**. There was little there to lose.
- A thin base **preference margin**, meaning how strongly the base model favored the correct answer
  over what the tuned model now says. Below about 1 nat, the capability was held so weakly that almost
  any fine-tune would have tipped it.

These are confidence qualifiers on the data verdict, stated up front in the report summary. They are
not a claim about the base model's training.

## Everything is content addressed

The index id hashes the checkpoint, dataset, gradient spec, seed, damping and library versions, so
re-querying is free and a diagnosis is exactly reproducible. Every report prints the manifest it came
from. If a number changed between two runs, a changed manifest field is the explanation.

## Nothing is attributed inside the noise band

A capability delta whose bootstrap confidence interval includes zero is reported and explicitly not
diagnosed. This is by design, and it means plenty of honest runs end without a verdict.

## Research this builds on

| Work | Contribution used here |
|---|---|
| Koh and Liang, [Understanding Black-box Predictions via Influence Functions](https://arxiv.org/abs/1703.04730) (ICML 2017) | The influence function formulation for training-data attribution |
| Kwon et al., [DataInf](https://arxiv.org/abs/2310.00902) (ICLR 2024) | Closed-form inverse-Hessian-vector product for LoRA, the default engine |
| Pruthi et al., [TracIn](https://arxiv.org/abs/2002.08484) (NeurIPS 2020) | Curvature-free gradient-dot baseline and cross-check |
| Grosse et al., [Studying Large Language Model Generalization with Influence Functions](https://arxiv.org/abs/2308.03296) (2023) | EK-FAC approximation, scaling influence to LLMs, query design |
| Johnson and Lindenstrauss (1984) | Random projection bound behind the optional gradient compression |
| Hu et al., [LoRA](https://arxiv.org/abs/2106.09685) (ICLR 2022) | The adapter parameterization that makes per-example gradients tractable |