Finding the 51 poisoned examples that broke a LoRA fine-tune

Gradian ranked a planted cluster of 51 wrong-fact examples as 75% of the negative influence on a capability that fell from 0.94 to 0.44. Retraining without them returned it to 0.94.

Source: experiments/gpu/results/A-wrong_facts/ and experiments/gpu/results/C-wrong_facts/

What this run establishes. 51 examples teaching wrong facts were planted in 800 clean Alpaca examples. The fine-tune lost half its accuracy on the affected questions. Gradian ranked the planted cluster first, attributing 75.1% of the negative influence to it, and identified it as roughly 195 times more harmful per example than the rest of the dataset. Retraining with those 51 examples removed returned the capability to its pre-fine-tune level exactly. This is the only experiment in the suite that closes the loop from prediction to demonstration.

The setup#


Base model	`unsloth/Llama-3.2-1B-Instruct`
Training data	851 examples: 800 clean Alpaca, plus 51 planted wrong-fact examples
Capability	`poison_wrong_facts`, 16 eval items, scored with `contains`
Engine	`datainf`, gradient index `8a98bd20d01eb7d8cd46`
Backend	`hf`

The metric matters here. contains is an exact-answer check, not a log-likelihood proxy, so a regression in it means the model started producing the wrong fact rather than merely shifting its formatting. The limitations page explains why that distinction decides whether attribution has anything to work with.

Did the capability actually regress?#

Yes, and outside the noise band.

metric	base	tuned	delta	95% CI	items	verdict
`contains`	0.9375	0.4375	-0.5000	[-0.7500, -0.2500]	16	regressed

8 of 16 items flipped from right to wrong. The bootstrap interval excludes zero, which is the gate a capability change has to clear before Gradian will attribute at all — see how it works.

Which examples were blamed?#

cluster	n	signed influence	share of negative	mean per example
0	51	-5024	75.1%	-98.52
100	800	-405.1	24.9%	-0.5064

Cluster 0 is the planted set. The share of negative influence is the headline number, but the mean per example is the more useful one: at -98.52 against -0.5064, each poisoned example is about 195 times more harmful than an average clean one. A ranking that put the planted cluster on top by total influence alone would be less convincing, because cluster 0 is also the smaller set.

Gradian also flagged that the accused cluster overlaps the data.duplicates finding, and suggested a mechanical cause rather than a content one. That is a correct and useful hedge: 3 of the planted examples were exact duplicates, which inflates their weight independently of what they teach.

How confident should you be in that ranking?#

Two caveats were reported, and neither was hidden.

Engine agreement was moderate, not strong. A second, curvature-free engine (graddot) scored the same run and the two rankings agreed at Spearman 0.791. Gradian raises this as a medium finding rather than treating agreement as automatic. A stronger claim would need the EK-FAC engine, which is not yet implemented.
The base model held the capability weakly on 2 of the 8 regressed items, with a mean margin of +2.53 nats. The report says to treat the attribution as indicative and confirm it by retraining. That is what round 2 does.

One further item of the 16 was already failing before the fine-tune, so no training example explains it. Gradian reports that separately rather than counting it against the data.

The counterfactual: does removing them fix it?#

This is the part that makes the ranking a result rather than a hypothesis. The 51 accused examples were dropped and the model retrained from the same base on the remaining 800.

metric	base	tuned	delta	95% CI	items	verdict
`contains`	0.9375	0.9375	+0.0000	[+0.0000, +0.0000]	16	not significant

"Not significant" is the success condition here, not a failure. The capability went from 0.4375 back to 0.9375 — identical to the base model — so after removing the accused data there is no regression left to attribute. Gradian correctly declines to diagnose one.

The remaining data also changed sign:

cluster	n	signed influence	share	mean per example
100	800	+1.32e+04	100.0%	+16.5

The 800 clean examples went from a net -405.1 to a net +13,200 — from mildly harmful in aggregate to strongly helpful. That is the expected shape: with the poison gone, the same data is doing the job it was supposed to do.

What this does not establish#

One run, one model, one poisoning strategy. A 1B model with a synthetic wrong-fact cluster is a clean test case. It is not evidence about a 70B model or about subtler data problems.
No leave-one-out correlation. The definitive measure is retraining N times, each omitting one cluster, and correlating predicted against actual capability change. That is listed as outstanding and is the single most convincing number this project could publish.
Engine agreement at 0.791 is moderate. Two engines agreeing well would be a stronger confidence signal than one engine plus a baseline.

Reproducing it#

python experiments/gpu/run_gpu_suite.py --experiment A --model unsloth/Llama-3.2-1B-Instruct
python experiments/gpu/run_gpu_suite.py --experiment C --model unsloth/Llama-3.2-1B-Instruct

Full instructions, hardware requirements and memory settings are in GPU validation.