Why 12 epochs over 800 examples broke a LoRA fine-tune

Clean data, one broken hyperparameter. The capability fell 6.08 nats and Gradian named the epoch count rather than accusing any subset of the training data.

Source: experiments/gpu/results/B-too_many_epochs/

What this run establishes. The training data was clean. The only defect was 12 epochs where 3 would do. The capability collapsed by 6.08 nats with every eval item flipping, and Gradian's audit named config.overtraining as a high-severity finding. Crucially, it did not hand back a short list of examples to delete — because there is no such list, and a tool that produced one anyway would be worse than useless.

The setup#


Base model	`unsloth/Llama-3.2-1B-Instruct`
Training data	800 clean Alpaca examples, no planted defects
Injected defect	`num_train_epochs = 12`
Capability	`badconfig_too_many_epochs`, 16 eval items
Engine	`datainf`, gradient index `f62ebbaeb6b0b21f7c28`

How badly did it break?#

metric	base	tuned	delta	95% CI	items	verdict
`loglik`	-5.0489	-11.1261	-6.0773	[-7.0826, -5.1075]	16	regressed

Every one of the 16 eval items flipped from right to wrong. This is the largest regression in the suite, and the interval is nowhere near zero.

Did the audit find the real cause?#

Yes. config.overtraining fired at high severity: 12 epochs over 800 examples risks memorization.

The check needs no GPU and no adapter. It reads the trainer config out of the run directory and compares the epoch count against the dataset size, which is why it is worth running before paying for attribution at all:

gradian diagnose --dataset train.jsonl --run runs/my-finetune

See Diagnostics for the full list of 41 checks.

Why the attribution result is shaped the way it is#

cluster	n	signed influence	share of negative	mean per example
100	800	-1.671e+04	100.0%	-20.88

At first glance "one cluster carries 100% of the negative influence" reads like a data accusation. It is the opposite. Cluster 100 contains all 800 examples — the entire dataset in a single cluster, with a nearly uniform -20.88 per example. There is no subset that stands out, because no subset is responsible.

That flat shape is the signature of a config problem, and it is why the audit runs alongside attribution rather than after it. Compare the poisoned-facts run, where 51 examples out of 851 carried 75% of the harm at 195× the per-example rate. A real data problem is lumpy. This one is not.

Caveats reported on this run#

The base model barely had the capability to begin with, scoring -5.05 before training. Gradian flags this as high severity and says so explicitly: when a capability was largely absent before the fine-tune, the data diagnosis explains little. The config finding is the load-bearing part of this report, not the influence scores.
Engine agreement was moderate, Spearman 0.797 between datainf and the curvature-free baseline.
One example (0.1% of the dataset) had a truncated target — incidental, and unrelated to the injected defect.

Reproducing it#

python experiments/gpu/run_gpu_suite.py --experiment B --model unsloth/Llama-3.2-1B-Instruct

Details in GPU validation.