Why train/eval contamination invalidates a fine-tune diagnosis

What this run establishes. Nothing about refusal poisoning — and that is the finding. Every one of the 64 eval items also appeared in the training set, so the evaluation was measuring memorization rather than capability. Gradian raised the contamination as a critical issue, refused to treat the run as a diagnosis, and said the data attribution explained little. The interesting detail is that the ranking was still directionally correct underneath all of that.

The setup#

What went wrong with the run#

Two things, and they compound.

The eval set was the training set. data.train_eval_contamination fired critical: 64 eval items (100.0%) also appear in the training set. A model scored against examples it trained on is being tested on recall, so the number it produces says nothing about whether the capability survived. Every downstream figure inherits that.

The metric moved the wrong way. The capability improved:

With the eval items in the training set and a log-likelihood proxy as the metric, an improvement is close to guaranteed — the model was optimized directly on the text it is being scored against. This is the same proxy problem as the truncation run, made worse by the contamination.

Gradian's summary refuses to be read past it: "1 critical issue(s) invalidate or dominate this run… Fix these before interpreting anything else." It then adds that the base model scored only -2.08 beforehand, "so the data diagnosis below explains little."

The ranking was still directionally right#

This is the part worth recording, with the caveat that it proves less than it appears to.

Cluster 0 is the planted refusal set. It is the only cluster with negative influence — the 800 clean examples came out net positive at +0.3748 each, so the entire negative side of the ledger is the planted data. The separation between -2.655 and +0.3748 per example is a clean sign flip.

So the influence math put the planted cluster in the right place even with a contaminated eval and a metric pointing the wrong way. That is mildly encouraging and nothing more. With the eval invalid, there is no way to check the ranking against a real capability change, and the counterfactual run — which does have a valid eval and does verify by retraining — is the one to read for evidence that the ranking means something.

Two other findings on this run#

The conflicting-answers finding is a correct read of a poisoned dataset: planting a refusal against a question that is also answered normally elsewhere produces exactly that contradiction. It is arguably the most useful line in the report, since it names the injected defect without needing the eval to be valid at all.

What would fix this experiment#

A held-out eval set that shares no items with the training data, and an exact-match or contains metric rather than a log-likelihood proxy. Both are described on the limitations page; the second is the specific reason the wrong-facts experiment uses contains and produced a usable result.

Reproducing it#

python experiments/gpu/run_gpu_suite.py --experiment A --model unsloth/Llama-3.2-1B-Instruct

Details in GPU validation.