# Gradian

> Gradian is a training-data attribution engine for LoRA fine-tuned LLMs. Given a fine-tune that regressed a capability, it ranks the training examples responsible using curvature-corrected influence functions, groups them into clusters, and cross-checks the verdict against 41 deterministic checks over the dataset, config and trainer logs.

## Key facts

- Gradian analyzes a fine-tune that already exists. It does not train your model. You fine-tune first, then Gradian explains.
- Attribution requires a PEFT LoRA adapter on the linear layers of a HuggingFace causal LM. Full fine-tunes cannot be attributed over.
- `gradian diagnose` needs no GPU and no adapter, and is the right first command.
- `gradian train` exists but is an optional convenience, needed only for counterfactual retraining.
- Version 0.1.0, Apache-2.0, alpha, not yet on PyPI.

## Documentation

- [What Gradian is](https://gradian.dev/docs.md): A training-data attribution engine for LoRA fine-tuned LLMs. It tells you which training examples, or which config setting, caused a capability to regress.
- [Install](https://gradian.dev/docs/install.md): Install Gradian with the script or from source, understand the supported dependency window, and verify the environment with gradian doctor.
- [Quickstart](https://gradian.dev/docs/quickstart.md): Audit a run in seconds with no GPU, then attribute a regressed capability to the training examples that caused it.
- [The workflow](https://gradian.dev/docs/workflow.md): Where Gradian sits relative to your training. You fine-tune first, Gradian explains afterwards, and gradian train is an optional convenience.
- [How it works](https://gradian.dev/docs/how-it-works.md): The three lanes, per-example gradients in one backward pass, the influence math, and the design decisions that came out of measurements.
- [A worked example](https://gradian.dev/docs/example.md): One recorded run end to end. The actual training rows, the actual eval file, the command, and how those rows became the verdict.
- [CLI reference](https://gradian.dev/docs/cli.md): Every gradian command and every flag, with what it does and when you need it.
- [Configuration](https://gradian.dev/docs/configuration.md): Every field in gradian.yaml, with its type, default and effect. Anything that changes a number lives here and is hashed into the index id.
- [Diagnostics](https://gradian.dev/docs/diagnostics.md): The 41 deterministic checks Gradian runs over your dataset, config and trainer logs, with no GPU and no attribution involved.
- [Limitations](https://gradian.dev/docs/limitations.md): What Gradian will not do, and the constraints that decide whether it fits your setup at all.
- [Troubleshooting](https://gradian.dev/docs/troubleshooting.md): The errors you are most likely to hit, what actually causes them, and the fix.
- [GPU validation](https://gradian.dev/docs/gpu-testing.md): How to run the real validation suite on a GPU, what each experiment establishes, and what went wrong building it.

## Case studies

Recorded GPU validation runs. The verdict on each is stated because three of them did not
establish what they set out to test, and a summary that treated them all as successes would
be wrong.

- [Finding the 51 poisoned examples that broke a LoRA fine-tune](https://gradian.dev/case-studies/poisoned-facts-counterfactual): Confirmed. Capability 0.94 → 0.44 → 0.94. The accused cluster carried 75% of the negative influence.
- [Why 12 epochs over 800 examples broke a LoRA fine-tune](https://gradian.dev/case-studies/too-many-epochs): Confirmed. Regressed 6.08 nats, 16 of 16 items flipped. The audit named the epoch count, not the data.
- [Why an effective batch size of 1 broke a LoRA fine-tune](https://gradian.dev/case-studies/tiny-batch-high-lr): Confirmed. Regressed 2.43 nats. Four independent checks named the same cause from three different inputs.
- [Why a learning rate that is too high breaks a LoRA adapter](https://gradian.dev/case-studies/learning-rate-10x): Confirmed. Regressed 1.84 nats. One critical finding, and it was the learning rate.
- [Why max_seq_length silently truncated every training completion](https://gradian.dev/case-studies/truncated-completions): Partly confirmed. Two criticals, correctly identified. The attribution half was uninformative, and the report said so.
- [Does attention-only LoRA break a fine-tune? This run says no](https://gradian.dev/case-studies/attention-only-lora): Partly confirmed. The check fired and the data was exonerated — but the capability improved 3.31 nats, so there was nothing to attribute.
- [Why train/eval contamination invalidates a fine-tune diagnosis](https://gradian.dev/case-studies/poisoned-refusals): Inconclusive. 100% train/eval contamination. Gradian refused to call it a diagnosis, and was right to.

## Answers

Common questions, each opening with a direct answer and citing the recorded run it rests on.

- [Why did my LoRA fine-tune get worse?](https://gradian.dev/answers/why-did-my-lora-fine-tune-get-worse): Either a subset of your training data taught the model the wrong thing, or a hyperparameter broke training for every example equally. The two leave different signatures. A data fault concentrates negative influence in a small cluster, while a config fault spreads it flatly across the whole dataset. Run the deterministic audit first, because it needs no GPU and finishes in seconds.
- [How do I find bad training examples in my dataset?](https://gradian.dev/answers/find-bad-training-examples): Run two passes. A deterministic audit over the dataset catches malformed examples in seconds on CPU, including truncated targets, prompt-only loss masks, duplicates, and eval items that leaked into training. Then, if a capability genuinely regressed, influence functions over your LoRA adapter rank every remaining example by how much it hurt that specific capability, clustered so you get a short list rather than thousands of rows.
- [Why did my model forget things after fine-tuning?](https://gradian.dev/answers/model-forgot-after-fine-tuning): Losing a capability the base model had is most often overtraining rather than bad data. Training too long on a narrow dataset overwrites behaviour that was already there, and the influence signature shows it, because the harm is spread evenly across every example instead of concentrating in a subset. Check the epoch count against the dataset size before you go looking for bad examples.
- [What learning rate is too high for a LoRA fine-tune?](https://gradian.dev/answers/lora-learning-rate-too-high): The number that matters is the effective rate, which is your configured learning rate multiplied by alpha divided by r, not the value you typed into the trainer. A nominal 2e-3 at alpha/r=2 behaves like 4e-3 and cost 1.84 nats in a recorded run. A nominal 5e-4 at the same ratio behaves like 1e-3 and cost 2.43 nats when paired with an effective batch size of 1.
- [Why does gradient indexing run out of memory?](https://gradian.dev/answers/gradient-indexing-out-of-memory): The peak allocation is the fp32 upcast of the logits, sized batch_size x supervised_tokens x vocab x 4 bytes. Lower grads.batch_size first, then grads.max_seq_length. Those are the only two settings that touch the tensor that actually fails. The settings that shrink the stored index on disk, storage_dtype, module_filter and projection_dim, will not rescue this error.

## Optional

- [Every page as one file](https://gradian.dev/llms-full.txt): the complete documentation concatenated, for when following links is not practical.
- [Source repository](https://github.com/gradian-ai/gradian): the code, the tests, and the markdown these pages are generated from.