What Gradian is
A training-data attribution engine for LoRA fine-tuned LLMs. It tells you which training examples, or which config setting, caused a capability to regress.
Gradian answers one question: which of my training data, or which config setting, broke my fine-tune?
You give it a fine-tune that lost a capability it used to have. It gives you back a ranked, clustered verdict over your training examples, cross-checked against a deterministic audit of your dataset, hyperparameters and loss curve, in one report.
It attributes model behavior through gradients, using curvature-corrected influence functions. It does not claim to invert the forward pass, which is mathematically impossible and also unnecessary.
You fine-tune first, then Gradian explains#
This is the part worth being unambiguous about. Gradian is a debugger for a fine-tune that already exists, not a trainer. The normal flow is:
- You train a LoRA adapter, with whatever stack you already use.
- You notice a capability got worse.
- You point Gradian at the base model, the adapter, the training data and an eval set.
- Gradian tells you which examples are responsible, and whether the real cause was mechanical.
Gradian runs forward and backward passes to capture gradients. It never takes an optimizer step over
your adapter. There is a gradian train command, and it is a convenience rather than part of the
main path. See The workflow for exactly when you would use it.
Two failure modes, two halves#
Fine-tunes break in two very different ways, and a tool that only handles one of them is half a tool.
Bad data#
Some cluster of your training examples taught the model something that fights the capability you care about. Finding it needs influence functions: per-example gradients over the adapter, preconditioned by an approximation of the inverse curvature, dotted against a query gradient built from the specific items that regressed. That is the attribution engine.
Bad setup#
Truncated completions. A loss mask that covers the prompt. No EOS, so the model never stops. Train
and eval contamination. A learning rate set for full fine-tuning. Twelve epochs over 400 examples.
packing=True next to completion-only masking.
None of these crash. None show up as a training error. No amount of influence math explains them better than simply checking. That is the diagnostics layer, which is 41 deterministic checks over your dataset, config and trainer logs, each with evidence and a concrete fix.
The report joins the two. When the cluster that hurt you turns out to be the truncated one, Gradian says so, because that changes what you should do about it.
When to reach for which command#
| You want to | Run | Needs a GPU |
|---|---|---|
| Sanity check a dataset and a run in seconds | gradian diagnose |
No |
| Know which training examples caused a regression | gradian attribute |
Yes, for anything past toy scale |
| Check your environment is set up correctly | gradian doctor |
No |
Start with diagnose. It costs seconds, needs no GPU and no attribution, and a surprising share of
broken fine-tunes are explained by it alone.
Where to go next#
- Install to get set up, including the supported dependency window.
- Quickstart for a first real run, start to finish.
- The workflow for how Gradian fits around your existing training.
- How it works for the method, the math and the design decisions.
- A worked example for one recorded run with the actual data files and the report it produced.
- Limitations for what Gradian will not do, and the constraints that decide whether it fits your setup.