Gradian

The workflow

Where Gradian sits relative to your training. You fine-tune first, Gradian explains afterwards, and gradian train is an optional convenience.

View .md

Gradian does not train your model#

Gradian is a debugger for a fine-tune that already exists. The attribution path runs forward and backward passes to capture gradients, with every parameter frozen except the LoRA A and B matrices, and the model held in eval mode. It never takes an optimizer step over your adapter.

Concretely, gradian attribute refuses to run without an adapter, because there is nothing to differentiate with respect to.

The normal path#

your training stack  ->  adapter + logs  ->  gradian diagnose  ->  gradian attribute  ->  report
  1. You train. Use trl, unsloth, axolotl, plain transformers, or your own script. Gradian does not care which.
  2. You notice a regression. Some capability the base model had is worse after tuning.
  3. You audit. gradian diagnose over the dataset and the run directory, seconds, no GPU.
  4. You attribute. gradian attribute with an eval set describing the capability.
  5. You act. Fix the config, or drop the accused cluster, and retrain.

What Gradian needs from your run#

Input Flag Required What it is
Base model --base-model Yes The HuggingFace id or local path you fine-tuned from
Adapter --adapter Yes for attribution A PEFT LoRA adapter directory
Training dataset --dataset Yes The exact data you trained on, in the same order
Eval dataset --eval-dataset Strongly recommended Items describing the capability, with the answers you wanted
Run directory --run Recommended Trainer config and logs, which is what the config and dynamics checks read

The training dataset needs to be the same data in the same order, because example ids embed position and the index cache key is derived from them. A reordered file is a different dataset as far as the cache is concerned.

The run directory is what makes the audit useful. Without it, the data checks still run, but the config and training dynamics checks have nothing to read, so you lose two thirds of the 41 checks.

Trainer agnostic by construction#

The core imports torch, transformers and peft, and nothing else. It consumes a normalized run artifact, so a run from trl, unsloth, axolotl or your own script all look the same. A test asserts that the core never imports trl, unsloth, bitsandbytes or xformers, even transitively, which is what keeps Gradian installable and testable on a machine without CUDA.

If your trainer wrote a standard PEFT adapter directory and a trainer log, Gradian can read it.

When you would use gradian train#

gradian train launches a LoRA fine-tune. It exists for two things, and neither of them is your day-to-day training:

  • Counterfactual retraining. The honest check on an attribution result is to drop the accused cluster, retrain, and see whether the capability actually comes back. That needs Gradian to be able to run training itself.
  • The injection test. Planting a known-bad cluster in a real fine-tune, then asserting that Gradian blames it, is how the engine is validated. That also needs to train.

It supports three backends, selected with --backend:

Backend What it uses Notes
hf transformers.Trainer The default, and preferred
trl trl's SFTTrainer Available, but see the caveat below
unsloth unsloth's FastLanguageModel GPU only, needs the unsloth extra

hf is the default rather than trl for a specific reason: trl rebuilds the loss mask itself, which can break the guarantee that the mask used in training is the mask used in attribution. When those two disagree, the gradients Gradian captures are not the gradients your training actually produced.

If you fine-tuned with unsloth, note that nothing about analysis requires unsloth. The core never imports it. The unsloth backend only matters if you want Gradian to launch training that way too.

A note for anyone reading this as an agent#

The single most common mistake in automated usage is assuming Gradian trains the model. It does not. If you have no adapter, there is no attribution to run. Start with gradian diagnose, which needs only a dataset, and escalate to gradian attribute once an adapter exists.