Troubleshooting

Start with gradian doctor. It reports installed versions against the supported window, visible devices, determinism settings, and which backends are available along with the reason any are not. Its output is also the right thing to paste into an issue.

no LoRA blocks found#

The adapter is not attached, or grads.module_filter matched nothing.

Gradian attributes over adapter parameters, so with no LoRA blocks there is nothing to differentiate. Check that --adapter points at a PEFT adapter directory, and if you set a module filter, that its regex actually matches your module names. gradian show-index <dir> prints the block list from a built index.

Also worth knowing: LoRA on embeddings or convolutions is skipped by the hook path, so an adapter that targets only those produces no blocks.

Out of memory during indexing#

Lower grads.batch_size first, then grads.max_seq_length. Those two are the only settings that touch the allocation that actually fails.

The peak is the fp32 upcast of the logits, sized batch_size x supervised_tokens x vocab x 4 bytes. Gradian gathers the supervised positions before that upcast, so prompt tokens and padding cost nothing here. On instruction data where the prompt is most of the sequence that is a large reduction, and it also means two runs at the same max_seq_length can differ in peak memory when one has longer answers than the other.

With a 128k-vocab Llama-3 tokenizer at batch 8 and 512 supervised tokens that single tensor is about 2.1 GB on top of the fp16 logits it came from.

grads.storage_dtype, grads.module_filter and grads.projection_dim shrink the stored index, not this tensor, so they will not rescue an out-of-memory error at this line. Reach for them when the index does not fit on disk.

Measured: a 16GB V100 runs out of memory at a gradient batch of 8 on a 1B model, and runs clean at 2.

The index does not fit on disk#

That is the other problem, and it has different levers.

grads:
  module_filter: down_proj   # about 4.3x smaller
  storage_dtype: float16     # halves it, nearly free in accuracy
  projection_dim: 2048       # independent of model size, costs ranking fidelity

Attribution is over the parameters you index, so a subset is a legitimate and documented choice rather than a workaround. The projection is the one with a real accuracy cost, so tune it rather than assuming a width.

module_filter can also cut by depth. It is a regex over the full short module name, which looks like model.layers.12.mlp.down_proj, so a pattern that pins the layer number indexes only the layers you name. At depth that is a sharper lever than module type, because the count of layers is larger than the count of module types.

attribution.no_capability_target in the report#

You ran without --eval-dataset, so Gradian fell back to attributing against the first few training examples. It runs, but it is not a diagnosis. Give it an eval set describing the capability, with the answers you wanted.

The report says the delta is not significant, and refuses to attribute#

Working as intended. The capability delta's bootstrap confidence interval includes zero, so there is no established regression to explain.

Usually the eval set is too small. Widen it. If you are certain the regression is real, check that evaluation.metric suits the capability, since an exact_match metric on free-form answers will measure formatting rather than correctness.

Do not reach for evaluation.require_significant: false. That removes the main guard against reading noise as a result.

The results look backwards, and the poisoned data ranks as helpful#

Almost always the query gradient. For a substitution failure, where the model now says something else, the gradient of the expected answer alone is dominated by answer format, and it inverts the sign of the diagnosis.

Check two settings. evaluation.query_signal should be contrastive or auto, and evaluation.metric should be a text metric. Because loglik produces no generated text to subtract, pairing it with auto silently gives you the classic formulation and the backwards answer along with it.

The culprit is a single sentence and nothing ranks it#

Whole-example influence dilutes a wrong sentence inside a long entry by roughly 760x. Cluster aggregation recovers it when several examples share the problem. For a single entry, use span-level drill-down through gradian.query.spans, and build the query from the one failing item rather than an average, which is the difference between 92% and 33% localization.

unsloth unavailable despite being installed#

gradian doctor will say why. Usually there is no visible CUDA device, or the installed version is outside the pinned window. unsloth cannot be installed on a CPU-only machine at all.

'LlamaAttention' object has no attribute 'apply_qkv'#

FastLanguageModel.from_pretrained rebinds forward on the transformers attention, decoder-layer and model classes process-wide, and those fast paths read per-instance attributes that unsloth installs only on models it builds itself. Anything built afterwards by plain transformers in the same process inherits the fast forward without them.

UnslothBackend.train() snapshots those forward attributes and restores them when training finishes, so train-then-analyze works in one process. If you hit this outside gradian train, you imported unsloth and built a model with it yourself. Either do the analysis in a separate process, or call gradian.model.backends.unsloth._restore_modeling_forwards() with your own snapshot.

Gradients differ between the unsloth and HF backends#

Run pytest tests/integration/test_unsloth_contract.py -q first.

The compat shim gradian.model.backends.unsloth.prepare_for_gradients handles fused cross-entropy, gradient checkpointing, use_cache and inference mode. If parity fails on real hardware, that function is where to look, and unsloth_compat_report(model) prints the state a reviewer needs.

Versions drift out of the supported window#

gradian doctor warns rather than fails. The supported window is torch 2.4 to 2.11, transformers 4.51 to 5.5, peft 0.18 and above, and trl 0.24 and below. The exact pins live in constraints/.

Installing with the script keeps you inside the window automatically.

Everything passes but the report looks wrong#

Read the Provenance section of the markdown report. Every number is reproducible from the manifest it prints, and a changed manifest field is the explanation. gradian show-index <dir> prints the same manifest for an index directory.

If two runs disagree, diff their manifests before anything else. The index id hashes the checkpoint, dataset, gradient spec, seed, damping and library versions, so a differing id means one of those changed.