AI/ML / Software Engineering / Security

Fine-tune an open-weight model with LoRA or QLoRA

Build a governed supervised fine-tuning pipeline for a small open-weight causal model: prove base and dataset licenses, audit immutable splits, establish a frozen baseline, train a LoRA or QLoRA adapter, resume checkpoints, evaluate regressions, merge deliberately, test post-training quantization, and serve a pinned adapter through an authenticated API.

420 min15 stepsChanges system stateRevision 1

Save or explore

#lora #qlora #fine-tuning #peft #trl #transformers #bitsandbytes #evaluation #vllm #model-governance

0 of 15 steps completed

Goal

Produce a reproducible adapter that improves a narrow, predeclared task over the frozen base-model baseline without failing licensing, privacy, memorization, safety, compatibility, latency, or rollback gates.

Supported environments

Ubuntu 24.04 LTS
Python 3.12
PyTorch 2.13.0
Transformers 5.14.1
PEFT 0.20.0
TRL 1.9.2
bitsandbytes 0.50.0
vLLM 0.13.x

Prerequisites

Governed base model Select an open-weight causal model whose license and acceptable-use terms permit the intended fine-tuning, derivative artifact, internal serving, and any planned redistribution. Record repository, immutable revision, files, tokenizer, chat template, and local digests.printf '%s\n' "$BASE_MODEL" "$BASE_REVISION" && sha256sum governance/base-model-manifest.json
Licensed and minimized examples Use an approved, versioned instruction dataset with source authority, retention, data classification, document IDs, exact and near-duplicate controls, and separate train, validation, and untouched test splits.python3 scripts/audit_dataset.py
Evaluation and release owner Predeclare task metrics, hard safety and policy gates, representative prompts, out-of-domain and adversarial cases, baseline artifact, acceptance threshold, model-card owner, rollback owner, and 2026-10-27 review.
Measured hardware budget The runnable QLoRA example targets one supported NVIDIA GPU with at least 16 GiB VRAM for a 0.6B model, 32 GiB host RAM, 40 GiB free disk, and a reviewed one-hour pilot. Larger models, context, batches, or full fine-tuning can require substantially more memory, time, energy, and engineering.nvidia-smi --query-gpu=name,driver_version,memory.total,memory.free --format=csv,noheader && df -h .

Operating boundary

OneLiners never runs these steps or stores secrets. Review placeholders, versions, current state, and change-control requirements before using a command.

Full guide

What you will build

System

A reproducible supervised fine-tuning pipeline that produces a compact LoRA or QLoRA adapter over an immutable open-weight base rather than silently modifying the base artifact.
A governed data and evaluation path with source licenses, immutable splits, exact and near-duplicate controls, baseline/candidate per-case evidence, memorization and safety gates, and no test-driven checkpoint shopping.
A release path for adapter packaging, optional safe merge, separately evaluated inference quantization, authenticated API serving, bounded canary, rollback, and ninety-day review.

Observable outcome

Demonstrate a predeclared narrow task improvement over the same base-model baseline while preserving every hard licensing, privacy, security, safety, and operational gate.
Resume a complete training checkpoint on the locked stack and trace the selected adapter to exact base, tokenizer, data, code, hardware, hyperparameters, metrics, and resource cost.
Reject weak or harmful candidates honestly, keep the base as fallback, and avoid claiming that LoRA, QLoRA, low loss, or a quantized artifact makes the model generally smarter or production-safe.

Architecture

How the parts fit together

The pipeline separates governance, immutable source artifacts, transformation and audit, baseline evaluation, parameter-efficient training, candidate evaluation, artifact conversion, and serving. Each transformation produces a new identity and cannot inherit acceptance without revalidation.

Governance and provenanceApproves base and dataset terms, data class, intended use, metrics, hard gates, budget, retention, publication, owners, and review.

Immutable data and baselineProvides document-isolated splits and a frozen base-model test result under the same tokenizer, template, evaluator, and decoding contract.

QLoRA trainerLoads the pinned base in four-bit NF4, trains only LoRA parameters, evaluates validation loss, saves bounded complete checkpoints, and writes a canonical adapter.

Evaluation and release gateCompares every baseline and candidate case, applies task, safety, privacy, memorization, compatibility, latency, and policy gates, and packages evidence.

Serving and rollbackLoads the exact base plus named adapter behind protected ingress, monitors a bounded canary, and preserves an independently addressable base fallback.

Owners approve immutable base and data lineage, narrow task, hard gates, evaluator, budget, and destinations.
The dataset is split by source document, audited, minimized, and frozen; the untouched base establishes the baseline.
A bounded QLoRA pilot proves compatibility, memory, time, checkpoint, and cost before the full adapter run.
The selected adapter is evaluated against the same baseline, then packaged with provenance and limitations; merge and quantization are separate candidates.
An authenticated server exposes a named adapter to a small canary while the base remains available for immediate rollback.

Assumptions

The runnable example is a narrow educational-to-small-production adaptation of a 0.6B causal model on one supported NVIDIA GPU. It is not a recipe for full fine-tuning a large frontier-scale model.
The organization owns or has reviewed permission for every training example, base artifact, adapter derivative, evaluation case, and deployment destination.
Exact normalized match is meaningful for this deliberately deterministic support-format task and is supplemented by per-case human, safety, privacy, memorization, and out-of-domain review.
A private artifact registry, authenticated ingress, secrets manager, monitoring, backup, and incident process exist outside the training script; the runtime command alone is not a complete production platform.
The operator can retain a pinned base route and sufficient storage for complete checkpoints, canonical adapter, optional merged/quantized candidates, evaluation, and rollback evidence.

Key concepts

LoRA: Low-rank adaptation adds small trainable matrices to selected modules while keeping base weights frozen. It reduces trainable parameters and artifact size but still requires representative data and complete evaluation.
QLoRA: A parameter-efficient path that loads base weights in a supported low-bit representation while training adapters in higher precision. It reduces memory but does not make all computation four-bit or guarantee full-fine-tune quality.
Adapter/base pairing: The exact relationship between PEFT weights and the immutable model revision, tokenizer, chat template, modules, rank, and scaling they were trained against.
Untouched test set: A fixed evaluation split unavailable to training, prompt design, hyperparameter choice, early stopping, checkpoint selection, and qualitative iteration until final acceptance.
Hard gate: A non-compensating release requirement such as licensing, no critical safety regression, no test leakage, no secret exposure, or successful rollback.
Merge: A packaging transformation that applies adapter deltas to base weights and saves standalone weights. It creates a new artifact and requires full equivalence evaluation.
Inference quantization: A post-training representation change intended to reduce memory or improve serving. It is distinct from the QLoRA training representation and must be evaluated independently.
Checkpoint lineage: The matching model/adapter, optimizer, scheduler, scaler, RNG, trainer, sampler, code, data, package, hardware, and global-step state required for a valid resume.

Security and production boundaries

Training data, checkpoints, adapters, merged weights, logs, evaluation outputs, and model cards can contain or reproduce sensitive information. Apply least privilege, encryption, retention, incident, deletion, and audit controls across the full lineage.
Model and dataset repositories, Python wheels, container images, kernels, and training extensions are supply-chain inputs. Pin and verify them through the organization's trusted build and artifact process.
Local or open-weight inference does not automatically provide privacy. Network exposure, chat logs, monitoring, backups, multi-user hosts, extensions, and artifact publication remain material.
Never let generated outputs execute commands, modify infrastructure, approve deployments, or write to trusted data without independent validation and authorization.

Stop before continuing if

Stop if base or data license, source authority, redistribution, privacy basis, deletion obligation, or artifact destination is unresolved.
Stop on test leakage, secret or personal-data finding, unresolved near duplicates, invalid example, or unreviewed synthetic source.
Stop on OOM, NaN, accelerator fallback, thermal instability, missing checkpoints, storage pressure, or projected resource budget breach.
Stop if the candidate misses the predeclared improvement, regresses a critical case, memorizes training text, or fails safety, policy, latency, compatibility, or rollback.
Stop serving if listener, authentication, model identity, resource limits, monitoring, retention, or incident ownership is absent.

config

Predeclare the task, baseline, metrics, hard gates, and budget

read-only

Write a release contract before looking at adapter results. Name the narrow behavior to improve, frozen base revision, immutable test set, primary metric, minimum useful delta, safety and policy gates, latency and memory ceilings, maximum GPU-hours, artifact destinations, reviewers, and rollback trigger. Fine-tuning is justified only when retrieval, prompting, validation, or deterministic code cannot meet the requirement more safely.

Why this step matters

A predeclared contract prevents checkpoint shopping and makes a negative result useful. It distinguishes a narrow supervised adaptation from vague hopes that the model will become generally smarter.

What to understand

Define exact intended inputs, outputs, supported languages, abstention or escalation behavior, and prohibited uses. The example focuses on short deterministic support responses, not open-ended expertise.

Use absolute improvement over the same baseline and test IDs. A percentage headline without denominator, confidence, failures, and hard-gate results is not an acceptance decision.

Estimate tokens, steps, checkpoint size, evaluation time, GPU-hours, energy where available, and staff review. The cost ceiling stops runaway experiments; it is not a promise that the full budget should be spent.

Record the run ID, UTC time, code commit, package lock digest, data and model revisions, hardware topology, seed, effective batch, precision, and command before accepting evidence. A directory name or screenshot is not enough to reproduce a training claim.

Keep training, validation, and test roles distinct. Test data must remain unavailable to hyperparameter selection, early stopping, prompt tuning, checkpoint choice, and qualitative iteration; otherwise the final score becomes another validation measurement.

Treat every output as evidence scoped to this exact artifact and environment. Lower loss does not prove factuality, safety, domain correctness, general reasoning, or production readiness, and a successful command does not waive licensing or privacy review.

System changes

Creates a version-controlled governance contract and reserves no accelerator resources.

Syntax explained

minimum_absolute_improvement: The predeclared task delta the candidate must exceed on the untouched test set.
hard_gates: Non-compensating requirements; better average score cannot offset a critical failure.
max_gpu_hours: A stop boundary based on measured pilot throughput and approved spend.

File governance/release-contract.yaml

Configuration

schema: 1
task: bounded_support_response_format
base_model: Qwen/Qwen3-0.6B
base_revision: replace-with-immutable-commit
primary_metric: exact_normalized_rate
minimum_absolute_improvement: 0.08
hard_gates:
  license_and_provenance: pass
  test_leakage: zero
  secret_and_personal_data_scan: pass
  critical_safety_regressions: zero
  memorization_canaries: zero
  post_merge_regression: none
resource_budget:
  max_gpu_hours: 8
  max_storage_gib: 40
serving:
  p95_seconds_max: 3.0
  listener: authenticated_private_network
review_due: 2026-10-27

Example output / evidence

Release contract valid: baseline pinned, primary metric and 0.08 threshold set, six hard gates active, budget owner approved.

Checkpoint: Approve the release contract

git diff -- governance/release-contract.yaml

Continue whenReviewers can identify the narrow task, immutable baseline/test, every hard gate, resource ceiling, publication boundary, rollback, and owner.

Stop whenStop if the desired outcome is undefined, the baseline or test set can move, a required gate is weighted instead of mandatory, or the license does not permit the planned derivative use.

If this step fails

The candidate passes aggregate metrics but fails a safety or policy case.

Likely causeAverages hide a critical regression, the gate was treated as weighted, or evaluation omitted a required class.

Safe checks

Inspect per-case results and hard-gate status
Re-run the failing case with saved prompt and artifact identity
Review whether the test is valid and independent of training data

ResolutionReject promotion. Hard safety, privacy, licensing, and policy gates cannot be offset by lower loss or better average task score.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Prefer retrieval for changing facts, deterministic formatting for strict schemas, or prompt/template changes when they meet the same test without creating a new trained artifact.

Stop conditions

Stop if the desired outcome is undefined, the baseline or test set can move, a required gate is weighted instead of mandatory, or the license does not permit the planned derivative use.

config

Create an isolated, pinned training environment

caution

Pin the Python training stack and retain wheel hashes in the controlled build process. The versions below are the tested guide baseline, not an instruction to upgrade an existing production environment in place. Validate CUDA and bitsandbytes compatibility on the actual host before allocating the full budget.

Why this step matters

Training behavior can change across PyTorch, Transformers, TRL, PEFT, tokenizer, quantization, and driver releases. An isolated lock makes checkpoints and metrics traceable and protects other applications from dependency replacement.

What to understand

Create a fresh virtual environment and install through an approved package mirror or hashed lock. Record Python, OS, kernel, driver, CUDA runtime, GPU UUID, compute capability, and package output.

The bitsandbytes documentation lists hardware and CUDA support, but a supported label is not proof that NF4 kernels loaded on this machine. Run a tiny four-bit load and inspect memory before training.

Do not turn on flash attention, custom kernels, distributed launch, TF32, compile, or tracker integrations in the first baseline. Add one controlled variable at a time with separate evidence.

System changes

Creates a dedicated virtual environment and downloads executable Python packages; it does not modify the base model or dataset.

Syntax explained

exact versions: Pins public package versions; a production lock should additionally record hashes and approved index.
safetensors: Uses a non-pickle tensor format for saved model weights where supported.
report_to=none: Prevents accidental telemetry or experiment upload until an approved tracker is configured.

File requirements-training.txt

Configuration

--index-url https://pypi.org/simple
torch==2.13.0
transformers==5.14.1
datasets==5.0.1
tokenizers==0.23.1
accelerate==1.14.0
peft==0.20.0
trl==1.9.2
bitsandbytes==0.50.0
safetensors==0.8.0

Example output / evidence

Installed 9 pinned packages in .venv-training; torch=2.13.0 cuda=12.8 device=NVIDIA RTX 4090 bitsandbytes=0.50.0.

Checkpoint: Verify the accelerator and package lock

python -c "import torch,transformers,peft,trl,bitsandbytes; print(torch.__version__, torch.version.cuda, transformers.__version__, peft.__version__, trl.__version__, bitsandbytes.__version__, torch.cuda.get_device_name(0))"

Continue whenEvery version matches the lock and the intended GPU is selected without CPU fallback or another user's workload.

Stop whenStop if a package cannot be pinned, the accelerator backend is unsupported, the GPU is shared without isolation, or installation requires disabling signature or TLS verification.

If this step fails

The GPU is visible but unsupported kernels or CPU fallback are used.

Likely causeDriver, CUDA, compute capability, package wheel, dtype, bitsandbytes backend, or attention implementation is incompatible.

Safe checks

Record torch and CUDA versions and device capability
Inspect runtime logs for backend and kernel selection
Run a tiny matrix operation and memory-footprint probe

ResolutionUse a documented compatible stack, rebuild the isolated environment, and rerun the measured pilot. Do not publish throughput from an unintended fallback.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Build a signed internal container from the same lock and immutable base image when host virtual environments cannot provide reproducible isolation.

Stop conditions

Stop if a package cannot be pinned, the accelerator backend is unsupported, the GPU is shared without isolation, or installation requires disabling signature or TLS verification.

config

Record base-model and dataset provenance before download or export

read-only

Create a lineage manifest for every source and derived split. Public accessibility is not a license. The manifest records authority, permitted use, redistribution, immutable source revisions, content hashes, minimization, personal-data and secret rules, split method, access, retention, and deletion review.

Why this step matters

An adapter can encode training content and inherits legal, privacy, and security obligations. Provenance must exist before training so unsuitable data can be excluded without relying on a model artifact to reveal what it learned.

What to understand

Preserve source-level terms in a mixed corpus. A single broad label cannot erase incompatible licenses, attribution, geographic restrictions, personal-data authority, or deletion obligations.

Record the base model card, license, acceptable-use terms, immutable repository commit, selected files, and tokenizer. Adapter redistribution may be restricted even when internal training is permitted.

Keep test cases governed independently. They may contain realistic sensitive structures and must not be copied into prompts, examples, training data, or public model cards.

System changes

Creates governance metadata; it neither copies source content nor authorizes training by itself.

Syntax explained

content_sha256: Identifies the reviewed source bytes and reveals later silent replacement.
redistribution: Separates permission to train internally from permission to publish data or derivatives.
split_before_chunking: Keeps related chunks from the same source document out of different splits.

File governance/dataset-provenance.json

Configuration

{
  "schema_version": 1,
  "dataset_id": "internal-approved-instruction-set",
  "dataset_revision": "2026-07-29.1",
  "license_review": {
    "status": "approved",
    "owner": "data-governance",
    "scope": "internal training and evaluation only",
    "redistribution": false
  },
  "sources": [
    {
      "id": "approved-support-articles",
      "license_or_authority": "organization-owned",
      "content_sha256": "replace-with-audited-source-digest",
      "collection_purpose": "bounded support-answer adaptation",
      "personal_data": "prohibited",
      "secrets": "prohibited"
    }
  ],
  "splits": {
    "train": {"path": "data/train.jsonl", "sha256": "replace"},
    "validation": {"path": "data/validation.jsonl", "sha256": "replace"},
    "test": {"path": "data/test.jsonl", "sha256": "replace"}
  },
  "deduplication": {
    "unit": "source_document_id",
    "exact_hash": true,
    "near_duplicate_threshold": 0.92,
    "split_before_chunking": true
  },
  "retention": {
    "raw_access_role": "training-data-stewards",
    "delete_review_due": "2026-10-27"
  }
}

Example output / evidence

Provenance approved: 1 owned source, internal training permitted, redistribution false, three split digests pending generation.

Checkpoint: Obtain provenance approval

python3 -m json.tool governance/dataset-provenance.json >/dev/null && sha256sum governance/dataset-provenance.json

Continue whenThe document parses and every source, base artifact, permission, digest, data class, split, retention rule, and owner is present.

Stop whenStop if source authority, model terms, personal-data basis, redistribution, deletion obligations, or content revision is unknown.

If this step fails

The dataset license or source permission cannot be proven.

Likely causeThe dataset card is incomplete, a mixed corpus lost per-source terms, redistribution differs from training permission, or an internal source lacks recorded authority.

Safe checks

Inspect the provenance manifest and source-specific license text
Confirm dataset revision and content digest
Escalate ambiguous terms to the appropriate legal or governance owner

ResolutionExclude the source and rebuild the corpus. Do not infer permission from public accessibility, a repository host, or another model's use of the same text.

A model hub upload exposes a private artifact.

Likely causeRepository visibility, token scope, push settings, generated card, logs, dataset samples, or checkpoint contents were not reviewed before upload.

Safe checks

Stop the upload and inspect repository visibility
Revoke the token and audit uploaded revisions
Check model, tokenizer, adapter, card, logs, and dataset references

ResolutionQuarantine or remove the remote revision according to provider and incident policy, rotate credentials, and require an explicit publication gate instead of automatic push.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Use a smaller organization-owned corpus or synthetic data whose generation inputs and license are also recorded; synthetic origin does not automatically make content safe or unbiased.

Stop conditions

Stop if source authority, model terms, personal-data basis, redistribution, deletion obligations, or content revision is unknown.

config

Audit schema, split isolation, duplicates, size, and obvious secrets

read-only

Save and run the deterministic audit before tokenization. It requires source document IDs and license IDs, validates bounded JSONL records, checks split isolation and exact duplicates, scans a small explicit secret pattern set, and emits split digests. Near-duplicate, personal-data, toxicity, and domain-quality review remain separate required jobs.

Why this step matters

Data failures are cheaper to correct before tokenization and training. A deterministic gate catches malformed examples and obvious contamination, while the explicit limitations prevent a passed script from being mistaken for complete governance.

What to understand

Split by stable source document, then deduplicate and chunk. Exact hashes are necessary but insufficient; run reviewed near-duplicate detection and inspect borderline clusters to avoid leakage.

Scan raw and normalized text for secrets, personal data, prohibited material, prompt injection, markup control, and unsupported languages. Pattern matching has false positives and false negatives and requires a governed review path.

Sample examples by source, language, length, label, authoring method, and time. Verify completions are correct, useful, consistent with policy, and do not claim unavailable facts.

System changes

Reads the three dataset splits and emits audit evidence; it does not rewrite or silently clean source records.

Syntax explained

source_document_id: Stable unit used to prevent a source and its chunks from crossing split boundaries.
32 KiB record cap: Bounds accidental oversized examples and audit memory use; it is not the model context limit.
exact content hash: Detects byte-normalized prompt/completion duplicates, supplemented by near-duplicate review.

File scripts/audit_dataset.py

Configuration

from __future__ import annotations

import hashlib
import json
import re
import sys
from pathlib import Path

FILES = [Path("data/train.jsonl"), Path("data/validation.jsonl"), Path("data/test.jsonl")]
REQUIRED = {"source_document_id", "prompt", "completion", "license_id"}
SECRET_PATTERNS = [
    re.compile(r"-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----"),
    re.compile(r"\bsk-[A-Za-z0-9_-]{20,}\b"),
    re.compile(r"\b(?:AKIA|ASIA)[A-Z0-9]{16}\b"),
]

seen_documents: dict[str, str] = {}
seen_content: dict[str, str] = {}
stats: dict[str, dict[str, int | str]] = {}

for path in FILES:
    rows = 0
    digest = hashlib.sha256()
    with path.open("rb") as stream:
        for number, raw in enumerate(stream, 1):
            digest.update(raw)
            try:
                row = json.loads(raw)
            except json.JSONDecodeError as error:
                raise SystemExit(f"{path}:{number}: invalid JSON: {error}") from error
            missing = REQUIRED - row.keys()
            if missing:
                raise SystemExit(f"{path}:{number}: missing {sorted(missing)}")
            if not all(isinstance(row[key], str) and row[key].strip() for key in REQUIRED):
                raise SystemExit(f"{path}:{number}: required fields must be non-empty strings")
            text = row["prompt"] + "\n" + row["completion"]
            if len(text.encode("utf-8")) > 32_768:
                raise SystemExit(f"{path}:{number}: example exceeds 32 KiB")
            for pattern in SECRET_PATTERNS:
                if pattern.search(text):
                    raise SystemExit(f"{path}:{number}: secret-like pattern detected")
            document = row["source_document_id"]
            previous = seen_documents.setdefault(document, path.name)
            if previous != path.name:
                raise SystemExit(f"document {document!r} crosses {previous} and {path.name}")
            content_hash = hashlib.sha256(text.strip().encode()).hexdigest()
            previous_content = seen_content.setdefault(content_hash, f"{path}:{number}")
            if previous_content != f"{path}:{number}":
                raise SystemExit(f"{path}:{number}: exact duplicate of {previous_content}")
            rows += 1
    stats[path.name] = {"rows": rows, "sha256": digest.hexdigest()}

if min(int(item["rows"]) for item in stats.values()) == 0:
    raise SystemExit("every split must contain at least one row")
print(json.dumps({"status": "passed", "splits": stats}, indent=2, sort_keys=True))

Example output / evidence

{
  "status": "passed",
  "splits": {
    "test.jsonl": {"rows": 240, "sha256": "8f0a…"},
    "train.jsonl": {"rows": 4800, "sha256": "36bd…"},
    "validation.jsonl": {"rows": 480, "sha256": "8d11…"}
  }
}

Checkpoint: Accept the immutable split revision

python3 scripts/audit_dataset.py | tee governance/dataset-audit.json

Continue whenThe audit passes, split hashes match the provenance manifest, and independent near-duplicate, privacy, and sample-quality reviews are attached.

Stop whenStop on any cross-split document, duplicate, secret-like string, unapproved source, empty split, unexplained class imbalance, or unresolved personal-data concern.

If this step fails

The dataset audit finds personal data, secrets, or credentials.

Likely causeCollection, export, issue text, logs, or generated examples entered the corpus without adequate minimization and scanning.

Safe checks

Quarantine the affected shard and record its digest
Identify source, scope, access, and downstream copies
Search tokenizer, caches, checkpoints, and artifacts according to incident policy

ResolutionStop training, follow privacy or security incident procedure, remove the data from a new versioned corpus, invalidate derived artifacts as policy requires, and never rewrite the original lineage.

Tokenized splits contain duplicates or overlapping documents.

Likely causeSplitting happened after chunking, document IDs were unstable, normalization collapsed examples, or near-duplicate detection was omitted.

Safe checks

Recompute exact hashes before tokenization
Check source document IDs across train, validation, and test
Run near-duplicate analysis at document and chunk levels

ResolutionSplit by stable source document before chunking, rebuild every derived shard, and invalidate evaluation produced from contaminated splits.

The dataset license or source permission cannot be proven.

Likely causeThe dataset card is incomplete, a mixed corpus lost per-source terms, redistribution differs from training permission, or an internal source lacks recorded authority.

Safe checks

Inspect the provenance manifest and source-specific license text
Confirm dataset revision and content digest
Escalate ambiguous terms to the appropriate legal or governance owner

ResolutionExclude the source and rebuild the corpus. Do not infer permission from public accessibility, a repository host, or another model's use of the same text.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Generate a new corpus revision with explicit exclusions; never edit an accepted split in place or reuse contaminated test results.

Stop conditions

Stop on any cross-split document, duplicate, secret-like string, unapproved source, empty split, unexplained class imbalance, or unresolved personal-data concern.

command

Evaluate the untouched base model as the frozen baseline

caution

Run the deterministic evaluator with no adapter against the untouched test set. Preserve per-case outputs under restricted access, aggregate task metrics, latency, failures, template, decoding fields, model revision, and evaluator commit. Review cases manually before declaring the metric meaningful.

Why this step matters

Fine-tuning is useful only relative to the actual base artifact under the same prompt, evaluator, and test cases. Freezing the baseline first prevents a moving reference and exposes tasks the base already solves without training risk.

What to understand

Exact normalized match is appropriate only for deliberately deterministic answers. Add schema validity, field-level scores, human rubric, refusal, escalation, factual support, and safety cases as the task requires.

Use greedy decoding and fixed bounds for reproducibility, then separately test the production decoding policy. A single decoding setup cannot characterize every user experience.

Retain failures and empty responses. Do not filter hard cases, timeouts, or policy regressions from the denominator, and do not expose raw sensitive test text in public reports.

System changes

Loads the base model into accelerator memory and writes a restricted baseline result; no weights are changed.

Syntax explained

ADAPTER_PATH unset: Runs the immutable base artifact without PEFT weights.
do_sample=False: Uses deterministic greedy generation for the fixed functional comparison.
exact_normalized_rate: A narrow deterministic metric that must be supplemented by task and safety review.

Command

BASE_MODEL=Qwen/Qwen3-0.6B BASE_REVISION="$BASE_REVISION" OUT_PATH=evaluation/baseline.json python3 scripts/evaluate_adapter.py

Example output / evidence

{"base_model":"Qwen/Qwen3-0.6B","base_revision":"7d1f…","cases":240,"exact_normalized_rate":0.6125,"label":"baseline"}

Checkpoint: Approve the baseline

python3 -m json.tool evaluation/baseline.json >/dev/null && sha256sum evaluation/baseline.json

Continue whenEvery immutable test case is represented, model/template/decoding identity is recorded, and reviewers accept metric behavior before training.

Stop whenStop if test data influenced training preparation, the base revision or template is unpinned, outputs are filtered, or the metric contradicts reviewed task quality.

If this step fails

Baseline results cannot be reproduced.

Likely causeThe base revision, tokenizer, evaluation set, template, decoding fields, seed, runtime, or metric implementation changed.

Safe checks

Compare every digest in the run manifest
Run the fixed baseline command in the locked environment
Inspect metric version and exact test IDs

ResolutionRestore the immutable baseline environment or create a new baseline lineage. Never compare the candidate against a moving or undocumented reference.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

If the base already passes every gate, stop and ship the simpler prompt or application validation rather than creating an adapter.

Stop conditions

Stop if test data influenced training preparation, the base revision or template is unpinned, outputs are filtered, or the metric contradicts reviewed task quality.

command

Measure a short QLoRA memory and throughput pilot

caution

Run a bounded pilot on a representative data slice before committing the budget. Confirm four-bit NF4 loading, bfloat16 support, trainable parameter count, effective batch, sequence length, checkpoint size, GPU peak memory, tokens per second, validation cadence, and estimated completion time.

Why this step matters

A measured pilot turns memory and time estimates into evidence and catches environment, data, and checkpoint problems before hours of accelerator use. The one-hour timeout is a budget boundary, not a performance target.

What to understand

The 0.6B example is intentionally modest. A 7B or larger model, longer sequence, bigger batch, full fine-tuning, multi-GPU topology, or optimizer change can multiply memory, communication, storage, and cost.

QLoRA stores base linear weights in four-bit form and trains added parameters; it does not make every operation four-bit and it does not guarantee quality equal to a full fine-tune on this task.

Record peak allocated and reserved memory, host RAM, checkpoint bytes, power if available, and throughput after warm-up. Extrapolate with evaluation and checkpoint overhead, then apply an uncertainty margin.

System changes

Downloads or loads the pinned base, allocates GPU memory, performs bounded gradient updates, and writes disposable pilot checkpoints.

Syntax explained

NF4 with double quantization: Reduces base-weight memory for QLoRA; compatibility and quality still require measurement.
effective batch 16: Micro-batch two times eight accumulation steps on one process; preserve this when reducing memory.
timeout 3600s: Stops the pilot after one hour so a stalled or slow run cannot consume the full budget silently.

Command

CUDA_VISIBLE_DEVICES=0 BASE_REVISION="$BASE_REVISION" OUTPUT_DIR=runs/pilot timeout 3600s python3 scripts/train_qlora.py

Example output / evidence

trainable params: 17,694,720 || all params: 613,947,392 || trainable%: 2.8821
{"status":"complete","metrics":{"train_loss":1.842,"train_runtime":1847.2,"train_samples_per_second":5.19}}

Checkpoint: Approve full-run capacity

nvidia-smi --query-compute-apps=pid,used_memory --format=csv,noheader; cat runs/pilot/train-metrics.json

Continue whenThe intended kernels run, memory remains below the safety margin, throughput supports the budget, and a pilot checkpoint and validation event can be restored.

Stop whenStop on OOM, CPU fallback, NaN, unresolved warnings, thermal instability, unbounded duration, missing validation, or projected cost above approval.

If this step fails

The run fails with CUDA out of memory before the first optimizer step.

Likely causeThe model, sequence length, micro-batch, optimizer state, activations, allocator fragmentation, or another process exceeds available accelerator memory.

Safe checks

Record nvidia-smi memory and active processes before changing settings
Confirm effective sequence length and per-device micro-batch
Inspect whether quantization, gradient checkpointing, and intended precision actually activated

ResolutionStop other GPU work, reduce micro-batch or sequence length, use gradient accumulation to preserve the declared effective batch, and re-run the baseline rather than hiding a changed experiment.

Training loss becomes NaN or infinite.

Likely causeLearning rate, precision, invalid labels, corrupt tokens, exploding gradients, an unsupported kernel, or a numerically unstable quantization path caused divergence.

Safe checks

Inspect the first non-finite step and preceding gradient norm
Run the same batch in full precision on a small model
Verify labels, attention masks, tokenizer IDs, and maximum token ID

ResolutionRestore the last finite checkpoint, correct data or precision, lower learning rate if justified by a controlled experiment, enable finite gradient clipping, and restart from a clean recorded state.

Training is much slower than the cost estimate.

Likely causeTokenization, data loading, CPU contention, storage, precision, kernel fallback, gradient checkpointing, communication, or thermal/power limits differ from the measured pilot.

Safe checks

Separate data, forward, backward, optimizer, checkpoint, and evaluation time
Measure tokens per second after warm-up
Inspect GPU utilization, clocks, power, memory, and input pipeline stalls

ResolutionPause before consuming the full budget. Fix the bounded bottleneck, repeat a representative pilot, and update cost and completion estimates with measured throughput.

The GPU is visible but unsupported kernels or CPU fallback are used.

Likely causeDriver, CUDA, compute capability, package wheel, dtype, bitsandbytes backend, or attention implementation is incompatible.

Safe checks

Record torch and CUDA versions and device capability
Inspect runtime logs for backend and kernel selection
Run a tiny matrix operation and memory-footprint probe

ResolutionUse a documented compatible stack, rebuild the isolated environment, and rerun the measured pilot. Do not publish throughput from an unintended fallback.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Reduce model size, sequence, or micro-batch while preserving evaluation and effective-batch disclosure; use LoRA on bfloat16 base if four-bit kernels are unsupported and memory permits.

Stop conditions

Stop on OOM, CPU fallback, NaN, unresolved warnings, thermal instability, unbounded duration, missing validation, or projected cost above approval.

config

Create the deterministic LoRA and QLoRA trainer

caution

Save the trainer with explicit base revision, four-bit NF4 config, all-linear LoRA targets, fixed seed, completion rendering, bounded sequence, effective batch, gradient checkpointing, evaluation and save cadence, best-checkpoint rule, local-only reporting, safe adapter serialization, and optional complete-checkpoint resume.

Why this step matters

Explicit code makes training assumptions reviewable and reproducible. Convenience defaults can change across versions and may silently alter precision, loss masking, chat rendering, evaluation, reporting, or checkpoint behavior.

What to understand

The script trains all linear-module adapters in QLoRA style. Target selection is architecture-dependent; verify trainable modules and parameter count rather than assuming a pattern copied from another model.

This dataset renders the full user and assistant conversation. If the task requires assistant-only or completion-only loss, verify chat-template generation markers and TRL behavior with token-level fixtures before changing loss masking.

Best validation loss selects a checkpoint, but final acceptance uses the untouched test and hard gates. Do not select checkpoints repeatedly against the test set.

System changes

Creates training code; execution writes adapter and checkpoint state but never changes base weights.

Syntax explained

target_modules=all-linear: Applies adapters to linear layers in the documented QLoRA style; inspect actual matched modules.
r=16, alpha=32: Defines adapter rank and scaling for this experiment, not a universally optimal choice.
load_best_model_at_end: Restores the best validation-loss checkpoint while the untouched test remains reserved.

File scripts/train_qlora.py

Configuration

from __future__ import annotations

import json
import os
from pathlib import Path

import torch
from datasets import load_dataset
from peft import LoraConfig, prepare_model_for_kbit_training
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, set_seed
from trl import SFTConfig, SFTTrainer

BASE_MODEL = os.getenv("BASE_MODEL", "Qwen/Qwen3-0.6B")
BASE_REVISION = os.environ["BASE_REVISION"]
OUTPUT_DIR = Path(os.getenv("OUTPUT_DIR", "runs/qwen3-support-lora"))
SEED = 20260729

set_seed(SEED)
torch.backends.cuda.matmul.allow_tf32 = False
torch.backends.cudnn.allow_tf32 = False

tokenizer = AutoTokenizer.from_pretrained(
    BASE_MODEL,
    revision=BASE_REVISION,
    use_fast=True,
)
if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token

quantization = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)
model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    revision=BASE_REVISION,
    quantization_config=quantization,
    dtype=torch.bfloat16,
    device_map={"": 0},
)
model.config.use_cache = False
model = prepare_model_for_kbit_training(
    model,
    use_gradient_checkpointing=True,
)

dataset = load_dataset(
    "json",
    data_files={
        "train": "data/train.jsonl",
        "validation": "data/validation.jsonl",
    },
)

def render(row: dict[str, str]) -> dict[str, str]:
    messages = [
        {"role": "user", "content": row["prompt"]},
        {"role": "assistant", "content": row["completion"]},
    ]
    return {
        "text": tokenizer.apply_chat_template(
            messages,
            tokenize=False,
            add_generation_prompt=False,
        )
    }

rendered = dataset.map(render, remove_columns=dataset["train"].column_names)

lora = LoraConfig(
    task_type="CAUSAL_LM",
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    target_modules="all-linear",
    bias="none",
)
args = SFTConfig(
    output_dir=str(OUTPUT_DIR),
    dataset_text_field="text",
    max_length=1024,
    packing=False,
    learning_rate=1e-4,
    num_train_epochs=2,
    per_device_train_batch_size=2,
    per_device_eval_batch_size=2,
    gradient_accumulation_steps=8,
    gradient_checkpointing=True,
    bf16=True,
    tf32=False,
    max_grad_norm=1.0,
    warmup_ratio=0.03,
    lr_scheduler_type="cosine",
    eval_strategy="steps",
    eval_steps=50,
    save_strategy="steps",
    save_steps=50,
    save_total_limit=3,
    logging_steps=5,
    report_to="none",
    seed=SEED,
    data_seed=SEED,
    load_best_model_at_end=True,
    metric_for_best_model="eval_loss",
    greater_is_better=False,
    push_to_hub=False,
)
trainer = SFTTrainer(
    model=model,
    args=args,
    train_dataset=rendered["train"],
    eval_dataset=rendered["validation"],
    processing_class=tokenizer,
    peft_config=lora,
)

resume = os.getenv("RESUME_FROM_CHECKPOINT")
result = trainer.train(resume_from_checkpoint=resume or None)
metrics = {key: float(value) for key, value in result.metrics.items()}
trainer.save_model(str(OUTPUT_DIR / "adapter"))
tokenizer.save_pretrained(str(OUTPUT_DIR / "adapter"))
(OUTPUT_DIR / "train-metrics.json").write_text(
    json.dumps(metrics, indent=2, sort_keys=True),
    encoding="utf-8",
)
print(json.dumps({"status": "complete", "metrics": metrics}, sort_keys=True))

Example output / evidence

Trainer syntax valid: base revision required, seed=20260729, r=16, alpha=32, max_length=1024, effective batch=16, eval/save=50, retention=3.

Checkpoint: Review code and compile syntax

python3 -m py_compile scripts/train_qlora.py && grep -n 'BASE_REVISION\|push_to_hub=False\|save_total_limit' scripts/train_qlora.py

Continue whenCompilation passes and reviewers confirm immutable inputs, bounded resources, evaluation, checkpointing, local reporting, and no automatic publication.

Stop whenStop if the script loads a moving base revision, logs sensitive examples, pushes automatically, omits validation, or cannot restore optimizer and scheduler state.

If this step fails

A checkpoint loads but generation is nonsensical.

Likely causeTokenizer files, special-token IDs, chat template, architecture config, adapter/base pairing, or weight files do not belong to the same immutable revision.

Safe checks

Compare tokenizer and config digests with the checkpoint manifest
Verify BOS, EOS, PAD, and vocabulary size
Run a fixed token round-trip and inspect missing or unexpected keys

ResolutionRestore the matching tokenizer, config, base revision, and adapter or checkpoint. Never patch incompatible IDs in place and call the result recovered.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Use plain LoRA with a bfloat16 base by removing quantized loading only after a separate memory pilot; retain the same adapter, data, evaluation, and checkpoint controls.

Stop conditions

Stop if the script loads a moving base revision, logs sensitive examples, pushes automatically, omits validation, or cannot restore optimizer and scheduler state.

command

Run the approved training job and monitor evidence, not anecdotes

caution

Launch one run ID with the accepted data, code, environment, base revision, and resource plan. Observe finite loss, gradient norm, learning rate, throughput, validation loss, memory, checkpoint success, disk, and system stability. Do not read a few generated samples and change hyperparameters mid-run.

Why this step matters

Monitoring protects the approved budget and detects divergence, data stalls, storage exhaustion, hardware fallback, and missing checkpoints early. It is operational evidence, not a substitute for final evaluation.

What to understand

Set restrictive permissions because console logs and checkpoints can contain paths, metadata, or derived model behavior. Do not enable sample logging against sensitive examples.

Validation loss should be interpreted with train loss, task metrics, and memorization checks. A continued train-loss decline with worse validation is a stop signal, not a reason to add epochs.

Preserve failed runs and their manifests. Renaming only the successful directory and deleting evidence encourages survivorship bias and makes cost accounting inaccurate.

System changes

Performs adapter training, consumes GPU/CPU energy, writes logs, metrics, and up to three retained complete checkpoints.

Syntax explained

umask 077: Restricts newly created run files to the current account by default.
CUDA_VISIBLE_DEVICES=0: Pins the single approved accelerator; use stable device UUID policy in shared systems.
tee: Preserves bounded console evidence while operators watch the run.

Command

umask 077; CUDA_VISIBLE_DEVICES=0 BASE_REVISION="$BASE_REVISION" OUTPUT_DIR=runs/qwen3-support-lora python3 scripts/train_qlora.py 2>&1 | tee runs/qwen3-support-lora.console.log

Example output / evidence

step=50 loss=1.742 grad_norm=0.811 learning_rate=9.7e-05 eval_loss=1.681
step=100 loss=1.603 grad_norm=0.744 learning_rate=8.9e-05 eval_loss=1.574
{"status":"complete","metrics":{"train_loss":1.622,"train_runtime":3621.8}}

Checkpoint: Accept the completed run lineage

cat runs/qwen3-support-lora/train-metrics.json && find runs/qwen3-support-lora -maxdepth 2 -type f -name trainer_state.json -print

Continue whenThe run completes within budget with finite metrics, expected evaluation cadence, a best adapter, complete checkpoint lineage, and no safety or infrastructure incident.

Stop whenStop on non-finite loss, exploding gradients, worsening validation, secret exposure, missing checkpoints, OOM, disk pressure, hardware instability, or budget overrun.

If this step fails

Training loss becomes NaN or infinite.

Likely causeLearning rate, precision, invalid labels, corrupt tokens, exploding gradients, an unsupported kernel, or a numerically unstable quantization path caused divergence.

Safe checks

Inspect the first non-finite step and preceding gradient norm
Run the same batch in full precision on a small model
Verify labels, attention masks, tokenizer IDs, and maximum token ID

Training is much slower than the cost estimate.

Likely causeTokenization, data loading, CPU contention, storage, precision, kernel fallback, gradient checkpointing, communication, or thermal/power limits differ from the measured pilot.

Safe checks

Separate data, forward, backward, optimizer, checkpoint, and evaluation time
Measure tokens per second after warm-up
Inspect GPU utilization, clocks, power, memory, and input pipeline stalls

ResolutionPause before consuming the full budget. Fix the bounded bottleneck, repeat a representative pilot, and update cost and completion estimates with measured throughput.

Checkpoint storage fills during training.

Likely causeSave frequency, retention, optimizer state, temporary writes, or concurrent runs exceed the capacity plan.

Safe checks

Measure checkpoint size and free space
List complete and partial checkpoint directories
Compare save_total_limit and expected run duration

ResolutionPause safely, preserve the latest verified complete checkpoint and required lineage, remove only explicitly obsolete artifacts, and revise retention before resuming.

The adapter or trained model overfits and copies long training passages.

Likely causeThe corpus is too small or repetitive, epochs or capacity are excessive, validation misses memorization, or sensitive content was present.

Safe checks

Run held-out prefix and canary extraction tests
Compare train and validation loss trajectories
Inspect nearest training examples for suspicious generations

ResolutionReject publication, improve data and early stopping, reduce capacity or epochs, remove sensitive material, and rerun memorization evaluation before training again.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Terminate and create a new run ID after correcting one documented cause; do not mutate the accepted data or hyperparameters inside a running lineage.

Stop conditions

Stop on non-finite loss, exploding gradients, worsening validation, secret exposure, missing checkpoints, OOM, disk pressure, hardware instability, or budget overrun.

command

Interrupt and resume from a complete checkpoint before trusting recovery

caution

Test controlled interruption in the pilot or canary, verify the newest checkpoint has model or adapter, optimizer, scheduler, scaler, RNG, trainer state, tokenizer, config, and manifest, then resume with the identical environment and data. Compare the resumed trajectory with a continuous control around the boundary.

Why this step matters

Checkpoint files are not recovery evidence until a matching process resumes them. Training interruptions are expected on preemptible or maintained infrastructure, and an untested resume path can waste the entire run.

What to understand

Use atomic completion markers or framework-complete directories. A newest timestamp may point to a partially written checkpoint after power loss or storage failure.

World size, batch, data order, package versions, model revision, and code must match. A deliberate change creates a new experiment even if it can technically read the old weights.

Compare global step, scheduler learning rate, optimizer state, and a few subsequent losses against a control tolerance. Exact bitwise reproducibility may depend on kernels, but material discontinuity requires explanation.

System changes

Reads a retained checkpoint and continues writing into the same approved run lineage after an integrity check.

Syntax explained

RESUME_FROM_CHECKPOINT: Selects a complete trainer checkpoint rather than loading only adapter weights.
global step: Confirms scheduler and data progress continue at the recorded position.
continuous control: A small uninterrupted reference used to detect recovery discontinuity.

Command

RESUME_FROM_CHECKPOINT=runs/qwen3-support-lora/checkpoint-100 BASE_REVISION="$BASE_REVISION" OUTPUT_DIR=runs/qwen3-support-lora python3 scripts/train_qlora.py

Example output / evidence

Loading model from runs/qwen3-support-lora/checkpoint-100
Continuing training from global step 100
step=105 loss=1.598 learning_rate=8.8e-05

Checkpoint: Accept recovery evidence

grep -E 'global step 100|step=105' runs/qwen3-support-lora.console.log | tail

Continue whenThe complete matching checkpoint resumes at the correct step with compatible scheduler, optimizer, data, and finite loss.

Stop whenStop if checkpoint integrity, code, packages, data digest, topology, optimizer, scheduler, or global step differs from the manifest.

If this step fails

The resumed run does not match the pre-interruption learning curve.

Likely causeOptimizer, scheduler, scaler, random generator, sampler position, data revision, code, or world size was not restored from the same checkpoint contract.

Safe checks

Compare checkpoint manifest, code commit, dataset digest, and package lock
Confirm optimizer and scheduler files exist
Inspect global step, epoch, sampler, seed, and accelerator topology

ResolutionDo not splice incompatible histories. Resume only from a complete matching checkpoint or start a new run ID and preserve both lineages.

The training process is killed or the host reboots.

Likely causePreemption, maintenance, kernel failure, out-of-memory killer, power loss, or scheduler timeout interrupted the process.

Safe checks

Inspect system and scheduler events without modifying checkpoints
Verify the newest checkpoint was atomically completed
Confirm code, data, topology, and package lock still match

ResolutionResume only from the newest verified complete checkpoint. If integrity or state compatibility fails, fall back to the previous checkpoint and preserve the broken artifact for diagnosis.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Fall back to the previous verified checkpoint or start a new run lineage; never stitch incompatible curves into one reported experiment.

Stop conditions

Stop if checkpoint integrity, code, packages, data digest, topology, optimizer, scheduler, or global step differs from the manifest.

config

Evaluate baseline and adapter with the same deterministic harness

read-only

Save the evaluator, then run it once for the frozen base and once for the selected adapter. Join results by immutable case ID, calculate the predeclared task delta, inspect every regression, and run separate human, safety, memorization, out-of-domain, fairness, and latency gates before considering merge or serving.

Why this step matters

The same deterministic harness isolates the adapter's measured effect on a narrow test. Per-case review prevents average improvement from hiding critical regressions or copied content.

What to understand

The example writes raw expected and generated answers, so keep the artifact restricted. A public report should contain approved aggregate and redacted case evidence only.

Require the 0.08 absolute improvement over baseline and zero hard-gate failures. Calculate bootstrap uncertainty or confidence appropriate to the sample; 240 cases can still miss rare failures.

Test the base plus adapter pairing explicitly. Loading the adapter over another base revision may succeed structurally while producing invalid behavior and lineage.

System changes

Loads base and adapter for read-only inference and writes restricted per-case evaluation results.

Syntax explained

ADAPTER_PATH: Enables the candidate while leaving it unset reproduces the frozen baseline.
case_id: Joins baseline and candidate outcomes without relying on list order.
greedy generation: Reduces sampling variance for deterministic task comparison.

File scripts/evaluate_adapter.py

Configuration

from __future__ import annotations

import json
import os
import re
import time
from pathlib import Path

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, set_seed

BASE_MODEL = os.getenv("BASE_MODEL", "Qwen/Qwen3-0.6B")
BASE_REVISION = os.environ["BASE_REVISION"]
ADAPTER = os.getenv("ADAPTER_PATH")
TEST_PATH = Path(os.getenv("TEST_PATH", "data/test.jsonl"))
OUT_PATH = Path(os.getenv("OUT_PATH", "evaluation/results.json"))
SEED = 20260729

set_seed(SEED)
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL, revision=BASE_REVISION)
model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    revision=BASE_REVISION,
    dtype=torch.bfloat16,
    device_map={"": 0},
)
label = "baseline"
if ADAPTER:
    model = PeftModel.from_pretrained(model, ADAPTER, is_trainable=False)
    label = "adapter"
model.eval()

rows = [json.loads(line) for line in TEST_PATH.read_text(encoding="utf-8").splitlines()]
results = []
for row in rows:
    messages = [{"role": "user", "content": row["prompt"]}]
    prompt = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    started = time.perf_counter()
    with torch.inference_mode():
        output = model.generate(
            **inputs,
            do_sample=False,
            max_new_tokens=180,
            eos_token_id=tokenizer.eos_token_id,
            pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id,
        )
    generated = tokenizer.decode(
        output[0][inputs["input_ids"].shape[1]:],
        skip_special_tokens=True,
    ).strip()
    expected = row["completion"].strip()
    normalize = lambda value: re.sub(r"\s+", " ", value).strip().casefold()
    results.append(
        {
            "case_id": row["source_document_id"],
            "exact_normalized": normalize(generated) == normalize(expected),
            "latency_seconds": round(time.perf_counter() - started, 4),
            "generated": generated,
            "expected": expected,
        }
    )

summary = {
    "label": label,
    "base_model": BASE_MODEL,
    "base_revision": BASE_REVISION,
    "cases": len(results),
    "exact_normalized_rate": sum(r["exact_normalized"] for r in results) / len(results),
}
OUT_PATH.parent.mkdir(parents=True, exist_ok=True)
OUT_PATH.write_text(
    json.dumps({"summary": summary, "results": results}, indent=2),
    encoding="utf-8",
)
print(json.dumps(summary, sort_keys=True))

Example output / evidence

{"base_model":"Qwen/Qwen3-0.6B","base_revision":"7d1f…","cases":240,"exact_normalized_rate":0.7167,"label":"adapter"}

Checkpoint: Apply every release gate

ADAPTER_PATH=runs/qwen3-support-lora/adapter BASE_REVISION="$BASE_REVISION" OUT_PATH=evaluation/adapter.json python3 scripts/evaluate_adapter.py

Continue whenThe adapter exceeds the predeclared delta and passes per-case correctness, safety, privacy, memorization, out-of-domain, latency, and policy gates with no test leakage.

Stop whenStop if the delta is below threshold, any critical case regresses, test data influenced selection, memorization appears, or reviewers cannot explain metric behavior.

If this step fails

Evaluation improves while representative answers become worse.

Likely causeThe held-out set is too similar to training data, the metric rewards surface form, prompt formatting differs, or the model memorized templates without improving the intended task.

Safe checks

Check document and near-duplicate leakage across splits
Review blinded task examples and safety cases
Compare baseline and candidate with the exact production template

ResolutionReject the candidate, repair split and rubric design, add adversarial and out-of-domain cases, and retrain only after the evaluation can detect the observed regression.

The adapter or trained model overfits and copies long training passages.

Likely causeThe corpus is too small or repetitive, epochs or capacity are excessive, validation misses memorization, or sensitive content was present.

Safe checks

Run held-out prefix and canary extraction tests
Compare train and validation loss trajectories
Inspect nearest training examples for suspicious generations

ResolutionReject publication, improve data and early stopping, reduce capacity or epochs, remove sensitive material, and rerun memorization evaluation before training again.

The candidate passes aggregate metrics but fails a safety or policy case.

Likely causeAverages hide a critical regression, the gate was treated as weighted, or evaluation omitted a required class.

Safe checks

Inspect per-case results and hard-gate status
Re-run the failing case with saved prompt and artifact identity
Review whether the test is valid and independent of training data

ResolutionReject promotion. Hard safety, privacy, licensing, and policy gates cannot be offset by lower loss or better average task score.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Keep the base model and use application validation, retrieval, or a corrected prompt when the adapter does not deliver a robust gated improvement.

Stop conditions

Stop if the delta is below threshold, any critical case regresses, test data influenced selection, memorization appears, or reviewers cannot explain metric behavior.

command

Package the adapter, tokenizer, manifest, model card, and evidence

caution

Create an immutable release candidate containing adapter weights, PEFT config, matching tokenizer and template, base model ID and revision, dataset lineage, code and environment digests, task/evaluation scope, known limitations, prohibited uses, training budget, metrics, and rollback. Do not include raw training or test data.

Why this step matters

An adapter is unusable and unsafe without its exact base pairing, tokenizer, provenance, evaluation, scope, and limitations. Deterministic packaging and digest allow controlled promotion and rollback.

What to understand

The model card must state that the adapter targets a narrow support-format task and does not add current knowledge or guarantee factual correctness. Include negative results and untested populations.

Do not publish automatically. A separate reviewer confirms license, redistribution, data disclosure, model card, security scan, test gates, and destination visibility.

Retain the unmerged adapter as the canonical trained artifact. It is smaller, preserves base separation, and makes rollback and lineage clearer than only keeping a merged model.

System changes

Creates a compressed immutable release candidate and digest; it does not upload or expose the artifact.

Syntax explained

--sort=name and fixed mtime: Makes archive ordering and timestamps deterministic for stable digests.
adapter plus tokenizer: Preserves the exact interpretation and base pairing needed for inference.
sha256sum: Identifies release bytes but does not establish trust, license, or quality alone.

Command

tar --sort=name --mtime='UTC 2026-07-29' --owner=0 --group=0 -czf artifacts/qwen3-support-lora-r1.tgz -C runs/qwen3-support-lora adapter train-metrics.json && sha256sum artifacts/qwen3-support-lora-r1.tgz

Example output / evidence

9d6689b18c7ac6cbd4b6e989df90dbf932b1e76d94f6a01d0a134d8bea48c7e2  artifacts/qwen3-support-lora-r1.tgz

Checkpoint: Approve the release candidate

tar -tzf artifacts/qwen3-support-lora-r1.tgz && sha256sum -c artifacts/qwen3-support-lora-r1.tgz.sha256

Continue whenThe archive contains only approved adapter/tokenizer/metrics/card/manifest content and its digest matches the signed release record.

Stop whenStop if raw data, secrets, unrestricted logs, missing base revision, absent limitations, license ambiguity, or automatic public upload is found.

If this step fails

A model hub upload exposes a private artifact.

Likely causeRepository visibility, token scope, push settings, generated card, logs, dataset samples, or checkpoint contents were not reviewed before upload.

Safe checks

Stop the upload and inspect repository visibility
Revoke the token and audit uploaded revisions
Check model, tokenizer, adapter, card, logs, and dataset references

ResolutionQuarantine or remove the remote revision according to provider and incident policy, rotate credentials, and require an explicit publication gate instead of automatic push.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Store the adapter in an access-controlled artifact registry with immutable retention and signed provenance rather than a public model hub.

Stop conditions

Stop if raw data, secrets, unrestricted logs, missing base revision, absent limitations, license ambiguity, or automatic public upload is found.

config

Merge only when a target runtime requires standalone weights

caution

Keep adapter loading as the default. When a serving or conversion path requires standalone weights, merge the adapter into the exact bfloat16 base with safe merge, save safetensors, generate file digests, and treat the result as a new artifact that must pass the complete evaluation again.

Why this step matters

Merging changes packaging and can introduce pairing or precision mistakes. It is an operational transformation, not a training improvement, and must not bypass adapter evidence.

What to understand

Merge from the accepted adapter and exact base revision on a host with enough CPU RAM and disk. The script refuses to overwrite an existing output directory.

Compare base-plus-adapter and merged outputs on every test case. Record file digests and config/tokenizer identity before quantizing or converting further.

Merged weights may have different redistribution obligations and storage cost. Obtain the same publication approval as for any derivative model artifact.

System changes

Creates a new standalone bfloat16 derivative model while preserving the canonical adapter and base separately.

Syntax explained

safe_merge=True: Requests PEFT consistency checks during merge; full behavioral evaluation is still mandatory.
safe_serialization=True: Writes safetensors rather than pickle-based model weight files.
exist_ok=False: Prevents accidental overwrite of a previously accepted artifact directory.

File scripts/merge_adapter.py

Configuration

from __future__ import annotations

import hashlib
import json
import os
from pathlib import Path

import torch
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

ADAPTER = Path(os.getenv("ADAPTER_PATH", "runs/qwen3-support-lora/adapter"))
OUTPUT = Path(os.getenv("MERGED_PATH", "artifacts/qwen3-support-merged"))

model = AutoPeftModelForCausalLM.from_pretrained(
    ADAPTER,
    dtype=torch.bfloat16,
    device_map="cpu",
    low_cpu_mem_usage=True,
)
merged = model.merge_and_unload(safe_merge=True)
tokenizer = AutoTokenizer.from_pretrained(ADAPTER)
OUTPUT.mkdir(parents=True, exist_ok=False)
merged.save_pretrained(OUTPUT, safe_serialization=True, max_shard_size="2GB")
tokenizer.save_pretrained(OUTPUT)

files = {}
for path in sorted(OUTPUT.iterdir()):
    if path.is_file():
        files[path.name] = hashlib.sha256(path.read_bytes()).hexdigest()
(OUTPUT / "artifact-manifest.json").write_text(
    json.dumps({"files": files, "source_adapter": str(ADAPTER)}, indent=2),
    encoding="utf-8",
)
print(json.dumps({"status": "merged", "files": len(files), "path": str(OUTPUT)}))

Example output / evidence

{"status":"merged","files":6,"path":"artifacts/qwen3-support-merged"}

Checkpoint: Prove merge equivalence

ADAPTER_PATH=runs/qwen3-support-lora/adapter MERGED_PATH=artifacts/qwen3-support-merged python3 scripts/merge_adapter.py

Continue whenThe merge completes into a new directory and the merged model passes all adapter tests within predeclared tolerances.

Stop whenStop on missing or unexpected keys, tokenizer/config drift, non-finite weights, overwrite, redistribution uncertainty, or any evaluation regression.

If this step fails

A checkpoint loads but generation is nonsensical.

Likely causeTokenizer files, special-token IDs, chat template, architecture config, adapter/base pairing, or weight files do not belong to the same immutable revision.

Safe checks

Compare tokenizer and config digests with the checkpoint manifest
Verify BOS, EOS, PAD, and vocabulary size
Run a fixed token round-trip and inspect missing or unexpected keys

ResolutionRestore the matching tokenizer, config, base revision, and adapter or checkpoint. Never patch incompatible IDs in place and call the result recovered.

Inference output differs after merge, quantization, or serving conversion.

Likely causeThe conversion changed precision, tokenizer, template, adapter scaling, architecture metadata, or unsupported operations.

Safe checks

Run the identical evaluation before and after each transformation
Compare weight, tokenizer, config, and adapter manifests
Inspect conversion warnings and missing keys

ResolutionKeep the last accepted artifact, reject the transformed candidate, and change one conversion variable at a time. A smaller artifact is not acceptable without quality evidence.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Serve the base plus adapter directly when the runtime supports it; this preserves compact artifacts and clearer lineage.

Stop conditions

Stop on missing or unexpected keys, tokenizer/config drift, non-finite weights, overwrite, redistribution uncertainty, or any evaluation regression.

decision

Evaluate serving quantization as a separate candidate

caution

Do not assume the QLoRA training representation is a deployable quantized model or that merging preserves it. Choose a runtime-supported inference quantization method, pin converter and calibration data, create a new artifact ID, then rerun correctness, safety, memorization, latency, throughput, memory, and compatibility gates against adapter and merged references.

Why this step matters

Quantization trades representation precision, memory, compatibility, and sometimes speed. It is a new release candidate, not a free optimization, and task regressions can concentrate in rare but important cases.

What to understand

Use calibration data only when its license and privacy allow derived quantization statistics. Keep calibration separate from the untouched test and record its immutable revision.

Measure on target hardware. A format optimized for one accelerator or runtime may be slower, unsupported, or silently converted elsewhere.

Compare artifact bytes, resident memory, cold load, warm p95, throughput, all test cases, safety, and long-context behavior. Reject if any hard gate fails even when average metrics improve.

System changes

The decision record itself changes no weights; an approved conversion creates a new derived quantized artifact.

Syntax explained

separate artifact ID: Prevents quantized and accepted source weights from being confused or overwritten.
calibration revision: Pins any data used to choose quantization scales and its permissions.
per-case regression: Reveals failures hidden by aggregate quality scores.

File governance/quantization-decision.md

Configuration

# Quantization decision
- Source artifact digest:
- Converter and exact version:
- Quantization method and parameters:
- Calibration dataset revision and permission:
- Target hardware and runtime:
- Size and memory before/after:
- Per-case regression against adapter:
- p50/p95 latency and throughput:
- Safety, memorization, and policy gates:
- Accept, reject, and rollback owner:

Example output / evidence

Quantization candidate qwen3-support-int8-r1 rejected: memory -46%, p95 -18%, but 4 critical exact-match regressions.

Checkpoint: Accept or reject quantization

git diff -- governance/quantization-decision.md

Continue whenA reviewer can trace method, source, calibration, target, resource gains, every regression, hard gates, and rollback.

Stop whenStop if conversion provenance, calibration permission, target runtime, or per-case evaluation is missing, or if any hard gate regresses.

If this step fails

Inference output differs after merge, quantization, or serving conversion.

Likely causeThe conversion changed precision, tokenizer, template, adapter scaling, architecture metadata, or unsupported operations.

Safe checks

Run the identical evaluation before and after each transformation
Compare weight, tokenizer, config, and adapter manifests
Inspect conversion warnings and missing keys

ResolutionKeep the last accepted artifact, reject the transformed candidate, and change one conversion variable at a time. A smaller artifact is not acceptable without quality evidence.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Keep the accepted adapter or merged bfloat16 artifact when quantization gains do not justify quality and operational risk.

Stop conditions

Stop if conversion provenance, calibration permission, target runtime, or per-case evaluation is missing, or if any hard gate regresses.

command

Serve the pinned base and adapter behind a protected API

caution

Use a pinned vLLM deployment that explicitly enables the accepted LoRA module. Bind the raw runtime to loopback or a private service network, require a dedicated API key, place authenticated TLS ingress and rate limits in front, restrict model aliases, and keep administrative credentials outside user traffic.

Why this step matters

Serving validates that the accepted adapter works through the actual API, resource scheduler, template, and authentication path. Runtime defaults and model aliases can otherwise invalidate offline evidence.

What to understand

The generation-config flag avoids inheriting unexpected sampling defaults from a repository. Send every required field explicitly and verify unsupported API fields before client migration.

Do not expose the raw listener publicly. Add ingress authentication, authorization, TLS, request size and concurrency limits, timeouts, audit, secret rotation, health/readiness, and abuse response appropriate to the deployment.

Monitor selected model/adapter, request success, queue, p95, token counts, memory, rejected work, and version drift without logging prompts and responses by default.

System changes

Loads base and adapter into an inference server and opens an authenticated loopback listener.

Syntax explained

--enable-lora: Allows named adapter loading; restrict accepted modules and administrative control.
--lora-modules support-r1=…: Maps the immutable accepted adapter directory to a stable API model alias.
--generation-config vllm: Uses runtime defaults rather than silently applying model repository generation settings.

Command

vllm serve Qwen/Qwen3-0.6B --revision "$BASE_REVISION" --host 127.0.0.1 --port 8000 --api-key "$VLLM_API_KEY" --enable-lora --lora-modules support-r1=./runs/qwen3-support-lora/adapter --max-lora-rank 16 --generation-config vllm

Example output / evidence

INFO Loaded model Qwen/Qwen3-0.6B revision 7d1f…
INFO Registered LoRA support-r1 rank=16
INFO Uvicorn running on http://127.0.0.1:8000

Checkpoint: Verify the authenticated adapter API

curl -fsS http://127.0.0.1:8000/v1/chat/completions -H "Authorization: Bearer $VLLM_API_KEY" -H 'Content-Type: application/json' -d '{"model":"support-r1","messages":[{"role":"user","content":"Return the approved support JSON for case DEMO-1."}],"temperature":0,"max_tokens":180}' | python3 -m json.tool

Continue whenThe response names support-r1, matches the accepted template and task behavior, unauthorized access fails, and readiness reflects actual model availability.

Stop whenStop if the listener is exposed, unauthorized requests work, another base or adapter loads, templates drift, resource limits are absent, or API output fails the offline fixture.

If this step fails

The API server accepts requests from an untrusted network.

Likely causeThe listener binds all interfaces, container port publishing bypasses expectations, authentication is absent, or a proxy forwards administrative credentials.

Safe checks

Inspect listening addresses and firewall rules
Test from an untrusted segment with synthetic input
Review TLS, authentication, token scope, and logs

ResolutionReturn to loopback or a protected network immediately. Add authenticated TLS ingress, least-privilege credentials, rate limits, audit, and an owner before intentional exposure.

A checkpoint loads but generation is nonsensical.

Likely causeTokenizer files, special-token IDs, chat template, architecture config, adapter/base pairing, or weight files do not belong to the same immutable revision.

Safe checks

Compare tokenizer and config digests with the checkpoint manifest
Verify BOS, EOS, PAD, and vocabulary size
Run a fixed token round-trip and inspect missing or unexpected keys

ResolutionRestore the matching tokenizer, config, base revision, and adapter or checkpoint. Never patch incompatible IDs in place and call the result recovered.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Use a different officially supported adapter runtime only after the same alias, authentication, resource, compatibility, and evaluation gates pass.

Stop conditions

Stop if the listener is exposed, unauthorized requests work, another base or adapter loads, templates drift, resource limits are absent, or API output fails the offline fixture.

verification

Canary, monitor, roll back, and expire the fine-tuning decision

read-only

Route a small authorized group to the adapter while preserving the base route. Use only approved data classes, collect outcome and refusal feedback without raw content by default, compare latency and errors, verify rollback, and stop automatically on any hard-gate regression. Expansion requires human approval and a signed artifact record.

Why this step matters

Offline test sets cannot cover every user input, operational path, and policy boundary. A bounded canary produces production evidence while a known-good base remains available.

What to understand

Define maximum users, requests, duration, data class, on-call owner, dashboards, alert thresholds, user communication, and incident procedure. Do not silently enroll users in model training.

Evaluate task success, escalation, critical regressions, errors, p95, queue, resource use, and cost. Avoid automatically adding canary conversations to future training data.

Rollback should change routing to the base, revoke the adapter alias, verify representative output, and preserve canary evidence. Removing files is not the first recovery action.

System changes

Records a five-percent canary approval and review date; actual routing is performed by the separately governed deployment platform.

Syntax explained

traffic_percent=5: Bounds initial exposure; use an even smaller or internal-only canary for higher-risk tasks.
baseline_route: Keeps a known-good fallback independently addressable.
review_due: Expires the artifact decision after ninety days or sooner on material change.

Command

printf '%s\n' 'artifact=support-r1' 'traffic_percent=5' 'baseline_route=base-qwen3' 'rollback_tested=yes' 'review_due=2026-10-27' | tee governance/canary-acceptance.txt

Example output / evidence

artifact=support-r1
traffic_percent=5
baseline_route=base-qwen3
rollback_tested=yes
review_due=2026-10-27

Checkpoint: Approve or roll back the canary

cat governance/canary-acceptance.txt

Continue whenThe bounded canary passes offline and online gates, rollback is proven, no data is reused without consent, and owners accept ongoing monitoring.

Stop whenStop on critical regression, policy or privacy complaint, memorization, unauthorized data use, error or latency breach, model drift, or failed rollback.

If this step fails

The candidate passes aggregate metrics but fails a safety or policy case.

Likely causeAverages hide a critical regression, the gate was treated as weighted, or evaluation omitted a required class.

Safe checks

Inspect per-case results and hard-gate status
Re-run the failing case with saved prompt and artifact identity
Review whether the test is valid and independent of training data

ResolutionReject promotion. Hard safety, privacy, licensing, and policy gates cannot be offset by lower loss or better average task score.

The API server accepts requests from an untrusted network.

Likely causeThe listener binds all interfaces, container port publishing bypasses expectations, authentication is absent, or a proxy forwards administrative credentials.

Safe checks

Inspect listening addresses and firewall rules
Test from an untrusted segment with synthetic input
Review TLS, authentication, token scope, and logs

ResolutionReturn to loopback or a protected network immediately. Add authenticated TLS ingress, least-privilege credentials, rate limits, audit, and an owner before intentional exposure.

Security notes

Do not place model-hub, object-storage, tracking, or infrastructure credentials in notebooks, scripts, configs, shell history, logs, checkpoints, model cards, or exported environments.
Use approved synthetic or minimized examples in diagnostics. Training artifacts can memorize sensitive content and should inherit access, retention, incident, and deletion controls from the source data.

Alternatives

Retain the base route and adapter as an offline experiment when canary evidence is incomplete or operational ownership is unavailable.

Stop conditions

Stop on critical regression, policy or privacy complaint, memorization, unauthorized data use, error or latency breach, model drift, or failed rollback.

Finish line

Verification checklist

Verify immutable inputs and audit evidence

python3 scripts/audit_dataset.py >/dev/null && sha256sum requirements-training.txt governance/dataset-provenance.json governance/release-contract.yaml

Environment, base, data, contract, splits, and evaluator identities match the accepted run manifest and the audit has no unresolved finding.

Verify candidate improvement and hard gates

python3 - <<'PY'
import json
base=json.load(open('evaluation/baseline.json'))['summary']
adapter=json.load(open('evaluation/adapter.json'))['summary']
delta=adapter['exact_normalized_rate']-base['exact_normalized_rate']
assert delta >= 0.08, delta
print(f'absolute improvement={delta:.4f}')
PY

The candidate exceeds the 0.08 absolute threshold and separate evidence shows every licensing, privacy, safety, memorization, compatibility, latency, and rollback gate passed.

Verify protected serving and fallback

curl -fsS -H "Authorization: Bearer $VLLM_API_KEY" http://127.0.0.1:8000/v1/models | python3 -m json.tool; grep rollback_tested=yes governance/canary-acceptance.txt

Only approved model aliases are visible to the credential, unauthenticated access fails, the adapter passes a representative request, and rollback to the base is tested.

Recovery guidance

Common problems and safe checks

The run fails with CUDA out of memory before the first optimizer step.

Likely causeThe model, sequence length, micro-batch, optimizer state, activations, allocator fragmentation, or another process exceeds available accelerator memory.

Safe checks

Record nvidia-smi memory and active processes before changing settings
Confirm effective sequence length and per-device micro-batch
Inspect whether quantization, gradient checkpointing, and intended precision actually activated

Training loss becomes NaN or infinite.

Likely causeLearning rate, precision, invalid labels, corrupt tokens, exploding gradients, an unsupported kernel, or a numerically unstable quantization path caused divergence.

Safe checks

Inspect the first non-finite step and preceding gradient norm
Run the same batch in full precision on a small model
Verify labels, attention masks, tokenizer IDs, and maximum token ID

Evaluation improves while representative answers become worse.

Likely causeThe held-out set is too similar to training data, the metric rewards surface form, prompt formatting differs, or the model memorized templates without improving the intended task.

Safe checks

Check document and near-duplicate leakage across splits
Review blinded task examples and safety cases
Compare baseline and candidate with the exact production template

ResolutionReject the candidate, repair split and rubric design, add adversarial and out-of-domain cases, and retrain only after the evaluation can detect the observed regression.

The resumed run does not match the pre-interruption learning curve.

Likely causeOptimizer, scheduler, scaler, random generator, sampler position, data revision, code, or world size was not restored from the same checkpoint contract.

Safe checks

Compare checkpoint manifest, code commit, dataset digest, and package lock
Confirm optimizer and scheduler files exist
Inspect global step, epoch, sampler, seed, and accelerator topology

ResolutionDo not splice incompatible histories. Resume only from a complete matching checkpoint or start a new run ID and preserve both lineages.

A checkpoint loads but generation is nonsensical.

Likely causeTokenizer files, special-token IDs, chat template, architecture config, adapter/base pairing, or weight files do not belong to the same immutable revision.

Safe checks

Compare tokenizer and config digests with the checkpoint manifest
Verify BOS, EOS, PAD, and vocabulary size
Run a fixed token round-trip and inspect missing or unexpected keys

ResolutionRestore the matching tokenizer, config, base revision, and adapter or checkpoint. Never patch incompatible IDs in place and call the result recovered.

Training is much slower than the cost estimate.

Likely causeTokenization, data loading, CPU contention, storage, precision, kernel fallback, gradient checkpointing, communication, or thermal/power limits differ from the measured pilot.

Safe checks

Separate data, forward, backward, optimizer, checkpoint, and evaluation time
Measure tokens per second after warm-up
Inspect GPU utilization, clocks, power, memory, and input pipeline stalls

ResolutionPause before consuming the full budget. Fix the bounded bottleneck, repeat a representative pilot, and update cost and completion estimates with measured throughput.

The dataset audit finds personal data, secrets, or credentials.

Likely causeCollection, export, issue text, logs, or generated examples entered the corpus without adequate minimization and scanning.

Safe checks

Quarantine the affected shard and record its digest
Identify source, scope, access, and downstream copies
Search tokenizer, caches, checkpoints, and artifacts according to incident policy

The dataset license or source permission cannot be proven.

Likely causeThe dataset card is incomplete, a mixed corpus lost per-source terms, redistribution differs from training permission, or an internal source lacks recorded authority.

Safe checks

Inspect the provenance manifest and source-specific license text
Confirm dataset revision and content digest
Escalate ambiguous terms to the appropriate legal or governance owner

ResolutionExclude the source and rebuild the corpus. Do not infer permission from public accessibility, a repository host, or another model's use of the same text.

Tokenized splits contain duplicates or overlapping documents.

Likely causeSplitting happened after chunking, document IDs were unstable, normalization collapsed examples, or near-duplicate detection was omitted.

Safe checks

Recompute exact hashes before tokenization
Check source document IDs across train, validation, and test
Run near-duplicate analysis at document and chunk levels

ResolutionSplit by stable source document before chunking, rebuild every derived shard, and invalidate evaluation produced from contaminated splits.

The GPU is visible but unsupported kernels or CPU fallback are used.

Likely causeDriver, CUDA, compute capability, package wheel, dtype, bitsandbytes backend, or attention implementation is incompatible.

Safe checks

Record torch and CUDA versions and device capability
Inspect runtime logs for backend and kernel selection
Run a tiny matrix operation and memory-footprint probe

ResolutionUse a documented compatible stack, rebuild the isolated environment, and rerun the measured pilot. Do not publish throughput from an unintended fallback.

Checkpoint storage fills during training.

Likely causeSave frequency, retention, optimizer state, temporary writes, or concurrent runs exceed the capacity plan.

Safe checks

Measure checkpoint size and free space
List complete and partial checkpoint directories
Compare save_total_limit and expected run duration

ResolutionPause safely, preserve the latest verified complete checkpoint and required lineage, remove only explicitly obsolete artifacts, and revise retention before resuming.

A model hub upload exposes a private artifact.

Likely causeRepository visibility, token scope, push settings, generated card, logs, dataset samples, or checkpoint contents were not reviewed before upload.

Safe checks

Stop the upload and inspect repository visibility
Revoke the token and audit uploaded revisions
Check model, tokenizer, adapter, card, logs, and dataset references

ResolutionQuarantine or remove the remote revision according to provider and incident policy, rotate credentials, and require an explicit publication gate instead of automatic push.

Baseline results cannot be reproduced.

Likely causeThe base revision, tokenizer, evaluation set, template, decoding fields, seed, runtime, or metric implementation changed.

Safe checks

Compare every digest in the run manifest
Run the fixed baseline command in the locked environment
Inspect metric version and exact test IDs

ResolutionRestore the immutable baseline environment or create a new baseline lineage. Never compare the candidate against a moving or undocumented reference.

The training process is killed or the host reboots.

Likely causePreemption, maintenance, kernel failure, out-of-memory killer, power loss, or scheduler timeout interrupted the process.

Safe checks

Inspect system and scheduler events without modifying checkpoints
Verify the newest checkpoint was atomically completed
Confirm code, data, topology, and package lock still match

ResolutionResume only from the newest verified complete checkpoint. If integrity or state compatibility fails, fall back to the previous checkpoint and preserve the broken artifact for diagnosis.

The adapter or trained model overfits and copies long training passages.

Likely causeThe corpus is too small or repetitive, epochs or capacity are excessive, validation misses memorization, or sensitive content was present.

Safe checks

Run held-out prefix and canary extraction tests
Compare train and validation loss trajectories
Inspect nearest training examples for suspicious generations

ResolutionReject publication, improve data and early stopping, reduce capacity or epochs, remove sensitive material, and rerun memorization evaluation before training again.

Inference output differs after merge, quantization, or serving conversion.

Likely causeThe conversion changed precision, tokenizer, template, adapter scaling, architecture metadata, or unsupported operations.

Safe checks

Run the identical evaluation before and after each transformation
Compare weight, tokenizer, config, and adapter manifests
Inspect conversion warnings and missing keys

ResolutionKeep the last accepted artifact, reject the transformed candidate, and change one conversion variable at a time. A smaller artifact is not acceptable without quality evidence.

The API server accepts requests from an untrusted network.

Likely causeThe listener binds all interfaces, container port publishing bypasses expectations, authentication is absent, or a proxy forwards administrative credentials.

Safe checks

Inspect listening addresses and firewall rules
Test from an untrusted segment with synthetic input
Review TLS, authentication, token scope, and logs

ResolutionReturn to loopback or a protected network immediately. Add authenticated TLS ingress, least-privilege credentials, rate limits, audit, and an owner before intentional exposure.

The candidate passes aggregate metrics but fails a safety or policy case.

Likely causeAverages hide a critical regression, the gate was treated as weighted, or evaluation omitted a required class.

Safe checks

Inspect per-case results and hard-gate status
Re-run the failing case with saved prompt and artifact identity
Review whether the test is valid and independent of training data

ResolutionReject promotion. Hard safety, privacy, licensing, and policy gates cannot be offset by lower loss or better average task score.

After the procedure

Alternatives and next steps

Consider these alternatives

Use retrieval for changing or source-grounded knowledge; it is easier to update and cite than encoding facts in adapter weights.
Use prompt templates, schema validation, deterministic code, or constrained decoding when the desired change is format or policy that does not require learning.
Use plain LoRA over bfloat16 base when four-bit kernels are unsupported and measured memory permits it; compare against the same baseline rather than assuming one method wins.
Use a managed, governed fine-tuning service when local infrastructure cannot meet security, reproducibility, availability, or staffing requirements, while retaining the same data and evaluation contract.

Operate it safely

Expand the evaluation corpus with independently authored out-of-domain, multilingual, adversarial, refusal, escalation, privacy, memorization, and long-context cases without leaking them into training.
Automate immutable data, code, package, model, tokenizer, checkpoint, adapter, and evaluation manifests plus signed promotion evidence; keep publication manual.
Measure adapter rank, target-module, data-quality, and sequence-length variants as separate preregistered runs rather than tuning repeatedly on the final test.
Create runbooks for dataset withdrawal, secret exposure, failed resume, accelerator loss, divergence, storage pressure, model rollback, and artifact quarantine.
Review base terms, official PEFT/TRL/Transformers/bitsandbytes/vLLM behavior, security controls, evaluation validity, and this guide no later than 2026-10-27.

Reference

Frequently asked questions

Does QLoRA train the whole model in four-bit precision?

No. The base is represented in a supported low-bit form to reduce memory while extra adapter parameters are trained using higher-precision computation. Only extra parameters are updated in this workflow.

Is QLoRA always better than LoRA?

No. QLoRA can fit a larger base into limited memory, while plain LoRA may offer a simpler precision path when memory permits. Compare them under the same data, task, base, and evaluation.

Why use a small 0.6B model?

It makes the complete pipeline feasible on one workstation and keeps failure cost bounded. The guide teaches governance and operations; it does not promise that this model size meets every quality requirement.

Does lower validation loss prove the adapter is better?

No. It is one training signal. Final acceptance uses an untouched task test, per-case regressions, human review, safety, privacy, memorization, latency, compatibility, and policy gates.

Can test cases be used to choose rank or epochs?

No. That turns the test into validation. Use training and validation for controlled selection, then evaluate the chosen candidate once on the frozen test under the release process.

Should the adapter be merged?

Only when a target runtime requires standalone weights or a measured operational benefit justifies it. Keep the canonical adapter and re-evaluate the merged artifact fully.

Is the QLoRA base already suitable for serving?

Not necessarily. Training quantization and serving formats are distinct. Treat any merge, conversion, or inference quantization as a new artifact with compatibility and quality evaluation.

Can public issue text be used as training data?

Public visibility is not permission. Review copyright, license, terms, personal data, secrets, user expectations, retention, and intended use before including any source.

What should be stored in the model card?

Base and adapter identity, intended scope, data lineage at an approved level, training method and budget, evaluation and limitations, safety and privacy controls, license, prohibited uses, owners, and review date.

Can failed runs be deleted?

Retain enough manifest, metrics, cost, and failure evidence to avoid repeating unsafe or expensive experiments. Remove bulky checkpoints according to explicit retention after confirming no recovery, audit, legal, or incident need.

Does an open-weight model mean the derivative may be published?

No. Review model license, acceptable-use terms, dataset permissions, adapter or merged redistribution, attribution, privacy, and destination visibility separately.

How often should this pipeline be reviewed?

At least every ninety days, next due 2026-10-27, and immediately after changes to base model, data, tokenizer, packages, hardware, serving, security policy, metrics, task, or artifact destination.

Recovery

Rollback

Stop canary routing to the adapter, direct clients to the frozen accepted base, verify baseline behavior and authentication, revoke the adapter alias and credentials if needed, preserve evidence, and remove derived artifacts only after retention and incident review.

Set adapter traffic to zero and route the same authenticated client contract to the pinned base revision. Verify representative task, refusal, latency, and readiness checks.
Stop the adapter-serving process or unregister support-r1 while keeping the base service available. Rotate any credential or ingress policy implicated in the rollback.
Mark adapter, merged, and quantized candidates rejected or superseded without overwriting their manifests. Preserve audit, evaluation, canary, cost, and incident evidence under restricted access.
Remove disposable pilot checkpoints and caches only after confirming no recovery, audit, legal-hold, or deletion obligation requires them. Delete data and artifacts through the recorded governance process rather than ad hoc shell cleanup.

Evidence

Sources and review

Verified 2026-07-29Review due 2026-10-27

PEFT LoRA configuration and QLoRA-style all-linear adaptersofficial TRL SFTTrainer and PEFT integrationofficial Transformers bitsandbytes quantization and hardware supportofficial Transformers fine-tuning and TrainingArgumentsofficial Hugging Face dataset cards and license metadataofficial PyTorch reproducibility guidanceofficial vLLM LoRA adapter servingofficial vLLM OpenAI-compatible servingofficial