ENGINEERING / WORKFLOW
From samples to adapters: SFT + LoRA
Follow ten modules through data gates, supervised training, low-rank adaptation and independent evaluation.
Educational simulation: examples are not training data. Playback does not measure runtime or predict model quality or performance.
Tab to nodes or controls, Enter to activate; use ← / → and Home / End on player controls. Playback is a teaching sequence, not runtime.
A thick border marks the current step, dashed borders mark visited steps, and thin borders mark pending steps. Arrows show the teaching data flow.
PT01
Dataset Preparation
Reviewed inputs and a target Python file form one sample.
- Input
- Fictional scenario: an invalid request must preserve state.
- Operation
- Separate rules, scenario, API, framework and reference from the target; record source_group and approval.
- Output
- A traceable, approved training candidate.
- Why it matters
- The target is supervision, never an inference-time input.
Before: Start → After: Data Audit
PT02
Data Audit
Structural checks precede training; business correctness still needs review.
- Input
- Candidate rows and optional frozen benchmark manifests.
- Operation
- Validate required fields, split/group isolation, approval, duplicate targets and Python syntax. Check benchmark exclusion when configured.
- Output
- Validated rows; malformed or overlapping data is rejected.
- Why it matters
- An empty exclude_manifests list does not prove benchmark isolation.
Before: Dataset Preparation → After: Prompt Construction
PT03
Prompt Construction
Training and generation share common.messages().
- Input
- The five input sections, without the target.
- Operation
- Construct system and user messages; apply the locked tokenizer chat template.
- Output
- The prompt prefix used by encoding and generation.
- Why it matters
- A shared template prevents training and inference format drift.
Before: Data Audit → After: Tokenization & Label Mask
PT04
Tokenization & Label Mask
Prompt tokens condition predictions but are excluded from direct loss.
- Input
- Prompt plus assistant target.
- Operation
- Verify the prompt prefix, encode the complete sequence and mask prompt labels with -100. Reject overlong sequences instead of truncating.
- Output
- input_ids, attention_mask and completion-only labels.
- Why it matters
- Every sample must retain supervised tokens and a correct answer boundary.
Before: Prompt Construction → After: Forward Pass & SFT Loss
PT05
Forward Pass & SFT Loss
Trainer invokes the causal language model; train.py does not implement a custom forward loop.
- Input
- Encoded sequences and masked labels.
- Operation
- Teacher forcing predicts the next token. Conceptually, L = −Σ log P(y_t | x, y_<t) / T over valid target positions; the model shifts logits and labels.
- Output
- A scalar supervised loss and an autograd graph.
- Why it matters
- Token loss supplies a training signal; it does not establish business correctness.
Before: Tokenization & Label Mask → After: Backward & Gradient Accumulation
PT06
Backward & Gradient Accumulation
Trainer/PyTorch manages differentiation and accumulation.
- Input
- Loss for successive micro-batches.
- Operation
- Backpropagate into trainable LoRA parameters; accumulate gradients until an optimizer update is due. The project fixes micro-batch size to 1.
- Output
- Accumulated gradients for the optimizer.
- Why it matters
- A micro-batch forward/backward is not necessarily one optimizer step.
Before: Forward Pass & SFT Loss → After: AdamW & LR Scheduler
PT07
AdamW & LR Scheduler
The project selects adamw_torch through TrainingArguments.
- Input
- Accumulated gradients, moment estimates and current learning rate.
- Operation
- AdamW updates trainable parameters, then the scheduler advances and gradients are cleared. Transformers 4.57.1 defaults to linear decay with no warmup here.
- Output
- Updated LoRA weights and the next learning rate.
- Why it matters
- Changing max_steps changes both training duration and the linear decay trajectory.
Before: Backward & Gradient Accumulation → After: LoRA Parameter Update & Adapter
PT08
LoRA Parameter Update & Adapter
LoRA is attached before training; this step inspects the update and final saved artifact.
- Input
- Frozen base W and trainable A, B on q_proj/v_proj.
- Operation
- Use W + (α/r)BA. After trainer.train(), verify a nonzero adapter change and save adapter plus tokenizer.
- Output
- An adapter paired with a locked base model revision.
- Why it matters
- LoRA reduces trainable parameters, but base weights and activations still occupy memory.
Before: AdamW & LR Scheduler → After: B0/B1 Generation
PT09
B0/B1 Generation
B0 uses the base model; B1 loads the compatible adapter.
- Input
- The same frozen benchmark prompts and generation settings.
- Operation
- Use generate_items with greedy decoding and inference_mode; check adapter/base revisions. max_new_tokens limits new output.
- Output
- Candidate scripts and generation records; no generated code is executed.
- Why it matters
- Matched settings make the comparison interpretable; generating code does not validate it.
Before: LoRA Parameter Update & Adapter → After: Benchmark Evaluation
PT10
Benchmark Evaluation
Automatic layers and independent execution/review answer different questions.
- Input
- B0/B1 candidates on the same suite, rules and scorer version.
- Operation
- Evaluate L0 form, L1 framework, L2 grounding and L2.5 oracle. L3 execution and L4 review are separate. Compare matched tasks.
- Output
- Layered evidence and paired changes, not a fabricated accuracy score.
- Why it matters
- Lower training loss alone does not establish useful or generalizable scripts.
Before: B0/B1 Generation → After: End of walkthrough
POST-TRAINING / PARAMETER LAB
Parameter laboratory
Deterministic teaching calculations; no model training or project config changes. N=56, matrix dimensions and token counts are assumptions. Results are not measured loss, accuracy, runtime or memory.
The 30/112 buttons change max_steps and bring an out-of-budget cursor or warmup back into range; other choices remain unchanged.
Training length check: Pass · Detailed formulas and curves ↓
Bounds: finite numbers, integer counts, warmup < max_steps, current step ≤ max_steps and r ≤ min(d_in,d_out). An overlong training sequence is shown as a rejection example.
Sample exposure plan
B_eff = b × g × w = 1
exposures ≈ S × B_eff = 30
epochs ≈ exposures / N = 0.5357
This plans full batches approximately. Partial final batches, accumulation across epochs and distributed sampler padding can change actual exposures; this is not a per-sample run log.
LoRA: one hypothetical linear layer
ΔW = (α/r)BA · scale = α/r = 2
P = r(d_in + d_out) = 16,384 parameters
W: d_out × d_in = 1,048,576 · 1.5625%
A: 8 × 1024; B: 1024 × 8; BA: 1024 × 1024
The percentage compares this layer's adapter and full matrix, not whole-model savings. Base weights and activations still occupy memory; larger rank does not guarantee better independent evaluation.
Learning-rate curve, not training loss
η(s) = η₀·s/W (s < W); then η₀·max(0, (S−s)/(S−W))
LR after current update: 5.0000e-5
Solid: selected plan; dashed: 30 steps / warmup=0; dotted: 112 steps / warmup=0. All use the same η₀ and update-count x axis. Matching plans overlap; zero extensions beyond each budget aid comparison and do not imply continued training.
Changing max_steps also changes decay progress at the same update count. These curves are formulas, not logs; the cursor denotes completed updates.
Training labels and generation budget
Bar left: prompt (label=-100); right: supervised suffix. Lengths are synthetic counts, not real tokenization.
Training sequence length: 1,280 · Passes length check
Generation prompt check: Passes · Prompt + new-token budget upper bound: 3,072
Overlong training samples are rejected, not truncated. Prompt tokens still participate in forward but not directly in loss. Generation also depends on EOS and actual model context limits; output need not reach max_new_tokens.