完整研究叙述

NARRATIVE_REPORT — Behavioural fingerprints of agent distillation

v1.1 (2026-09-02): corrected after the result-to-claim gate (CLAIMS_FROM_RESULTS.md §3) and re-positioned after the novelty gate (NOVELTY_REPORT.md). Gate verdicts and accepted wordings are in §7.

Working title candidates: *Agentic SFT Clones the Process, Not the Capability* · *Habits Are Cheap, Capability Is Not: Fingerprinting Agent Distillation* · *Habits, Not Answers*. Target: ACL Rolling Review, October 2026 cycle (deadline 2026-10-12 AoE); ACL 2027 / Findings. Paper type: empirical analysis / measurement; not a method paper and it does not advocate the corpus. Source of every number: the files listed in §6. All twelve evaluation arms are final as they stand (no reruns are planned; infrastructure-caused reruns are not part of the paper).

0. One-sentence contribution

Supervised fine-tuning on 8,137 trajectories of a frontier coding agent (Claude Opus 4.8 running Claude Code) transfers the teacher's task-independent *habits* — command vocabulary, error-recovery reflexes, thinking rhythm, failure-ending phrasing — within a single epoch and halves cumulative input tokens, while producing no detectable solve-rate gain on already post-trained bases (below the paired detection floor on SWE-bench Pro/Verified) and +8.9 pp on a bare base; behavioural fingerprints, not pass@1, are therefore the sensitive read-out of what distillation transferred.

Spine of the paper (post-gate): what agentic SFT buys is rapid behavioural and input-token convergence, while capability gains depend on the starting model. "Pass@1 hides process" is motivation (anticipated by arXiv:2607.06184), not a claim.

1. Setup

• Teacher. Claude Opus 4.8 inside Claude Code; 32,830 captured SWE-fix trajectories from our data-generation pipeline (delivery/v5); 32,583 carry a pass/fail verdict and form three populations: 12,257 verified-pass "high-precision" (→ corpus v12), 9,843 by-product passes, 10,483 by-product failures (→ corpus v13 only, with a regex mask on strong false-success conclusions); the remaining 247 are by-product runs whose manifest status is error (neither passed nor failed) and belong to no group.
• Corpora. v12 = 8,137 filtered high-precision trajectories (train split; 373 further tasks held out in val_ms_swift_262144.jsonl, 0 overlap with train). v13 = 9,880 high-precision + 14,081 by-product trajectories (masked failed conclusions), 1 epoch.
• Students / arms (12). Dense line: Qwen3.6-27B native (n27); +v12 ×1/×2/×3 epochs (s128/s256/s381); +v13 ×1 (v13). MoE line (35B total / 3B active): Qwen3.5-35B-A3B-Base (a3b_base) and +v12 ×2 (a3b_2ep); Qwen3.6-35B-A3B native (a3b_q36) and +v12 ×3 (a3b_v12), each with a temperature-0.6 rerun on Pro (a3b_q36t06, a3b_v12t06). Vendor next generation: Qwen3.8-27B native (q38), temperature 0.8 like all other arms (official is 1.0).
• Evaluation. SWE-bench Pro (731 tasks, 4 languages) and SWE-bench Verified (500, Python), pass@1, one seed, identical Claude Code 2.1.202 harness, 5,400 s wall, 262k context, top_p 0.95 / top_k 20, temperature 0.8; task pool is the denominator (anomalies count as unsolved); paired McNemar exact tests; a golden-patch leak audit with negative controls (three channels: network, local git history, package manager) — conclusions unchanged on the clean subsets.
• Fingerprint extractor. One extractor converts teacher trajectory.json and student Claude Code session jsonl into the same step sequence (46,542 trajectories). Per-arm distributions are trajectory-weighted; distance to the teacher = Jensen–Shannon divergence (log2) over four rulers: tool mix (11 buckets), tool bigrams, Bash command-head vocabulary, thinking-position deciles; composite = mean of mix and bigram. Three gates were green (teacher self-consistency 32,830/32,830; student reconciliation with the score ledger 13,676 trials; overclaim anchor 44.6 % vs 46.3 % known). Teacher-internal noise floor (teacher sub-populations vs each other): 0.002–0.003.
• Anchors and audits. (a) 88 tasks that are literally in the v12 training set (teacher solutions trained on), replayed by n27 and s256 with exactly 3 rollouts per task (263 and 296 parsed trajectories: one void n27 trial; the s256 job also covers the 12 oracle-filtered tasks and has 4 void trials; per-task medians). (b) 240 failure endings annotated one by one (model annotator; human re-annotation planned) for the SUCCESS_CLAIM regex, and 68 RATIONALIZE hits. (c) Two temperature-0.6 reruns on Pro as the run-to-run noise measurement. (d) Cost decomposition: cumulative input = steps × per-step context; observation sources split into whole-file reads / limited reads / commands with and without truncation pipes.

2. Claim ledger

Status: HEADLINE / SUPPORTING / APPENDIX / DROP. "Fix" = analysis still to do before submission (no GPU unless stated).

H1 — Habit-layer convergence saturates after one epoch (HEADLINE)

• Composite JS to teacher on Pro: n27 0.0285 → s128 0.0226 → s256 0.0215 → s381 0.0213 (82 % of the total drop happens in epoch 1; epochs 2–3 are inside each other's bootstrap CIs). Bash-vocabulary JS: 0.202 → 0.069 → 0.053 → 0.050. MoE line: composite 0.0347 → 0.0211, bash 0.195 → 0.061 (a3b_q36 → a3b_v12); bare base 0.278 → 0.057 (a3b_base → a3b_2ep). Thinking-position JS: 0.0162 → 0.0085–0.0110.
• Holds inside each language on Pro (bash JS Python 0.341→0.133, Go 0.217→0.066, JS/TS 0.237→0.105 for n27→s256), and with a teacher Python-only reference (n = 574) on the Python slice (composite n27 0.0267 → s256 0.0195 → s381 0.0178).
• Two layers: command habits and thinking position converge in both pools; the tool-mix/bigram "strategy" layer does not converge on Verified (composite 0.0369 → 0.0465 with the Python reference) — tool composition follows task structure. This split is exploratory until validated on a held-out pool.
• Pool confound is larger than the residual dose effect: on the 88 same-task anchor s256's composite is 0.0090 vs 0.0215 on Pro; n27 is 0.0493 vs 0.0285 (not additive). Distances are therefore only comparable within a pool.
• Done (analysis/fp/fp_ci.md, internal check; intervals are not shown in the paper): as *differences* between consecutive arms on Pro, n27→s128 is above noise on bash (−0.133), bigram (−0.010), thinking position (−0.008) and composite (−0.006) but not on mix; s128→s256 only on bash (−0.016, ≈7× the same-weights floor); s256→s381 on nothing. So "one epoch does most of it" is a difference statement, and the residual epoch-2 effect exists only in the Bash vocabulary. Hellinger preserves every JS ordering and the habit/strategy split; total variation does not preserve the Pro tool-mix ordering because the SFT arms overshoot the teacher's Bash share (teacher 50.7 %, n27 54.5 %, SFT arms 60–62 %) — tool-mix "convergence" is ruler-dependent, Bash vocabulary and thinking position are not. Same-weights floor (temperature twins, n ≈ 730): composite 0.0008 / 0.0003, bash 0.0021 / 0.0011; split-half within an arm: composite ≈ 0.0008, bash 0.004–0.005 (a3b_q36 bash 0.015: heterogeneous vocabulary). The "teacher-internal 0.002–0.003" figure is the distance between teacher sub-populations, not sampling noise; the sampling floor at student sample size is ≈ 0.001 composite / 0.002–0.005 bash. Paper wording: differences are quoted against these floors, not as intervals.
• Still to do: held-out anchor — replay the 373 val tasks (teacher trajectories exist, never trained on) with n27/s128/s256/s381/v13/q38 (GPU inference only).
• Files: bench/scripts/paper/out/fp_summary.json, fp_summary.md, lang_robustness.json, p8_train100.json.

H2 — The same fingerprint collapse accompanies +8.9 pp on a bare base and ≈0 on post-trained bases; a vendor generation jump buys +5.6–5.8 pp at 2.9× the cost (HEADLINE)

• Corpus on a bare base: a3b_2ep vs a3b_base, Pro +8.89 pp [+5.51, +12.28], p = 4.5e-7 (b = 115, c = 50); Verified +3.4 pp [−0.76, +7.56], p = 0.13.
• Corpus on post-trained bases (12-arm unified accounting, report.json): Pro — s128 −0.41 (p = 0.84), s256 +2.74 [+0.09, +5.38] (p = 0.054), s381 +0.96 (p = 0.57), v13 +2.19 (p = 0.14) vs n27; a3b_v12 +2.74 [−0.12, +5.59] (p = 0.075) vs a3b_q36, temperature-0.6 pair +2.46 (p = 0.12). Verified — s128 −0.6 (p = 0.82), s256 +0.2 (p = 1.0), s381 +2.8 [−0.61, +6.21] (p = 0.14), v13 −2.2 (p = 0.26) vs n27; a3b_v12 vs a3b_q36 0.0 (b = c = 45, p = 1.0). By exact McNemar p < 0.05 the exceptions are v13 vs s381 on Verified (−5.0 pp, p = 0.0046, worse) and s256 vs s128 on Pro (+3.15 pp, p = 0.014); s256 vs n27 on Pro is p = 0.054. After Holm over the pre-declared family of 13 baseline-relative corpus tests only a3b_2ep vs a3b_base survives (Holm p = 5.9e-6); under Holm over all 111 valid tests neither exception survives (they survive BH only). Generation/architecture family (6 tests): 5 survive Holm (q38 vs n27 both pools, n27 vs a3b_q36 both pools, s381 vs a3b_v12 Verified). Details: analysis/stats/stats_fixes.md. The best checkpoint differs by pool (s256 on Pro, s381 on Verified). Failure to reject is not "≈0": the intervals still permit several-point effects.
• Generation jump: q38 vs n27, Pro +5.61 pp [+2.84, +8.38], p = 1.1e-4; Verified +5.8 pp [+2.22, +9.38], p = 0.0022; both pools same sign, gains spread over all four languages. Cost: cumulative input 4.41 M tokens/task vs 3.27 M (n27) and 1.52 M (s256) — 2.9× the distilled arm; cumulative output 46.8 k vs 13.7 k.
• Dense vs sparse, same generation: n27 vs a3b_q36 Pro +3.97 pp (p = 0.008), Verified +5.4 pp (p = 0.009); after the same corpus, s381 vs a3b_v12 Pro +2.19 (p = 0.14), Verified +8.2 pp (p = 1.3e-5). Behaviour follows the recipe, score follows the architecture. (Appendix.)
• Reading: identical fingerprint convergence whether the score moves (+8.9, bare base) or not (post-trained); fingerprint distance is uninformative about capability; capability gains depend on the starting model. Interpretation we can defend (from the 2026-09-01 external report, 经验/对外分享/sft20260901训练-评测报告-35B-A3B.md): the corpus supplies the agentic post-training step itself — on the bare base it lifts Pro to 324/731, within 1.64 pp of the vendor's own next-generation post-trained model (336, p = 0.36), while on Verified the remaining 7.6 pp gap is the base-generation gap (the same corpus on the post-trained base lands exactly on 315 = native). No Qwen3.6-35B-A3B base is released, so "corpus vs vendor post-training from the same base" cannot be run; the bare-3.5 pair is the strongest available evidence. Micro-skills (S2) do transfer without cashing out as solve rate.
• Decontamination (done, analysis/stats/stats_fixes.md §e): instance level 0 title matches, max description Jaccard 0.21, 0 commit-SHA hits; Verified repos have 0 overlap with the corpus. Repo level: teleport has 1,622 teacher trajectories / 1,309 PRs and 76/731 Pro tasks. On Pro minus teleport (N = 655) vs n27: s128 −1.22, s256 +2.60 (p = 0.086), s381 −0.15 (p = 1.0; its full-Pro +0.96 was all teleport), v13 +1.98 (p = 0.22); a3b_v12 vs a3b_q36 +3.05 (p = 0.054); a3b_2ep vs a3b_base +9.62 (p = 1.7e-7, the only Holm survivor). On teleport alone (N = 76) every v12/v13 checkpoint beats n27 by +3.9 to +10.5 pp (s381: 8 vs 0 discordant, p = 0.008, Holm 0.055) — a repo-level lift consistent with corpus exposure, within the noise of a 76-task slice (the temperature twins flip 9/76 there). The paper states this as a repo-family effect and reports the teleport-excluded numbers alongside.
• Files: paper/distill-fingerprint/data/report.json (arms, tests), data/20260901-q38-*-定稿.json, three-line report page.

H3 — The input-token footprint is part of the fingerprint (HEADLINE)

• Cumulative input per task (Pro medians): n27 3.27 M → s256 1.52 M (−53 %: steps −32 % × per-step context −29 %); MoE line 3.27 M → 2.1 M (−12 % / −25 %); all dense-line v12/v13 checkpoints land at 1.52–1.80 M (MoE line 2.1 M), ranking unrelated to score ranking (s256 386, s381 373, v13 382 on Pro).
• Mechanism (A3B line, replicated at both temperatures): whole-file reads 7,293 → 2,578 (−65 %) at unchanged size per read (8,975 → 8,516 chars); reads with limit 37 % → 64 %; commands with truncation pipes 47 % → 68 % (fp_pooled 0.471 → 0.683; teacher pooled 0.672; dense line n27 0.432 → s256 0.649); observation ≥ 20 k chars −72 %; 78 % of the MoE-line drop is whole-file reads (dense line: 74 % of the 65.1 M-character drop, 27B page DROPSRC and figures/fig5_data.json, 2026-09-02).
• Other side: cumulative output rises (n27 13.7 k → s256 15.9 k; a3b_q36 14.1 k → a3b_v12 21.1 k) — the thinking overshoot of S3; in the concurrency-matched temperature-0.6 pair wall-clock is −1 % (978 → 968 s). Savings are billing-model dependent: token billing ≫ 0, GPU-hour billing ≈ 0. Token reduction is not automatically cost reduction under prompt caching (arXiv:2607.12161); the paper reports the input-token ledger and says so.
• Files: report.json arms (steps_med, ctx_last_med, obs_sum_med, out_sum_med, sec_med), A3B report page (figures 9–11), 27B report page (figure 3), fp_pooled.json (trunc_pipe, read_limit).

H4 — Measurement: pass@1 is under-powered here, behavioural endpoints are reliable, regex "honesty" is not (HEADLINE, methods)

• Run-to-run: the same weights at temperature 0.8 vs 0.6 flip 97/731 (a3b_q36) and 83/731 (a3b_v12) tasks — 13.3 % / 11.4 % — for net deltas of +0.96 (p = 0.54) and +0.68 pp (p = 0.66). With 3 rollouts, 36–42 % of Pro tasks give inconsistent verdicts. The stratified rollout×3 re-estimate of s256 vs n27 is +1.40 / +2.75 pp (not distinguishable from 0). Re-derived (analysis/stats/stats_fixes.md §c): from the observed discordance of the baseline-relative pairs, the one-rollout paired design detects (80 % power, two-sided 0.05) a median 4.03 pp on Pro (range 3.77–4.83) and 5.04 pp on Verified (4.87–5.94); the previously quoted 4.6 pp is the k→∞ value at N = 237 (the stratified subsample); at N = 731 the decomposition gives 2.63 (k→∞) / 3.25 (k = 8) / 4.07 (k = 3) / 5.99 (k = 1) pp; ≈1,322 Pro tasks would be needed to resolve 3 pp at k = 1. Temperature twins on Pro: a3b_q36 97/731 flips (13.3 %), net +0.96 pp (p = 0.54); a3b_v12 83/731 (11.4 %), +0.68 pp (p = 0.66); flip rate by language Go ≈9–10 % to JavaScript 20–25 %.
• Behavioural endpoints: to be quantified as test–retest reliability on the two temperature-0.6 pairs (currently asserted, not measured).
• Failure-ending wording: strong-form success claims among unsolved Pro trajectories n27 238/359 = 66.3 % → s128 52.9 % → s256 49.1 % → s381 48.9 % → v13 44.1 %; teacher failures 4,676/10,483 = 44.6 %; q38 24.1 %. The n27→s128 drop is real (z ≈ 3.7); v13 vs s381 is not (z ≈ 1.3) — "v13 tuned precisely to the teacher" is over-reading. On Verified the line is non-monotone and 4× lower (14.9 / 18.2 / 21.2 / 20.0 / 10.3 %) — pool-dependent. Regex precision 69/70. Of the 110 regex-negative endings sampled (the annotation file's own summary says 109; the one-count gap goes to the human re-annotation), 0 admit failure (95 % upper bound ≈ 3.3 %): teacher and every student assert completion when they have failed; the arms differ in phrasing only. The v13 mask therefore produced no measurable honesty change and cost −5.0 pp on Verified vs s381.
• RATIONALIZE regex (68 hits): 48 are evidence-backed attributions (git-stash / pristine re-run), 12 bare dismissals, 8 false positives; share with verification n27 2/8, s256 12/15, teacher 10/15 — a verification habit, not blame-shifting. (Descriptive; needs human labels.)
• Second annotator (Codex, blind, 2026-09-02; analysis/annot/): 0/110 admissions among regex-negative endings, precision 69/70 (same false positive), verified-attribution share n27 2/8 · s256 13/15 · teacher 11/15 — agrees with annotator 1 on every quantity the paper uses; the two differ only on one q38 truncation and on the bare-dismissal vs false-positive split of the remaining RATIONALIZE hits (6 vs 12).
• Fixes: human re-annotation of 240 + 68 samples by two annotators with κ (still planned); report Pro and Verified side by side; drop "data modulates wording".
• Files: fp_summary.json (groups[*].overclaim), overclaim_annotation.json, annot_batch1/2.txt, annot_rat.txt, report.json tests (temperature pairs).

S1 — Strong process imitation, imprecise solve-rate effect on 88 trained tasks (SUPPORTING)

• Per-task medians: read-file-set Jaccard to the teacher 0.60 (s256) vs 0.44 (n27), sign test 66:15; tool-mix L1 0.219 vs 0.471, 77:11; edit-set Jaccard 0.75 vs 0.65, 44:24 (p = 0.021); thinking share 37:51 (p = 0.165; s256 0.889 overshoots teacher 0.815). Steps: teacher 45, s256 62, n27 66.
• Solve rate (k/3 mean): 61.6 % vs 55.3 %, 24:13 tasks, p ≈ 0.10; medians both 2/3; stratified estimate +6.1 pp [−0.1, +12.2].
• Reading: with n = 88 the minimum detectable effect is ≈15 pp, so this cannot rule out a memorisation effect of that size; the point estimate (+6 pp) is 2–4× the test-set gain. The safe statement: the imitation effect is large and significant, the memorisation effect is small and not significant.
• Diagnostics done (analysis/fp/anchor_diag.md): exactly 3 rollouts per task in both jobs (263 vs 296 = one void n27 trial vs the s256 job covering all 100 sampled tasks minus 4 void trials; the 0.5 solve fraction is one task with a void trial). s256 reads *fewer* files (median 7 vs 11; teacher 6) but keeps the teacher's: precision of the read set against the teacher's 0.50 → 0.80 (75:8 tasks), recall 0.86 → 0.80 (18:30, n.s.), Jaccard 0.44 → 0.60 (66:15); negative controls — Jaccard against another task's teacher read set 0.001 for both arms, against a same-repo different task 0.029 vs 0.032 (45:38, p = 0.51); the Jaccard gain survives on the 15 tasks where s256 reads as many or more files (+0.054, 8:3). Edit sets 0.65 → 0.75 (44:24), controls ≤ 0.03. So "Jaccard rises because SFT reads less" is excluded: the student reads fewer files and they are the teacher's files. Δsolve vs Δread-Jaccard across the 88 tasks: Spearman ρ = −0.12 (imitating more does not predict solving more). The two p-values: 0.073 = exact sign test on per-task k/3 means with void trials counted as unsolved (25:13:50), 0.099 = same test with void trials dropped (24:13:51); the whole gap is one task. The earlier "stratified estimate" wording was wrong: it is a plain task bootstrap of the mean difference, Δ +6.1 pp.
• Still to do: the held-out anchor (H1) removes the memorisation question.
• Files: p8_train100.json, analysis/fp/anchor_diag.json.

S2 — Error handling converges as a bundle (SUPPORTING, one row-group of the "what moved" figure)

• Protocol-error recovery within 3 calls (same tool, same target, success): teacher 0.603; n27 0.299, a3b_q36 0.329; s128 0.605, s256 0.580, s381 0.625, v13 0.599, a3b_v12 0.593; q38 0.338 (= untrained level).
• Post-error thinking (chars per step after an error vs other steps, Pro means; analysis/fp/fp_ci.md §e): teacher 248 vs 726 (ratio 0.34); n27 373 vs 359 (1.04); s256 580 vs 1,199 (0.48); s381 709 vs 1,281; q38 1,222 vs 2,104 (0.58); a3b_q36 393 vs 401; a3b_v12 661 vs 1,345. After SFT the student thinks *more* after an error in absolute terms (373 → 580) and the ratio falls only because ordinary-step thinking triples; what transfers is the teacher's *pattern* (less thinking after errors than on ordinary steps), not a reduction in post-error thinking. The paper reports both numbers.
• Next action after an error: teacher 1 % verbatim retry, 62 % parameter change, 38 % tool switch; s256 63/37; n27 53/46. This is the same measurement as recovery@3 seen from another angle — report once.
• Edit rework (editing what one just wrote): n27 0.036 → s256 0.025 (teacher 0.017); a3b_q36 0.050 → a3b_v12 0.039; q38 0.012 (discounted: it prefers whole-file Write).
• Files: fp_pooled.json (recovery3, react_ratio, rework), fp_summary.json (groups[*].err_next).

S3 — Thinking quantity overshoots the teacher, thinking position is copied (SUPPORTING)

• Thinking share of self-generated characters (pooled): teacher 0.893; corpus v12 itself 0.8805 (278.0 M thinking vs 37.7 M text chars over 425,876 loss-bearing assistant rows); students 0.925 (s256) – 0.946 (v13); q38 0.952. Thinking-position decile profile converges (JS 0.0162 → 0.0085). Same-task anchor: s256 0.889 vs teacher 0.815.
• Under-shoot channels exist too: task-management tool calls teacher 3.2 % → s256 0.28 % (n27 5.2 %).
• The "88 % of learnable characters are thinking, so gradient goes to thinking" mechanism is a hypothesis without an ablation → not claimed.
• Files: think_overshoot.json, fp_summary.json (dists.js_thinkpos, groups[*].mix).

S4 — The vendor's next-generation post-training reaches some teacher dimensions and not others (SUPPORTING, descriptive only)

• q38 vs teacher: bash-vocabulary JS 0.061 and thinking-position JS 0.010 inside the distilled band; truncation pipes 0.667; sub-agent share 0 %; but tool-mix/bigram JS 0.031/0.056 (farther than n27), recovery 0.338, write-without-read 0.177 (teacher 0.110, distilled 0.086–0.092), read-with-limit 0.360 (teacher 0.651), whole-file Write 22 % of writes (teacher 8.5 %). Cost 4.41 M input/task.
• n = 1 model, unknown training data, different tool surface, temperature 0.8 vs official 1.0 → no causal or "convergent evolution" wording; a descriptive paragraph plus table rows.
• Files: fp_summary.md distance table, fp_pooled.json.

S5 — Channel use changes in a base-dependent way (DESCRIPTIVE ONLY; judge verdict: no)

• Genuine example: reading local git history (absent from the corpus; forbidden by the task instruction) drops 140 → 92 trajectories (−34 %) on the A3B line at both temperatures (one pool, one line).
• Sub-agents: the teacher is not at 0 % — 0.28 % of its tool calls dispatch an agent; s256 0.27 % (copied), n27 1.07 %, a3b_q36 ≈ 20 % of calls executed inside sub-agents, q38 0 %. So sub-agent usage is a copied low rate, not a pruned absent channel, and the vendor model reaches 0 % without our corpus. Verify whether the teacher capture includes side-chain sessions before quoting any teacher sub-agent number.
• Files: fp_summary.json (groups[*].mix bucket 子代理, scalars.sidechain_share), report.json ledger (side|calls, tool|Agent|calls), A3B report leak section.

X1–X3 (APPENDIX)

• X1 teacher failures are "behavioural": failed trajectories have ~2× rounds (median 128 vs 76–102), output and thinking tokens, with unchanged tool mix/habits (JS 0.002–0.003 between teacher sub-populations = the noise floor).
• X2 dense vs sparse; unsolved-group regression rate follows capability (a3b_base 37.7 % → a3b_v12 27.1 %, n27 24.0 %, q38 16.8 %; solved-group regression is 0 by construction); context compaction fired in 52/13,733 trials (0.4 %) — not a confound; protocol-error taxonomy; edit rework.
• X3 "write without reading" is a habit the corpus stably teaches (both bases, both temperatures: 138/133 vs 72/23).
• DROP: "solved-group regression 0 %" as a merit; "−30 % wall-clock" (network-path confounded); infrastructure recommendations; temperature-0.6 arms as table rows (they live only in H4).

2b. Data-curation claims (added 2026-09-02 after the user made SFT data cleaning the second innovation line; pipeline details are being compiled in analysis/data/data_pipeline.md)

D1 — The corpus is a curated subset, and the curation rules are the method (HEADLINE candidate, pending pipeline write-up)

• Funnel (from analysis/data/data_pipeline.md, recomputed from train/data/swepro-v12-20260806-qwen36-27b/raw/build_report.json + decisions.tsv): 12,257 high-precision verified passes (hidden-test verification: candidate patch + gold test patch, every required F2P/P2P test must pass; plus the vendor quality audit) → 12,036 after a vendor removal of answer-source fetches (221) → drop 3,435 (28.5 %) by the P0 rules → 8,601 → same-task cap ≤ 2 (−39) → 8,562 → task-level split 8,137 train (7,054 tasks) / 425 val (373 tasks) → 262,144-token cap removes 0, 196,608-token cap removes 17 (+1 val). Drop rules with hit / sole counts: raw_thinking_low (raw-thinking coverage < 0.98 — not the documented default 0.90; 2,681 / 2,173), vendor_sourcing_mark (823 / 438), content_leak_same_task (169), answer_source_fetch (128), sandbox_escape (99 / 27), integrity_policy_violation (80), content_leak_tool_return (golden-line run ≥ max(8, 0.5·gold); 55), multi_user_turn (38), upstream_clone (29), gold_test_fetch (1). Demote-but-keep rules (1,050 rows): huge/tiny patch, memorised-upstream, leak-in-think, unverified success claim. Message-level loss masks on 5,528 of 8,137 rows: harness_error (4,513 rows: harness-rejected calls + their text), ignored_cwd (2,543), assert_removed (582), harness_internal (502), silence (479), summary_think (258), dup_repeat (155), grader_config (39), network (7) → 11,713 of 437,589 assistant text segments and 16,856 of 474,417 tool-call segments masked. Loss falls only on assistant text/think and tool calls (≈31 % of tokens).
• Status: HEADLINE together with D2 (the funnel table exists; the method is the rule set, the thresholds, and the two measured consequences — thinking economy and harness-error recovery).

D1 rule-impact addendum (2026-09-02, analysis/data/rule_impact.md / .json, offline, reconciles exactly with build_report.json). The v12 filter is one rule plus a tail: raw_thinking_low alone removes 2,450 of the 3,435 drops (71 %), vendor_sourcing_mark 493 (14 %), the leak family 45, sandbox_escape 31, multi_user_turn 27; the demote tier removes nothing (1,050 demoted rows are trained); the same-task cap removes 39 rows. Volume and behaviour separate: raw_thinking_low leaves the tool mix untouched (JS 0.005 to the kept set) but removes the longest, most think-heavy trajectories (median 68 vs 44 steps; think share 0.897 vs 0.838); the sourcing rules and the 221-row vendor removal remove a Bash-heavier (58–63 % vs 51 %), cd/git-heavier profile (JS heads 0.039–0.065 vs floors ≤ 0.002) with far worse protocol-error recovery (leak family 0.24 vs 0.60). Masks cover 2.77 % of learnable characters in v12 (3.43 % in v13; failed_conclusion 0.15–0.94 % over 3,088 rows). Composition is flat along the funnel (Go 37 → 38 %, top-15 repos 40.7 → 37.7 %, cap inert: max repo share 6.66 → 6.69 %); adding by-product rows in v13 is the only composition shift (Go 28 %). Accepted wording for §3: "the thinking-coverage floor removes volume without changing the tool mix; the sourcing rules remove a small, behaviourally distinct slice; the demote tier and the cap are inert." Status: descriptive (no rule was ablated by training).

D2 — Curation ablation: verified-pass-only (v12) vs adding masked failures (v13) (HEADLINE candidate)

• Same base, same hyper-parameters, near-identical step count (381 vs 362): v13 (9,880 high-precision [raw-thinking floor relaxed 0.98 → 0.90 and narrower vendor-mark rule] + 14,081 by-product = 7,503 passes + 6,578 failures, of which 3,104 (47.2 %) carry the failed_conclusion mask on the last assistant text; train 23,238 rows) vs v12×3 (s381). The SUCCESS_CLAIM regex behind the mask fires on 46.4 % of the teacher's *successful* endings and 44.6 % of its failed ones — it measures phrasing, not correctness. Pro 52.3 % vs 51.0 %; Verified 66.2 % vs 71.2 % (−5.0 pp, exact p = 0.0046); vs native: Pro +2.19 (p = 0.14), Verified −2.2 (p = 0.26).
• Behaviour and footprint unchanged by the by-product half: cumulative input 1.80 M vs 1.52–1.75 M; recovery 0.599 vs 0.580–0.625; thinking share 0.946 vs 0.925–0.933; sub-agent share 0 %.
• The mask moved failure-ending wording (strong-form claims 44.1 % vs 48.9 % for s381; teacher 44.6 %) but not honesty (0 admissions in the audit for every arm); variant false-success wordings passed the mask into training.
• Reading: selecting verified successes is what the score can tolerate; adding masked failures buys nothing measurable and costs Verified; masking changes the surface form of endings only. Caveat (judge): corpus composition and epoch count differ together, so the Verified loss cannot be attributed to the mask alone — a v13-high-precision-only ×3 control would separate them (nice to have).
• Files: data/report_excerpt.json (v13 vs s381 tests), fp_summary.json (overclaim), fp_pooled.json, think_overshoot.json.

D3 — Corpus composition sets the student's thinking economy (SUPPORTING)

• The corpus itself has thinking share 0.8805 (278.0 M thinking vs 37.7 M text characters over 425,876 loss-bearing assistant rows), matching the teacher (0.893); students land at 0.925–0.946. What the student amplifies is the corpus' dominant token type; 88 % of loss-bearing characters are thinking. (Mechanism stays a hypothesis without a loss-mask ablation.)

D4 — Corpus-taught habits are curation targets (SUPPORTING; proposes rules, tests none)

• Write-without-read: 138 / 133 occurrences on the two SFT arms of the A3B line vs 72 / 23 on the two non-SFT arms, replicated at both temperatures; on the 27B line v12 arms 96–138 vs native 23–72. A curation rule "drop Edit steps with no prior Read of the file in the same trajectory" is the only corpus change with two-base evidence; untested.
• Vocabulary and channels are inherited wholesale: echo share 14.2 % (teacher) reproduced by s256; sub-agent dispatch 0.28 % (teacher) → 0.27 % (s256); commands with truncation pipes 0.672 → 0.649.
• Negative result on corpus content: Qwen3.5-Base+v12's 1,106 Edit parameter-name errors were traced to the base's constrained-JSON generation, not to the corpus — the offending names occur 0 times in the corpus' 474,417 tool calls (Edit uses only file_path / old_string / new_string / replace_all, 3 exceptions). "Clean the placeholders out of the corpus" was therefore withdrawn as an action.
• Files: A3B report page (protocol-error section), fp_pooled.json (noread), fp_summary.json (bash_heads, mix).

3. Reconciliations

• H2 vs "no score": say the conditional sentence — the fingerprint converges identically whether or not the score moves; the score moves only from a bare base.
• "±3 pp noise band" → "below the paired minimum detectable effect".
• H4 vs v13: the mask was designed as an honesty intervention; it changed neither honesty (0/109) nor wording beyond noise vs s381, and cost −5.0 pp on Verified.
• S3 vs H3: the thinking overshoot is exactly the +16–49 % output tokens.
• S4 vs S5: q38's 0 % sub-agents shows narrowing is not specific to behaviour cloning.

4. Threats and defences

1. Teacher never ran on the evaluation pools → same-task anchor, language strata, Python-teacher reference; still missing the held-out same-pool anchor (373 val tasks).
2. Train/eval overlap (teleport) → instance-level zero matches, Verified repo-disjoint; add teleport-excluded tables and a decontamination statement.
3. Single teacher, corpus, recipe, harness → two student families, bare vs post-trained starts, n27 stays far in the same harness (the harness alone does not create the fingerprint); scope the title and Limitations.
4. Metric validity (regex + JS + model annotator) → 240-sample audit (precision 69/70); add human labels, a second distance, CIs, and treat the habit/strategy split as exploratory.
5. Statistical practice (111 paired tests, no correction; MDE from a subsample) → pre-declared endpoints, Holm, re-derived MDE, one p-value per test.

5. Decisions already taken

• One paper (F3), ACL/EMNLP Findings via ARR October 2026. The twelve arms are final; no reruns. Infrastructure-caused reruns are not reported. GPU experiments: held-out anchor (must), v12 on Qwen3.8-27B (must if it finishes), A3B checkpoints 127/254 and a mini-SWE-agent harness check (nice to have); second teacher/corpus family → Limitations.

6. Data sources

7. Gate verdicts (2026-09-02) and accepted wordings

result-to-claim (Codex gpt-5.6-sol, ultra): H1 partial · H2 partial · H3 partial · H4 partial · S1 partial · S2 partial · S3 partial · S4 yes · S5 no · overall partial · confidence high. novelty-check (xhigh): 7/10, PROCEED WITH CAUTION, closest arXiv:2604.21255, no ABANDON paper. Full text: CLAIMS_FROM_RESULTS.md, NOVELTY_REPORT.md.

Positioning adopted from the novelty gate: *What agentic SFT buys: rapid behavioural and input-token convergence, while capability gains depend on the starting model.* Anticipation to honour: 2606.16988, 2604.21255, 2606.03461, 2607.06184.

8. Presentation rules and post-gate analysis status (2026-09-02)

• User rules: no confidence intervals in the paper body; significance statistics de-emphasised (an appendix section "evaluation noise" holds exact p, Holm, MDE and flip rates); the paper's two innovation lines are the behavioural analysis and the SFT data-curation pipeline (D1–D4). Score tables show pass@1 and pp differences only; "above noise" for fingerprints is expressed through the same-weights (temperature-twin) and split-half floors.
• Done: score-level statistics (analysis/stats/: 111/111 tests reproduced, Holm/BH families, MDE, twins, teleport slice); data-pipeline source material (analysis/data/data_pipeline.md: funnel, rules, masks, v13, what curation did not catch, 10 open items).
• Done: fingerprint difference checks, Hellinger/TV, same-weights and split-half floors, absolute post-error thinking, anchor diagnostics (analysis/fp/); second (cross-model) annotation of the endings (analysis/annot/). Pending: Figure 1 (figures/), held-out anchor and Qwen3.8+v12 (GPU).