SFT 学到了行为，能力收益取决于起点

paper/distill-fingerprint/tex/main.tex

We curate 8,137 \Teacher/\Harness trajectories from 32,830 captured runs with an explicit rule set, fine-tune two open Qwen families on them for one to three epochs from bare and post-trained bases, and evaluate twelve arms in one harness on \SWEPro and \SWEVerified; one extractor fingerprints all 46,542 teacher and student trajectories, and two same-pool anchors put teacher and students on the same tasks.

Fingerprint proximity changes whether or not \passat changes: the corpus lifts a bare base by $+8.9$\pp on Pro ($+3.4$\pp on Verified), every comparison against a post-trained base ($-2.2$ to $+2.8$\pp) stays below the paired design's detectable effect, and the next Qwen generation gains $+5.6$ to $+5.8$\pp over the native dense model at $1.35\times$ its input.

\input{sections/01_intro}

\input{sections/02_related}

\input{sections/03_corpus}

\input{appendix/A_evaluation_noise}

\input{appendix/B_fingerprint_robustness}

\input{appendix/C_extras}

SHA-256 033790b2f28e199e122e496621afe54544eafdce2e1981d9920025ca381d2998

paper/distill-fingerprint/PAPER_PLAN.md

**One-sentence contribution**: A curated corpus of 8,137 frontier-agent (Claude Opus 4.8 / Claude Code) trajectories, built by a rule set we describe and measure, moves open students' command vocabulary, error recovery, thinking rhythm and input-token footprint toward the teacher — most of the dense model's convergence after one epoch — while pass@1 response depends on the starting model: +8.9 pp on a bare base (Pro), and −2.2 to +2.8 pp against post-trained baselines with no comparison resolved at this evaluation scale.

- One-sentence contribution (above). Approach: curate → train 1/2/3 epochs on two families → evaluate 12 arms → fingerprint everything → anchor → audit.

- [x] GPU: held-out anchor (must) — landed 2026-09-06: val373 → 345 oracle-passed tasks × 8 arms (`analysis/anchors/val373/`), pro100 grew to 359 Pro tasks run by the teacher (`analysis/anchors/pro359/`); Appendix E (Tables E1–E5) + body Figure 15; "task-independent" claimed (composite 0.051 → 0.006 at one epoch on never-trained tasks; read precision 0.56 → 0.78 held-out vs 0.50 → 0.80 trained); corrections applied: teacher makes 0 task-management calls on the benchmark (3.6 % on val, 3.2 % corpus), thinking position within 2× floor on the same tasks

- [x] Same-condition retest (reviewer item 3): STOPPED 2026-09-06 15:50 at 9/345 by user decision (no further experiments unless strictly necessary; temperature twins + split-half floors stand as the reliability reference) — had started 13:20 on inst1 engine + gpu23 sandboxes: n27 and s256 again on val373 at T=0.8 (`chain_val373.sh LINE=C ARMS="n27 s256" TAG_SUFFIX=-retest`); when done, add flip rate and fingerprint test–retest to Appendix A/E and §7

- [x] Qwen3.8-27B + v12 graft: dropped 2026-09-06 (user decision, "不测了"); the generation comparison stays descriptive with the native Qwen3.8-27B arm only

SHA-256 d4b96d91203a84a8aacd3cdac2539750773e8aebc34492c58ca5d6dcd7b0697e

paper/distill-fingerprint/tex/tables/tab1_arms.tex

\nTwoSeven & 50.1 & 68.4 & -- & -- & -- & 3.27 & 0.202 \\

\sOneTwoEight & 49.7 & 67.8 & $-0.41$ & $-0.60$ & $-1.22$ & 1.69 & 0.069 \\

\sTwoFiveSix & 52.8 & 68.6 & $+2.74$ & $+0.20$ & $+2.60$ & 1.52 & 0.053 \\

\sThreeEightOne & 51.0 & 71.2 & $+0.96$ & $+2.80$ & $-0.15$ & 1.75 & 0.050 \\

\aBase & 35.4 & 52.0 & -- & -- & -- & 3.40 & 0.278 \\

\aTwoEp & 44.3 & 55.4 & $+8.89$ & $+3.40$ & $+9.62$ & 2.44 & 0.057 \\

\caption{The twelve evaluation arms, named by base model and corpus dose (\dose{v12}{2} = two epochs of \vTwelve; \dose{v13}{1} = one epoch of \vThirteenCorpus, 362 updates). \passat = solved tasks over the pool (731 \SWEPro, 500 \SWEVerified; anomalous trials count as unsolved), one rollout. $\Delta$: percentage points against the line's native arm (\qThreeEight: against \nTwoSeven, the generation jump). $\Delta$Pro$-$tp: Pro without the 76 teleport tasks ($N=655$). Input: median cumulative input tokens per task on Pro, millions. Bash-\JS: Jensen--Shannon distance ($\log_2$) of the Bash head-word distribution to the teacher on Pro. The two $T{=}0.6$ rows are the same weights as the rows above them, run a second time (the run-to-run reference, \S\ref{sec:measurement}). ``--'': not defined for a native arm.}

SHA-256 49818f7b9d95804c8cbcad1363ee17f1b9f21f0ec202568c5ea7d8027e4e7ac7

paper/distill-fingerprint/findings.md

- Full draft: abstract, §1–8, Limitations, Ethics, appendix A (7 tables + endings audit), B (8), C (5) — every appendix table generated from the analysis JSON/MD by `tex/tables/make_tabA/B/C.py`; `references.bib` filtered to the 45 cited entries (`tables/make_refs.py`). Body ends on page 8 of the ACL two-column build (28 pages with appendix), 0 errors, 0 undefined references, no figure fallback text in the PDF.

s256 and s381 come from the same run (`output_dir=/output/qwen36-27b-v12-3ep/v0-20260809-124327`,

`max_length=196608`, `save_steps=64`); **s128 does not** (`output_dir=/v12/ckpt128-hf`,

`max_length=262144`, no `save_steps`). All three start from `/models/Qwen3.6-27B` on corpus v12,

All eight arms finished 345/345 (`bench/results/anchors/val373/summary.md`, regenerated 09-06):

- 🔴 The two MoE arms are **not same-family** with the six 27B dense arms, and not with each other

SHA-256 84f0cc4512a7c02d4047df72e2b9ba1fc352695ed9e3f130bb6115f840f12034

paper/distill-fingerprint/tex/sections/09_limitations.tex

Every student learns from trajectories of a single frontier agent captured in a single harness, and every arm is evaluated in that same harness.

The teacher's benchmark trajectories cover 359 of the 731 \SWEPro tasks and none of \SWEVerified, and its thinking is returned encrypted there, so same-task comparisons on the benchmark use tool rulers only; the held-out anchor shares repositories, not tasks, with the corpus, and its two MoE arms start from different bases.

The 180 endings (110 regex-negative, 70 regex-positive) and 68 rationalization windows were labeled by two model annotators, which agree on the two quantities the body uses (0 of 110 admissions of failure; regex precision 69 of 70) and differ on the rationalization split (48 versus 49 verified attributions, 12 versus 6 bare dismissals); no human pass was run, and the body states only the agreed quantities.

\paragraph{No single-rule ablation.}

The MoE dose curve has two points per base and every arm has one rollout per task, so the behavioral rulers are read against split-half floors and the two repeat-sample arms.

SHA-256 9b04b60a149f76f17836cda5db7d0ede47a7f2bf1adfe82ad86fc24c1e313e57

paper/distill-fingerprint/CLAIMS_FROM_RESULTS.md

verdict: PARTIAL

integrity audit: not run (`EXPERIMENT_AUDIT.json` absent) → this verdict is provisional per the skill's rules.

| H2 fingerprint vs capability | partial | high | "Fingerprint proximity changes substantially whether or not pass@1 changes: bare-A3B SFT improves Pro by 8.89 pp but is inconclusive on Verified, while no baseline-relative post-trained comparison is significant and its intervals still permit several-point effects." | Holm over the 111 valid paired tests; MDE re-derivation on 731; teleport-excluded tables; equivalence-style inference instead of "≈0"; v12 on Qwen3.8-27B; drop causal "score comes from the base" wording |

| H3 cost | partial | high | "Under the Claude Code Pro harness, v12/v13 SFT reproducibly reduces cumulative input tokens through shorter runs and more bounded observations, while increasing output tokens; this is a token-use signature, not a general monetary or wall-clock saving." | paired CIs per task; mini-SWE-agent harness check (nice to have); rename to "input-token footprint is part of the fingerprint" |

SHA-256 415be75a7d8c48ee154b3ad6afc41715c608a2b517b85993c8b2614f84df36c4

paper/distill-fingerprint/CITATION_AUDIT.md

**Date**: 2026-09-06 **Bib file**: tex/references.bib (61 cited entries, generated from lit/refs.bib + lit/refs_manual.bib) **Reviewer**: gpt-5.6-sol (xhigh), four fresh threads; web access blocked in the sandbox, existence checked against the arXiv API record of 2026-09-02/03, vendor URLs re-checked by the orchestrator.

## Priority fixes (all applied 2026-09-06)

- **advani2026from / cloud2025subliminal** — unverified venues (FAGEN@ICML 2026; Nature) dropped -> arXiv-only entries

- **singh2025from** — clause removed; entry no longer cited

| singh2025from | REPLACE | context | clause removed; entry no longer cited |

SHA-256 c22361b34fe7aa618d098c42b813c85211be459548b704fe4996e8ae7f3edec1