% 09_limitations.tex -- mandatory ARR Limitations section, not counted toward the page limit.
\section*{Limitations}
\label{sec:limitations}
\paragraph{One teacher, one corpus, one harness.}
Every student learns from trajectories of a single frontier agent captured in a single harness, and every arm is evaluated in that same harness.
Two student families and three starting points (bare MoE, post-trained MoE, post-trained dense) vary the receiver, and the native dense arm stays far from the teacher inside the same harness, so the harness alone does not create the fingerprint; but the transfer of a different teacher's habits, or of these habits into a different scaffold, is not measured.
\paragraph{Anchors cover Pro, not Verified.}
The teacher's benchmark trajectories cover 359 of the 731 \SWEPro tasks and none of \SWEVerified, and its thinking is returned encrypted there, so same-task comparisons on the benchmark use tool rulers only; the held-out anchor shares repositories, not tasks, with the corpus, and its two MoE arms start from different bases.
\paragraph{Model-annotated endings.}
The 180 endings (110 regex-negative, 70 regex-positive) and 68 rationalization windows were labeled by two model annotators, which agree on the two quantities the body uses (0 of 110 admissions of failure; regex precision 69 of 70) and differ on the rationalization split (48 versus 49 verified attributions, 12 versus 6 bare dismissals); no human pass was run, and the body states only the agreed quantities.
\paragraph{Repository-level exposure.}
No \SWEPro or \SWEVerified instance appears in the corpus (zero title matches, description Jaccard at most 0.21, zero commit matches), and the Verified repositories are disjoint from it; but the corpus contains 1,622 teacher trajectories from the \code{teleport} repository, which supplies 76 of the 731 Pro tasks.
Teleport-excluded differences are reported next to the full-pool ones (Table~\ref{tab:arms}); every \vTwelve and \vThirteenCorpus checkpoint gains more on those 76 tasks than on the other 655.
\paragraph{No single-rule ablation.}
The rule-impact analysis is offline: it measures what each rule removed, not what training without it would have changed.
The one trained variation, \vThirteenCorpus, changes the thinking floor, the by-product inclusion, the task cap and the epoch count together, and is reported as a composite.
The bare-base result likewise comes from one base and one corpus; the design does not isolate a general base-to-instruct mechanism.
\paragraph{Sparse dose curve and one rollout.}
The MoE dose curve has two points per base and every arm has one rollout per task, so the behavioral rulers are read against split-half floors and the two repeat-sample arms.
完整资料 / S5
现稿 Limitations
paper/distill-fingerprint/tex/sections/09_limitations.tex
公开内容 SHA-256 9b04b60a149f76f17836cda5db7d0ede47a7f2bf1adfe82ad86fc24c1e313e57