4_experiments · 完整稿件源码

\section{Experimental Design}
\label{sec:design}

\paragraph{Comparisons.}
The primary planned contrast is M versus B at step 80 on the designated 100-task holdout. It tests the allocation rule for artifact-derived feedback, including the concentration that localization creates. M versus O tests its value beyond the common terminal-only recipe; M42 versus E42 provides a single-training-seed comparison with terminal edit focus. \Cref{tab:design} specifies the four arms. B and M use identical eligibility, query-rate, utility, coefficient, and cap rules. Their online policies generate different trajectories after training diverges; pointwise coefficient equivalence is checked on a common frozen trajectory set, rather than assumed between unpaired online rollouts.

\input{../figures/TABLE_counterfactual_design}

\paragraph{Tasks and splits.}
The reinforcement-learning pool contains 149 tasks and 2,384 aliases, retaining 16 aliases per task from the original 150-task candidate pool. A task whose expected scoring list omitted its issue-specific failing test was excluded prospectively from all successor training arms. The development subset therefore contains 39 tasks, all inside the training 149. Each alias is a repeated entry for the same task, not an additional unique issue. The original 40-task pilot and its fixed factual replay sample remain intact; their denominators are not reduced retrospectively. The final evaluation protocol prescribes the original 500-task list and designated 100-task holdout with their original grading labels.

For the new training runs, the evaluation complement contains 351 tasks: the designated holdout 100, the other original 250, and the excluded candidate task. This denotes exclusion from successor RL training, not a guarantee of absence from base-model pretraining. The designated holdout of 100 tasks is reported separately and is not used to choose successor checkpoints or coefficients. For comparison with the original partition, supplementary tables will also expose the original training 150, the remaining 250, and the excluded task's membership explicitly.

\paragraph{Model, budget, and execution.}
All planned arms start from the same Qwen3.5-9B base checkpoint. The fixed primary endpoint is 80 training steps. O/B/M have planned training seeds 42, 314159, and 271828; E has a separately allocated seed-42 run. Evaluation repetitions are distinct from these training replications. The policy comparisons remain incomplete at this manuscript snapshot.

The common recipe uses 16 prompt groups per update and eight rollouts per group, a learning rate of $10^{-6}$, prompt-mean loss reduction, and the binary-TV DPPO threshold $0.15$. Equal-reward groups are retained. The credit scale is $\lambda=0.1$, the cap is $c=1$, and the query rate among eligible M/B rollouts is one. The common allocation is 10 inference GPUs and four policy GPUs, with 64 concurrent primary sandboxes. Sampling uses temperature 1, top-$p=1$, and no top-$k$ or min-$p$ restriction; the context budget is 98,304 tokens. Task timeouts use a multiplier of two and the budget reminder ratio is 0.35.

\paragraph{Shared terminal reward and environment.}
The resolved verdict supplies the ordinary success reward. Any format error overrides the terminal reward to $-0.5$; an unresolved trajectory with a detected pass-to-pass regression receives $-0.25$. These rules are identical for O/B/M/E and are separate from the auxiliary utility in \cref{eq:utility}. Counterfactual credit is added to advantages, not to this logged terminal reward.

The agent sees a repository restricted to the task baseline Git state and runs without network access. Original and counterfactual verification use the same task preparation. Package versions normally inferred from Git history are supplied from the original task image's package metadata. Two task-specific resource corrections provide two CPUs to a test that requires them and bound numerical-library threading under an existing one-CPU quota. The same profiles apply across arms; no expected test label is rewritten.

\paragraph{Evaluation and accounting.}
The planned main evaluation samples two rollouts per task on all 500 tasks. The higher-repetition holdout evaluation samples 16 per task and includes base, O, B, M, and E. The 16-rollout holdout cohort is independent of the two-rollout cohort. Both report mean single-rollout resolution, never the best outcome among repetitions. The primary estimator averages the three paired M--B training-seed differences on the holdout; the analysis and missing-outcome rules were fixed before evaluation and are specified in \cref{app:statistics}. Each evaluation records its model, task list, decoding settings, and attempt identities; missing attempts remain visible in the planned denominator. Actual allocated GPU time, generated tokens, verifier trials, and unavailable feedback will be reported alongside the outcome comparisons.