Peking University Distills Agent Harness Into Model, Lifting Task Success by 21 Points
Updated
Updated · O'Reilly Media · Oct 27
Peking University Distills Agent Harness Into Model, Lifting Task Success by 21 Points
1 articles · Updated · O'Reilly Media · Oct 27
Summary
A Peking University team fine-tuned Qwen3.5-9B on “corrected runs” and found the distilled model beat the original model-plus-harness setup, raising average task success by 21 points.
The gain came from training on near-miss trajectories that a stronger reviewer minimally fixed; three other signals — stronger-model successes, harness-completed runs, and answer-key reviews — left scores flat or caused overfitting.
The distilled model still needed a minimal loop and one Bash tool, and it lagged the harnessed version by 8 points on chemistry tasks where domain knowledge, candidate generation and molecule validation mattered.
That suggests harnesses are narrowing rather than disappearing, with context management and application-specific tools, data and environment remaining harder to absorb into the model.
The report argues teams should treat harnesses as disposable infrastructure: log every intervention, compare versions against new models, and remove components as model capabilities catch up.