v0.1 · one line of vibe · val-gated rollback

SkillOpt-Lite and
HarnessOpt.

Two coding-agent slash-commands that iteratively improve any language agent from its own scored rollouts, with automatic git tag rollback whenever the val split says the change hurt. Better and faster agent self-evolution — no fine-tuning, no inference-time overhead. /skillopt-loop outputs a shippable skill.md; /harnessopt-loop also co-evolves the agent harness (rollout, react-agent, code-gen, executor, adapter).

Star on GitHub Paper (PDF) Poster 🤗 Dataset

~/copilot_example/spreadsheetbench

# in your coding agent, open the env folder + slash-command palette

› /skillopt-loop rounds=3 batch=20

round 1/3 · train 20 items ·······················

rollout ok 0.612 → samples/*.md

exploration ok edits: skill.md +12 / -4

val gate 0.601 → 0.674 ✓ keep

round 2/3 · val gate reverted ····· 0.674 → 0.669 ✗

exploration ok edits: skill.md +5 / -1

round 3/3 · val gate kept ········· 0.674 → 0.712 ✓
# best skill archived → workspace/.skillopt/history/

# wider surface: also let the coding agent edit the harness code

› /harnessopt-loop rounds=3 batch=20

rollout ok 0.641

exploration ok skill.md +8, react_agent.py +14, executor.py +3

val gate 0.641 → 0.7758 ✓ keep # GPT-5.4-nano+HarnessOpt > GPT-5.5+SkillOpt+std harness (0.7620)

› bash run.sh --split test --skill workspace/skill.md

6Benchmarks

+25.4LiveMath vs SkillOpt (GPT-5.4-nano)

nano > 5.5SpreadsheetBench GPT-5.4-nano+HarnessOpt beats GPT-5.5+SkillOpt+standard harness

2Slash commands, one loop

01 · Pipeline

The loop, at a glance.

Each round runs a small training batch, hands the failed traces to a coding agent, lets it patch the skill.md (and, in /harnessopt-loop, the agent code), re-runs val, and either keeps the edit or reverts. When it stops improving, grab the best artifact and ship it — nothing to install at inference time.

SkillOpt-Lite pipeline: rollout on training batch, coding-agent patch, held-out val gate, keep-or-revert, archive to history/ — End-to-end loop for both `/skillopt-loop` and `/harnessopt-loop`.

02 · Two commands

One line, two flavors.

Type it as a slash command in your coding agent, with the env folder open as workspace root. Ships wired for VS Code Copilot Chat today; Codex CLI, Claude Code, kimi-code, glm-code and deepseek-tui are on the roadmap. The prompt lives at <env>/.github/prompts/*.prompt.md — read it if you want to tweak the loop policy.

/skillopt-loop

6 envs

Skill-only optimization. Ships across all six benchmarks — the loop only ever edits one file.

edits skill.md

envs searchqa · livemath · alfworld

docvqa · officeqa · spreadsheetbench

runs rounds=3 batch=20

/harnessopt-loop

code + skill

Same loop, but the coding agent is also allowed to edit an allow-listed slice of the agent code. Pauses for user approval before each patch.

edits skill.md +

code rollout · react_agent

codegen_agent · executor · adapter

envs spreadsheetbench (more coming)

03 · How it works

Four moves per round.

A round is a self-contained atomic update — either it beats val or it gets reverted. History is git-tagged so you can always trace back.

ROLLOUT

Run run.sh on a batch-item slice of train. Each item's trace + score is dumped to .skillopt/samples/*.md. HarnessOpt uses the same rollout but also records per-step tool traces from the harness code path.

EXPLORATION

A coding agent reads the failed samples and proposes a bounded edit to skill.md. In /harnessopt-loop the exploration surface widens to an allow-listed slice of the agent code — rollout · react_agent · codegen_agent · executor · adapter — so the agent can co-evolve prompts and harness logic.

VAL GATE

Re-run run.sh --split val. If the new score dips below the previous best minus a dead-band, roll back with git reset. Same gate for both loops — HarnessOpt just reverts more files at once.

Wins on every benchmark we tried.

All numbers below are from Table 1 of the paper (identical evaluation protocol as SkillOpt — 6 benchmarks, 5 model scales) and reproduce from the shipped checkpoints via scripts/eval_only.py.

SkillOpt vs Ours

Benchmark · Model	SkillOpt	Ours
SpreadsheetBench · GPT-5.4-nano · HarnessOpt	76.20 (GPT-5.5)	77.58 HarnessOpt
SpreadsheetBench · GPT-5.4-nano	51.6	66.2 (+14.6)
SpreadsheetBench · GPT-5.5	76.2	79.7 (+3.5)
LiveMath · GPT-5.4-nano	30.3	55.7 (+25.4)
LiveMath · GPT-5.5	64.8	73.6 (+8.8)
ALFWorld · GPT-5.4-nano	71.8	81.3 (+9.5)
OfficeQA · GPT-5.5	72.2	76.2 (+4.0)
DocVQA · GPT-5.5	91.2	94.2 (+3.0)

Performance chart across 6 benchmarks — SkillOpt-Lite vs SkillOpt baseline — Per-benchmark deltas vs. full SkillOpt.

Numbers are absolute accuracy (%). Top row is the HarnessOpt headline: GPT-5.4-nano with an evolved harness beats GPT-5.5 running standard harness + full SkillOpt. Rows below are SkillOpt-Lite vs SkillOpt in matched settings; see paper Section 4 for the full 5-model × 6-benchmark grid.

05 · Roadmap

Coming next.

The loop is small on purpose — but the surface it wraps is agent-shaped and IDE-shaped. Two obvious expansion axes:

upcoming

Agent-agnostic runner

A drop-in script that wraps the loop around any agent codebase. Bring your own rollout(item) → trace, score and a skill.md entry point.

planned

Coding agents

Port the slash-commands beyond VS Code Copilot Chat. Same loop, packaged as a slash-command extension for each host: codex-cli, claude-code, kimi-code, glm-code, deepseek-tui.

planned

More envs for HarnessOpt

Roll /harnessopt-loop out to the other five benchmarks — same allow-list pattern, different harness shapes.

Want to help land any of these? Open an issue on GitHub.

06 · Cite

Cite the paper.

If SkillOpt-Lite or HarnessOpt is useful in your work, please cite the preprint. arXiv link coming soon.

@article{shen2026skilloptlite,
  title  = {SkillOpt-Lite: Better and Faster Agent Self-evolution with One Line of Vibe},
  author = {Shen, Yifei and Li, Bo and Zhang, Xinjie},
  year   = {2026},
  note   = {arXiv link coming soon}
}

SkillOpt-Lite and HarnessOpt.