Skip to main content

Capabilities

Self-improvement is built from a few capabilities. Some every agent has by default; some you turn on for an agent; some only the agents that improve other agents have.

"Self-improvement agents" below means the agents that do Learn and Heal, and the assistant when you ask it to improve an agent.

CapabilityExecuteReadWrite
Trace-Self-improvement agents and your own agents, with the run-reading tools. Self-improvement agents also list an agent's runs in a time range, with each run's cost, tool errors and checksEvery agent: a trace is built for every run
EvalsAny agent, by attaching evals in agent.yamlSelf-improvement agents: a run's eval results, with each rubric's score and the judge's reasoningThrough Git
Memory—Any agent you turn it on for: search and read. Self-improvement agents also list every memory and the runs it changedAny agent you turn it on for
Skills—Every agent, by defaultThrough Git
BenchmarkSelf-improvement agents: run a benchmark on a branch or a commitSelf-improvement agents: datasets, cases, evaluators, benchmarks and executionsSelf-improvement agents: datasets, cases, evaluators and benchmarks
Git——The healer and the assistant, on a branch
Context Graph—Every agentAny agent whose system_tools filter keeps context_graph_write (Agent tools)

Why so much goes through Git​

Agents, skills, eval rubrics and artifact templates are all files in your project's Git repository. An agent that can edit files on a branch can change any of them. It doesn't need a separate tool for each kind of file.

That's why the table has one Git capability rather than a "write rubric" tool, a "write skill" tool and so on. It also means every change an agent makes is a branch you can diff, review and revert, the same as a change a person makes.

Benchmark datasets don't live in Git yet, so writing them needs its own tools for now. Moving datasets into Git will remove those tools, and the Git capability will cover them too.

Memories don't live in Git. They live in the memory store and go through the Memories review queue.

  • Observe: the trace, and how a run is read
  • Evaluate: attaching evals to an agent
  • Benchmark: datasets, benchmarks and executions