Self-Improvement
An agent that worked last quarter can be wrong today. The services it investigates get new dependencies, an API changes shape, a team renames its dashboards. Self-improvement is how Autoheal notices that and fixes the agent's context before people stop trusting it.
It works in five steps. Each one has its own page.
| Step | What happens | Scope |
|---|---|---|
| Observe | Every run gets a trace: totals, a headline per turn, and SQL access to every message and tool call | One run |
| Evaluate | Rubrics from your Git repository score runs or identify run features for analysis | One run at a time, summarized across many |
| Learn | A finished run is reflected on, and turned into memories or a skill | One run, or a few you pick |
| Benchmark | A proposed change is re-run on past cases and compared with the current agent | A dataset of cases |
| Heal | Weak or unusual scores across many runs are diagnosed and fixed with a pull request | Many runs |
review queuepull requestagent.yamlskilltemplaterubricbenchmarkTactical and strategic
There are two speeds of improvement, and they answer different questions.
Learning is tactical. It looks at one run, or a handful, right after they finish, and asks what's worth keeping. A person corrected the agent; the agent spent 40 tool calls finding which cluster a service runs on; a procedure worked and could be repeated. That becomes a memory or a skill that the next run can use.
Healing is strategic. It looks at hundreds of runs through their eval scores and asks what's wrong with the agent itself. Root-cause accuracy dropped after a model change; runs on database incidents cost three times the average; no run on a tier-1 customer's incident produced a postmortem. The fix is a change to the agent: its agent.yaml, a skill, an artifact template, or the rubrics and benchmarks that watch it.
A memory is cheap to add and easy to throw away. A change to the agent affects every future run. So learning runs on its own after each run, and healing goes through a benchmark and a reviewer.
Everything is in Git
Agents, skills, eval rubrics and artifact templates are files in your project's Git repository. That has three consequences:
- A change is a branch. Healing proposes a fix on a branch. You review it like any other pull request.
- Scores carry versions. Every eval result records which agent version and which rubric version produced it, so a score from before a change is never mixed with one from after.
- One write path. The agents that improve other agents don't need a separate tool for each kind of file. They edit files in a branch. See Capabilities.
Memories are the exception. They live in the memory store and go through the Memories review queue.
Engineers approve every change
Nothing changes an agent's behavior without a person approving it:
- A skill,
agent.yaml, artifact template, rubric or benchmark change is a pull request. - A proposed memory waits in the Memories review queue, under that page's auto-accept rules.
Agent tool calls that write to your systems, like a rollback, are governed separately by approval policies.
Limits
The loop only improves what a rubric can see. If no rubric checks for a failure, and no person corrects it in a run, the failure stays invisible. Writing a rubric for it is usually the first fix. See Evaluate.
Related
- Engineering Context Graph: the entities, relationships, skills, and memories the loop improves
- Sovereignty: where evaluation runs and what data stays in your environment