Skip to main content

1. Split the Agent into Steps

The pr-review agent from Build Your First Agent is one model step. In this track, you build a copy of it, pr-review-steps, with more than one step. The model fetches the pull request itself, and it can do this differently in each run. In this page, a code step downloads the diff and counts the changed lines. The code step does the same work in each run and costs no model tokens. The model step then reads the diff and writes the review. A condition between the two steps stops the review of a pull request that is too large.

Step 1: Make a copy and split it into two steps​

The other tracks use pr-review as you left it in the tutorial, so this track works on a copy. Go to Agent Studio → Agents, click Create agent, type pr-review-steps as the agent name, and click Create blank. In the editor, copy the content of agents/pr-review/instructions.md into a new file agents/pr-review-steps/instructions.md.

Open agents/pr-review-steps/agent.yaml. Replace its content with this:

YAML
apiVersion: agents.autoheal.ai/v1
name: PR Review Steps
icon: git-pull-request
description: >-
Reviews one pull request and gives a verdict, the riskiest change and a
question for the author. Give it the pull request URL.
inputs:
prompt:
type: string
required: true
description: The pull request URL.
focus:
type: string
description: What the review looks at hardest.
enum: [correctness, security, tests]
default: correctness
post_comment:
type: boolean
description: Post the review as a comment on the pull request.
default: false
timeout: 5m
steps:
- id: fetch
name: Fetch the pull request
runtime: bash
env:
- name: PR_URL
value: ${{ inputs.prompt }}
code: |
set -euo pipefail
mkdir -p /tmp/pr
curl -fsSL "${PR_URL%/}.diff" -o /tmp/pr/pr.diff
files=$(awk '/^diff --git /{n++} END{print n+0}' /tmp/pr/pr.diff)
lines=$(awk '/^(\+\+\+|---) /{next} /^[+-]/{n++} END{print n+0}' /tmp/pr/pr.diff)
printf '{"diff_file": "/tmp/pr/pr.diff", "files": %d, "lines": %d}' \
"$files" "$lines" > "$AUTOHEAL_OUTPUT"
- id: review
name: Review the diff
instructions: ./instructions.md
model:
name: claude-haiku-5-5
max_tokens: 4096
limits:
model_requests: 20
skills:
- skills:review-rules
system_tools: operation_type == "read"
integration_tools: integration_type == "github" && operation_type == "read"
with:
prompt: |
Review ${{ inputs.prompt }}.
The diff is in ${{ steps.fetch.output.diff_file }}: ${{ steps.fetch.output.files }} files, ${{ steps.fetch.output.lines }} changed lines.

focus: ${{ inputs.focus }}
post_comment: ${{ inputs.post_comment }}

If your model from chapter 2 is different, keep your model in the review step.

PartWhat it does
stepsA list of steps. The steps run in the order of the list. Each step has an id.
fetchA code step: runtime and code. It runs the script in the run's sandbox.
envEnvironment variables for the script. ${{ inputs.prompt }} gives the pull request URL.
$AUTOHEAL_OUTPUTThe script writes one JSON object to this file. The fields of the object are the step's output.
reviewA model step. The top-level fields of the tutorial agent (instructions, model, limits, skills and the tool filters) are now in this step.
with.promptThe first message to the model. ${{ steps.fetch.output.lines }} reads the lines field of the fetch step's output.

The code step does not use git clone. A code step cannot call tools, and curl on the .diff address of a pull request gives the diff of a public repository.

Step 2: Update the instructions​

Replace the content of instructions.md with this:

MARKDOWN
You review one pull request on GitHub.

1. An earlier step downloaded the diff of the pull request to a file. The prompt gives its path. Read it.
2. Clone the repository with the `git_clone` tool. Give it the repository's https URL and the ref `pull/<number>/head`. Read the changed files in full in the clone, not only the diff.
3. Write a verdict: merge, merge with changes, or do not merge.
4. Name the riskiest change, with its file and line.
5. Ask the author one question.

Look hardest at the `focus` input. For `correctness`, look for bugs and errors that the code does not handle. For `security`, look for secrets, unsafe input and permissions. For `tests`, say what the tests do not cover.

If the `post_comment` input is true, post the review as one comment on the pull request with `gh pr comment <URL> --body-file <file>`. If it is false, do not comment.

Do not push, approve, merge or close anything.

All steps of one run use the same sandbox. The model step can read /tmp/pr/pr.diff, because the fetch step wrote it.

Step 3: Commit and run​

  1. Commit

    In the Changes panel, click Commit changes, then Commit 2 files. Check that the editor shows no errors for agent.yaml.

  2. Run

    Open the agent's Overview tab. Paste https://github.com/autohealai/astronomy-shop-template/pull/1 and click Start run.

  3. Read the steps

    Open the Transcript tab. Check that it shows two rows: a function row named Fetch the pull request and an llm row named Review the diff. Open the first row. Check that its Result shows files: 2 and lines: 21.

  4. Read the review

    Open the Summary tab. Check that the review finds the timeout bug: the code gives gRPC a value in milliseconds, but gRPC reads the timeout in seconds.

Step 4: Skip a pull request that is too large​

The review rules say that a change of more than 400 lines must say why it is not split. You can check this before the model starts. Add this input below post_comment:

YAML
  max_lines:
type: integer
description: The review does not run when the pull request changes more lines than this.
default: 400

Then replace the whole review step with this size step. The review step moves into then, with the same fields:

YAML
  - id: size
name: Check the size
if: ${{ steps.fetch.output.lines <= inputs.max_lines }}
then:
- id: review
name: Review the diff
instructions: ./instructions.md
model:
name: claude-haiku-5-5
max_tokens: 4096
limits:
model_requests: 20
skills:
- skills:review-rules
system_tools: operation_type == "read"
integration_tools: integration_type == "github" && operation_type == "read"
with:
prompt: |
Review ${{ inputs.prompt }}.
The diff is in ${{ steps.fetch.output.diff_file }}: ${{ steps.fetch.output.files }} files, ${{ steps.fetch.output.lines }} changed lines.

focus: ${{ inputs.focus }}
post_comment: ${{ inputs.post_comment }}
else:
- id: too_large
name: Refuse a large pull request
runtime: bash
env:
- name: LINES
value: ${{ steps.fetch.output.lines }}
- name: MAX_LINES
value: ${{ inputs.max_lines }}
code: |
printf '{"message": "This pull request changes %d lines. The limit is %d. Split it, or say why not."}' \
"$LINES" "$MAX_LINES" > "$AUTOHEAL_OUTPUT"

When the condition in if is true, the steps in then run. When it is false, the steps in else run. The too_large step uses no model, so a large pull request costs no tokens.

Commit, and run the agent two times on the example pull request:

  1. Run it with the default max_lines. Check that the Transcript shows the if row Check the size, and that the row says that the run took then. Check that the review ran.
  2. Open the more options fold of the run launcher, set max_lines to 10, and run it again. Check that the run took else, and that Review the diff did not run. The result is the message "This pull request changes 21 lines. The limit is 10."

What to know​

  • In an agent with steps, the model fields go in a step. If you leave model at the top level next to steps, the push fails with field model is not allowed here; the fields allowed here are apiVersion, name, icon, labels, description, timeout, inputs, steps, outputs, env.
  • A model step in steps receives only the message that you write in with.prompt. Write each input that the model needs into that message, as focus and post_comment are above.
  • Instructions cannot read a text input. If instructions.md has ${{ inputs.prompt }}, the push fails with reads inputs.prompt, which takes any value. Put the text input in with.prompt.
  • A list of steps runs in order. Do not add after to a step in a list: the push fails with not allowed. after is for steps written as a map. See Structure.
  • Write code as a block (code: |). A one-line code: echo '{"a": 1}' > ... is not valid YAML, and the push fails with mapping values are not allowed in this context.
  • A code step's output is the JSON object in $AUTOHEAL_OUTPUT, not what the script prints. A script that prints the JSON has the output {}, and a later step that reads a field fails, for example with invalid operation: <nil> <= int. A file with key=value lines fails the step with AUTOHEAL_OUTPUT must contain a JSON object.
  • The word is output, not outputs. ${{ steps.fetch.outputs.lines }} passes the push, but the run fails with cannot fetch lines from <nil>.
  • The steps in then and else belong to the if step. A step after size cannot read steps.review. Page 3 shows how to read it.
  • To repeat steps, use a while step with max_iterations. See Loops (while).
  • A code step stops after 3 minutes unless you set timeout on the step.

From the CLI​

BASH
autoheal agents pull pr-review
cp -R agents/pr-review agents/pr-review-steps
rm agents/pr-review-steps/.autoheal-base.json
# edit agents/pr-review-steps/agent.yaml and instructions.md
autoheal agents push agents/pr-review-steps -m "Fetch the diff in a code step"
autoheal run pr-review-steps "https://github.com/autohealai/astronomy-shop-template/pull/1" --commit <sha> --wait -o json
autoheal run pr-review-steps "https://github.com/autohealai/astronomy-shop-template/pull/1" --arg max_lines=10 --commit <sha> --wait -o json

The push prints the commit. In the JSON of a run, .steps has one entry for each step that ran, with its step_path (for example size.then.review), status and output.

See Steps, Code execution and Conditionals (if/else) in the reference.