What is loop engineering? No standard definition — the common core is the stop condition

1. Summary

In the paper that scanned public repositories, a loop’s configuration, such as the GitHub Actions workflows that define its trigger (the schedule or event that starts a run), stayed in the repository, but the state files, budget files and verifier-agent definitions that practitioner posts recommend were effectively never committed. The paper sums this up as the loops’ runtime state remaining outside version control. [1]

All figures are as reported in the papers. The 217 repositories are the result of the authors manually checking 256 candidates found by their heuristics; it is not an adoption rate for public repositories at large. Caveats are in sections 6 and 8.

2. Terms — ‘loop’ means three things

The word ‘loop’ is used for three different things, which causes confusion. Macedo’s paper says to keep the three apart. [2]

The agent loop is what the official Claude Code documentation describes as repeating three phases: gather context, take action, verify results. The Agent SDK documentation calls one round of tool calls and their results a turn, and says the loop ends when the model answers without calling a tool. [3] [4]

A harness is the software layer that wraps the model, provides its tools and manages the context the model sees. The Claude Code documentation describes Claude Code itself as this layer. [3] The paper by Lulla et al. arranges the ways developers direct agents into four layers and places loop engineering one layer above the harness. [1]

Source: Table 1 of Lulla et al. [1]. The authors note that each layer includes the one below it.

In this article a trigger is a schedule or a repository event, such as a PR being opened, that starts a run. A stop condition is a condition written in advance that says “end the run once this holds”, for example “npm test exits 0 and no other test file was changed”. A terminal state is the label attached when a run ends, distinguishing success, nothing to do, blocked, no progress and budget exhausted. [2]

3. Where the term came from

Lulla et al. trace the term to three statements made within a few days of each other in early June 2026. The quotations below are taken from the paper; of these, only Osmani’s post was checked directly at the source. [1]

Osmani’s post lists what a loop needs: automations, worktrees, skills, plugins and connectors, subagents, and a memory that lives on disk. With the line “the agent forgets, the repo doesn’t”, he recommends keeping state in files outside the conversation. In the same post he writes that even with a separate verifier agent, “done” is a claim and not a proof. [5]

There was plenty of pushback on whether this is a new idea at all. Among the practitioner reactions the paper collects is the verdict “a renamed cron job”, and Gergely Orosz of The Pragmatic Engineer, after gathering about 210 replies, found that most concrete examples fell into two buckets: cron jobs and event triggers. The paper concludes that it is not yet settled whether the term names a new discipline, a transitional technique or a marketing label. [1]

The two papers define it as follows.

• Lulla et al.’s working definition — designing automated control structures that repeatedly invoke coding agents on a schedule or on events, with each run bounded by a machine-checkable stop condition. A well-engineered loop persists state across runs, verifies results independently of the implementation agents, bounds inference costs, restricts what an unattended run may change without approval, and defines when humans should intervene. [1]
• Macedo’s definition — the discipline of designing and evaluating ‘loop specifications’ that a person writes and hands to a harness. A specification has five parts: a trigger, a verifiable goal, an execution step, a verification step, and a stopping rule that ends the run in a named terminal state. On top of these comes a memory kept on disk rather than in the conversation. [2]

The wording differs, but both put a stop condition and verification inside the definition. It is Lulla et al. who include verification independent of the implementation agent in the definition itself; Macedo treats separating the maker from the checker as a design principle outside the definition. The title of this article refers to what the two definitions share: the stop condition.

4. What the tools document

In the first half of 2026, both Claude Code and Codex added official commands that keep working until a goal condition holds. Claude Code’s weekly changelog introduced /goal in the week of May 11–15, 2026, and the OpenAI cookbook says Codex Goals are available from Codex 0.128.0. What follows is what the official Claude Code documentation and the OpenAI cookbook entry on Codex Goals said when checked on 2026-10-06. [6] [9]

Sources: Claude Code docs for /goal [6], /loop [7], the hooks guide [8], the Agent SDK agent loop [4], and the OpenAI cookbook “Using Goals in Codex” [9]. The Codex entry is a cookbook article written by OpenAI staff, not an official reference page.

The Claude Code /goal evaluator does not run commands or read files

According to the Claude Code documentation, /goal is a wrapper around a session-scoped, prompt-based Stop hook. Each time a turn ends, the condition and the conversation so far are sent to a small model, which returns one of “not yet met / met / impossible” with a short reason. The docs state that this evaluator does not run commands or read files; it judges only from what Claude has surfaced in the conversation. That is why the docs recommend writing the condition as something Claude’s output can demonstrate: one measurable end state such as “npm test exits 0”, a stated way to check it, and what must not change. [6]

The docs list other constraints too. A condition can be up to 4,000 characters, and to limit how long it runs you add a clause such as “or stop after 20 turns” to the condition. If Claude answers the evaluator for several turns in a row without using any tool, Claude Code stops the loop and hands control back to the user. /goal does not change the permission mode, so in Manual mode Claude still asks before each tool call that your settings do not already allow. [6]

Codex Goals do not treat an exhausted budget as completion

The OpenAI cookbook calls a Goal a thread-scoped “completion contract”: what should be true, how success is checked, and what must stay intact. A Goal is active, paused, complete or budget-limited; the model may mark it complete only when the evidence supports it, while pausing, resuming, clearing and budget-limited transitions are controlled by the user or the system. The cookbook states that reaching the budget is not the same as completing the objective, and says Codex should then summarise progress and blockers. The same document advises against using a Goal for work with a vague finish line such as “make this better”. [9]

The stopping mechanisms are documented as numbers

• Stop-hook block cap: if a Stop hook blocks stopping eight times in a row with no tool call from Claude in between, Claude Code overrides the hook. The docs advise checking the stop_hook_active field in the input and exiting early if a continuation was already requested. [8]
• Seven-day expiry for recurring tasks: recurring tasks created with /loop or the cron tools fire one last time and delete themselves seven days after creation. The docs describe this as bounding how long a forgotten loop can run. These tasks are session-scoped, so they fire only while Claude Code is running and idle. [7]
• SDK defaults are unlimited: the Agent SDK’s max_turns and max_budget_usd both default to no limit, and reaching either returns an error_max_turns or error_max_budget_usd result. The docs recommend setting a budget as a default for production agents. [4]

This design points the same way as the principles Anthropic set out on 2024-12-19 in “Building effective agents”. That post says an agent should gain ground truth from the environment at each step, such as tool call results or code execution, to assess its progress, and that it is common to include stop conditions such as a maximum number of iterations to maintain control. The post now carries a note that much of the tooling landscape has changed since it was written. [10]

5. Who checks completion, and with what

Macedo divides verification into five levels of rigour. This article uses the ladder as a way to ask, when designing a loop, “which level is my check actually at?” [2]

When Macedo hand-coded the 50 loops in one public collection (the Loop Library), 74% named their terminal states, 70% verified at levels 1–2, and 66% set a verifiable goal. By contrast, only 22% automated their trigger (12% scheduled, 10% event-driven), and 78% were started by hand. Eleven loops (22%) relied mainly on a model judge, and most of those paired it with a separate maker and checker or a separate critic. The author’s reading is that today’s loops are much better at deciding whether the work is done than at running without a person. [2]

The same paper names five designs to avoid. Three of them apply directly to loop design.

• The self-approving loop: when the same model produces and grades the work, the score can rise while the quality does not. The paper recommends separating maker from checker and preferring a level 1–2 check over a self-score. Separating permissions over grading criteria and run records is covered in our report on agent reward hacking.
• Reporting a model judge as a deterministic check: relying on a model’s opinion while reporting the confidence of a deterministic check. The paper recommends stating which level the verifier is really at.
• The unattended runaway: a loop with no task-related stopping rule, no no-progress detector and no budget ceiling. The paper recommends named terminal states, a no-progress detector and a budget ceiling, with human approval for anything irreversible.

Macedo also lists cases where a loop is the wrong tool. If new feedback does not change the next action, a single run (scheduled if needed) is enough; loops also do not fit matters of taste with no reproducible check, ambiguous greenfield work where even the direction is unknown, or work where the cost of looping does not pay for itself. [2]

6. Loops found in public repositories

Lulla et al. scanned, once on 2026-08-19, the 36,710 repositories classified as engineered software projects in an existing AI configuration dataset. Excluding 65 that had been removed from GitHub leaves 36,645. They looked for traces of eight tools (Claude Code, Codex, Copilot, Cursor, Gemini CLI, OpenCode, Hermes and OpenClaw) being run on a schedule or an event. [1]

Source: selected rows of Table 2 in Lulla et al. [1]. The denominator is the 36,645 scanned repositories. ‘Repositories where the detection was correct’ counts repositories where manual inspection confirmed the artifact was the mechanism the rule claimed; whether an autonomous run actually happened was judged separately, and that result is the 217 below, so 242 and 217 measure different things. The two state files were rejected on inspection as unrelated files that happened to share loop vocabulary. The zero for stop conditions means none of the phrases in the fixed list matched; among the 256 repositories the authors inspected, 55 recorded some goal or stop-condition mechanism, several of them as terminal states or retry caps written into committed prompts.

Checking the 256 repositories the heuristics surfaced one by one, the authors confirmed actual autonomous agent runs in 217, or 0.59% of the scanned repositories. Thirty-three were not confirmed and six were left unclear. Of the 217, 180 ran only on repository events, mostly automatic review of newly opened PRs; 21 ran only on a schedule, 15 used both, and one ran until its workflow broke. By tool, Claude Code accounted for 189, Codex 11, OpenCode 6, Gemini CLI 5, Cursor 4 and Copilot 2. [1]

The authors explain the missing state files this way. A PR review run reads the PR, posts its review and finishes, and the next run starts fresh from the next PR, so there is no state to keep. The scheduled runs were mostly issue triage, and the issue tracker already held the backlog between runs. In other words, the practitioner advice to “commit a state file” often did not apply to the common loops in public repositories today. [1]

A run that ends in success is not proof that it did any work

In one example from the paper, the hourly Gemini issue-triage workflow in the AI-Hypercomputer/xpk repository logged 6,290 runs, but because its gating issue search matched nothing, the agent never executed once. GitHub’s run history shows only success or failure; whether the agent actually ran could be seen only in the job and step records. The authors make three recommendations to teams running loops. [1]

• When there is nothing to do, report “nothing to do” instead of success.
• Log how long the agent step ran.
• Log what each run produced (a PR, a comment, a commit).

How to count real calls and avoid duplicate execution when retries stack across layers is covered in our report on agent retry design.

7. Checklist before you build a loop

These are this article’s recommendations, drawn from the documents and papers above. In the sources this article reviewed, no study has measured the effect of any of them.

• First, check whether the work needs a loop at all. If the result does not change the next action, a single scheduled run is enough. [2]
• Write the stop condition as the result of running a command. State how it is checked, as in “npm test exits 0”, together with what must not change. The Claude Code /goal evaluator sees only output surfaced in the conversation. [6]
• Write down who decides completion. Decide whether it is the working model, another model, or a deterministic check such as tests, and report honestly which level that check is at. [2]
• Set turn and cost caps. The SDK’s max_turns and max_budget_usd default to no limit. For /goal, add a clause such as “or stop after 20 turns”. [4] [6]
• Name the terminal states. Distinguish success, nothing to do, blocked, no progress and budget exhausted, and never count an error or an exhausted budget as success. [2] [9]
• Record whether a run actually did work. Keep the “nothing to do” report, the agent step’s run time and the artifacts produced. [1]
• Require human approval for irreversible actions. Then check whether the people giving that approval can keep up with the volume of PRs the loop produces. Practitioner reactions quoted in the paper note that nightly runs can open more PRs than a team can review, until human approval becomes a formality. [1]

8. What is not settled yet

• This article rewrites the initial June 16, 2026 note. That note drew on practitioner and vendor blogs and product documentation to present a six-element architecture, team adoption stages, metrics and templates, but it did not attach a source to each figure and claim. When the sources were rechecked on 2026-10-06, unverifiable claims, three paper citations whose support we did not recheck, and the templates we had written ourselves were removed.
• We did no measurements of our own. This article did not run /goal, /loop or Codex Goals to compare them. Section 4 describes the official Claude Code documentation and the OpenAI cookbook.
• There is no systematic evidence yet of what loops achieve. Lulla et al. note that claims such as Stripe’s roughly 1,300 agent-written PRs per week, or a loop pointed at Firefox producing 423 security fixes in a month, are self-reports from a podcast interview and social media posts, and that they know of no study or dataset supporting them. This article did not verify those figures either. The authors’ controlled experiment (comparing interactive use, goal-driven runs and scheduled loops) is still at the planning stage.
• The repository scan has a narrow scope. Lulla et al.’s paper is a workshop paper under review. The 256 labels were first assigned by an LLM and then adjudicated by one author; no inter-rater agreement between two independent coders is reported. The scan was a single pass about ten weeks after the term appeared, over a list of public repositories fixed in March 2026. In-session /loop and /goal runs, private repositories and platform-native schedulers leave no trace and were not counted. The 217 figure may understate actual adoption.
• Macedo’s figures cover 50 loops from one public collection. It is a single-author position paper, and the coding was done by that one author. The corpus is not a random sample, and the author states that no causal conclusion is drawn from the percentages. The author also acknowledges that many of the design principles are an analogy from research on the inner agent loop to the outer loop.
• Tool documentation changes quickly. Section 4 reflects the online documentation as of 2026-10-06. The Claude Code docs tie some behaviour to versions such as v2.1.234, v2.1.239, v2.1.246 and v2.1.269, so the version you have installed may behave differently. For Codex, the source is a cookbook article, not an official reference page.
• The term itself may not last. Practitioner opinions quoted in the paper hold that, now that /loop and /goal are built in, the term is already losing its use for engineers building ordinary software, and in July 2026 a successor label, ‘graph engineering’, appeared. The authors argue, however, that the design problem of bounding a non-deterministic worker with verification, budgets and escalation remains whatever it is called. [1]

9. References

All sources were checked on 2026-10-06. For both papers we downloaded the arXiv LaTeX source and checked the tables and limitations sections. Statements by Cherny, Steinberger, Orosz and others are re-quoted from Lulla et al. [1]; we did not open the original posts.

The initial June 16, 2026 note was rewritten to the current standard on 2026-10-06, and this translation was updated the same day. The Kilo and Langfuse posts and arXiv 2605.23135, 2603.16975 and 2606.08793 cited in the initial note are not used as evidence in this version. If the papers are revised or the tool documentation changes, we will update this article and note the revision date.