Hmm… for now, based on a quick look at the implementation:
My tentative boundary would be:
Keep execution below the model boundary only while plausible intermediate results cannot change the remaining target, hypothesis, ordering, authorization, resource plan, or recovery path. Return control to the model as soon as any of those may change.
So I would treat this less as a question of “how many commands fit in one macro?” and more as an explicit termination and replanning contract.
A practical default might look like this:
Can an intermediate result change the next target, hypothesis, or arguments?
├─ Yes
│ └─ Return the evidence to the model.
└─ No
├─ Is this a long-running job?
│ └─ Register it as a durable job and wake the model on a terminal
│ or decision-relevant event.
├─ Do the commands share an exclusive resource, ordered state,
│ capacity limit, or irreversible effect?
│ └─ Keep the macro if useful, but serialize, gate, or limit concurrency.
├─ Is the downstream postcondition mechanically checkable?
│ ├─ Yes → continue below the model boundary.
│ └─ No → return the evidence to the model.
└─ Does an external or irreversible action require approval?
└─ Stop at an approval boundary.
In condensed form:
| Execute below the model boundary |
Keep the macro, but serialize or gate |
Return to the model |
| Known independent reads |
Commands sharing one build directory or cache |
Search results determine the edit target |
| Independent searches with fixed arguments |
Ordered workspace mutations |
A failure changes the diagnosis |
| Formatting after a settled edit |
Shared ports, DBs, browser profiles, locks, GPUs, or rate limits |
Semantic success cannot be checked mechanically |
| Independent validation with isolated resources |
External effects requiring approval |
The remaining plan may no longer be valid |
| Machine-checkable cleanup or audit |
Long-job registration |
An unexpected or ambiguous result appears |
The distinction I would make explicit is:
Batching and parallelism are separate optimizations.
Batching reduces provider-visible turns, repeated context, and repeated tool-selection reasoning. It can still work when all commands inside the macro are executed serially.
Parallelism may additionally reduce wall time, but it introduces another class of dependencies: shared locks, caches, ports, processes, capacity limits, and timeout interactions.
For that reason, a conservative default could be:
- use macro-serial whenever several already-known operations can be issued in one model turn;
- upgrade a group to macro-parallel only when both its data dependencies and resource dependencies are understood;
- return to the model when semantic interpretation or replanning is required.
A small runtime contract could make that boundary inspectable:
id: focused-tests
depends_on:
- build.Succeeded
resources:
mutex:
- cargo-target:/workspace/target
effects:
read_only: true
idempotent: true
on_failure:
siblings: continue
dependents: cancel
cleanup: run
control: return_to_model
postcondition:
structural: test-report-schema
semantic: tests-correspond-to-current-build
lifecycle: short_command
I do not mean that this exact YAML should become the API. The useful part is separating:
- result dependencies;
- resource dependencies;
- effects and approval requirements;
- failure propagation;
- short commands versus durable jobs;
- structural output validation versus semantic validity.
That would preserve the main benefit of macro execution while making the stop/replan boundary something the runtime can record, test, and improve rather than leaving it entirely implicit in the model prompt.
I also ran a small CPU-only Colab sanity check against Tura commit a0bdd1008e32.
The focused repository tests confirmed the behavior at that commit:
- an ordinary shell failure can coexist with later results in the same batch;
- a failed
apply_patch cancels later commands.
A separate deterministic lock probe produced this contrast:
| Synthetic workload |
Parallel |
Serial |
| Three independent short commands |
3/3 succeeded in every repetition |
3/3 succeeded |
| Three source-non-mutating commands sharing one exclusive lock |
1/3 succeeded in all five repetitions |
3/3 succeeded in all five repetitions |
The timeout was deliberately chosen to expose contention, so this does not identify the cause of any particular benchmark trace. It is only a controlled counterexample to:
“The command does not modify source files, therefore it is safe to run in parallel.”
A stronger condition would be:
The command has no relevant data, state, resource, effect, or recovery dependency on its siblings.
For an incremental implementation route, my default order would be:
- separate macro batching from runtime parallelism;
- record typed outcomes such as
Succeeded, Failed, TimedOut, Skipped, Omitted, and Cancelled;
- add result-sensitive dependency predicates;
- add mutex/semaphore-style resource groups;
- move long-running work into a durable task lifecycle;
- log why a macro continued, stopped, skipped work, or returned control.
Background: how this connects to existing agent and workflow ideas
The overall direction appears related to several existing lines of work, although none is exactly the same problem.
Plan/execution separation
LLMCompiler separates planning, dependency-aware task dispatch, and execution so that independent function calls can be scheduled without one model turn per operation.
That is close to the shape of command_run:
- the model emits a bounded plan;
- the runtime schedules known work;
- independent operations may run together;
- combined evidence is returned for another reasoning step.
The important difference is that repository work introduces more shared state than typical information-retrieval tasks:
- mutable source files;
- build outputs;
- compiler and package-manager caches;
- test databases;
- ports and servers;
- background processes;
- external side effects.
A plan that is safe for independent retrieval calls is therefore not automatically safe for repository mutation and validation.
Feedback-driven replanning
AdaPlanner is relevant to the other side of the boundary: a static plan becomes invalid when environmental feedback changes what should happen next.
That makes the central runtime question the replanning trigger:
- Was an expected file absent?
- Did a failure invalidate the current root-cause hypothesis?
- Did a patch apply differently from what was expected?
- Did validation run against a stale artifact?
- Did a command reveal a new dependency?
- Did the environment enter a recovery state rather than the expected state?
Those are not ordinary scheduler decisions. They change the meaning of the remaining plan and are natural points for returning control to the model.
Parallel tool execution
Anthropic’s parallel tool-use documentation gives a similar practical rule:
- independent read-only operations are good candidates for parallel execution;
- operations involving shared state, side effects, or ordering requirements may need serial execution;
- calls not executed because of an earlier failure should still receive explicit error results.
That last point matters for model recovery. The model should be able to distinguish:
succeeded
failed
errored
timed_out
skipped
omitted_due_to_dependency
cancelled
approval_required
Collapsing these into one generic error makes replanning unnecessarily difficult.
Agent-computer interface design
This also fits the broader Agent-Computer Interface argument made by SWE-agent: coding-agent behavior depends substantially on how search, editing, execution, and feedback are exposed to the model.
In that framing, macro execution is not only a prompting technique. It is part of the runtime and ACI design:
- what the model must decide;
- what the runtime can execute deterministically;
- what state is persisted;
- what evidence is returned;
- when authority is handed back.
What the current Tura behavior appears to encode
The current command_run documentation describes several useful guarantees:
- one provider-visible call contains multiple commands;
step represents a dependency group rather than an individual serial number;- same-step commands must not depend on each other’s output;
- read-only macro commands may run concurrently;
- mutating shell commands use an exclusive workspace path;
apply_patch is exclusive;- a failed
apply_patch cancels later commands;
- the runtime returns normalized per-command results.
At the exact commit used for the Colab probe, the StreamingCommandRunExecutor schedules commands considered macro-safe into concurrent batches, flushes those batches when the step changes, and explicitly halts after a failed apply_patch.
The two focused tests were:
Preserving partial results is useful. If three independent searches are issued and one executable is unavailable, the other two results should not disappear.
The more general case, however, is not simply “continue” or “stop”:
A failed
├─ B is an independent sibling → continue
├─ C requires A.Succeeded → cancel or omit
├─ D is diagnostic after A.Failed → run
├─ E is cleanup → run
└─ F requires a new decision → return to the model
A numbered step captures temporal groups, but not all result-sensitive relationships.
Workflow systems already expose vocabulary for this. Argo Workflows supports dependencies on states such as task.Succeeded, task.Failed, task.Errored, task.Skipped, and task.Omitted, including boolean combinations.
A lightweight subset of that idea could make failure propagation explicit without turning command_run into a general-purpose workflow language.
Cleanup can also be treated independently from ordinary dependencies. Argo’s exit handlers are one established example: cleanup or notification can run regardless of whether the main workflow succeeded or failed.
Hidden dependencies: resources rather than source files
A command can be read-only with respect to source files while still competing for:
- a Cargo target directory;
- a compiler or package-manager lock;
- a dependency cache;
- a test database;
- a local server port;
- a browser profile;
- a GPU;
- RAM or disk I/O;
- an API rate limit;
- a background process;
- an external account or environment.
The archived pest-character-class-coalescing agent trace contains a useful example.
Several Cargo validation operations timed out. The agent then investigated lingering Cargo/rustc processes and the shared build directory, changed its execution strategy, and used an isolated target directory.
That trace does not prove one unique root cause. It does illustrate why these should remain separate concepts:
source mutation
resource interference
A resource declaration could be small:
resources:
mutex:
- cargo-target:/repo/target
or:
concurrency_group: cargo-build
max_parallel: 1
For capacity-limited rather than exclusive resources:
resources:
semaphore:
key: external-api
slots: 2
This is a standard workflow pattern. Argo, for example, supports mutexes, semaphores, and workflow-level parallelism limits.
The runtime would not need perfect static knowledge on day one. A staged approach could be:
- built-in rules for common commands and resources;
- operation-manual declarations for project-specific tools;
- model-provided resource hints;
- runtime detection of known lock or collision messages;
- conservative fallback to macro-serial;
- trace analysis to turn repeated collisions into new rules.
Model-provided metadata should be treated as a hint rather than a trusted guarantee.
That is also how the current MCP tool annotation schema frames properties such as readOnlyHint, destructiveHint, idempotentHint, and openWorldHint: they are hints, and clients should not make security-sensitive decisions from annotations supplied by an untrusted server.
Structural output validity is not necessarily valid evidence
A runtime can often validate output structure:
- the process has an exit code;
- JSON parses;
- a test report conforms to a schema;
- an expected artifact exists;
- a patch was syntactically accepted;
- a job reached a terminal state.
For example, MCP supports an outputSchema for validating structured tool results.
Coding workflows often require a stronger semantic postcondition:
- Does the test report correspond to the current source revision?
- Was the test run against a fresh build rather than a stale binary?
- Did it validate the changed package or another workspace member?
- Is an empty search result meaningful, or was the search command unavailable?
- Did the patch implement the requested behavior rather than only apply cleanly?
- Does command-level success establish task-level success?
Those can be separated:
postcondition:
structural:
exit_code: 0
output_schema: test-report-v1
semantic:
artifact_revision: current_workspace
coverage: changed_component
interpretation: model_required
When structural validation is sufficient, the runtime can continue.
When semantic interpretation is required, that is a natural point for returning the evidence to the model.
This is also why command success alone cannot define the whole boundary. A command may exit successfully while producing evidence that invalidates every remaining step.
Long-running work probably needs a separate lifecycle
A long-running compiler, browser suite, server, or external job is different from a normal macro command.
If the model repeatedly needs to ask:
Is it finished?
Is it finished now?
What about now?
provider turns reappear as polling overhead.
A cleaner structure is:
start_job
→ task_id
runtime monitors task_id
→ completed / failed / timed_out / cancelled
model resumes only on a terminal or decision-relevant event
This resembles the standard asynchronous job pattern: a short operation starts external work, the workflow waits, and an event or callback resumes execution.
A minimal coding-agent lifecycle could be:
lifecycle: durable_job
status:
- queued
- running
- succeeded
- failed
- timed_out
- cancelled
wake_model_on:
- terminal_state
- approval_required
- unexpected_state
This would reduce polling rounds while preserving interruptibility and auditability.
Interrupt and resume also introduce idempotency requirements. The LangGraph interrupt documentation notes that nodes may be restarted during resume, so side effects before an interrupt should be idempotent, moved after the interrupt, or isolated in another node.
That becomes particularly important if Tura later expands from local repository work into:
- issue creation;
- deployment;
- cloud changes;
- database writes;
- notifications;
- external API mutations.
Benchmark interpretation and useful controls
The aggregate result is interesting because macro execution attacks a larger denominator than ordinary output compression.
The earlier token-cost thread showed why compressing one prompt fragment or one class of tool output may have a limited end-to-end effect: cached input, repeated rounds, retries, and trajectory variance can dominate total cost.
Macro execution targets:
- provider-visible model passes;
- base-context replay;
- repeated tool-choice reasoning;
- round-trip latency.
That makes it plausible that it can affect total trajectory cost more directly.
At the same time, the Tura project already makes an important distinction in its request for more benchmark data: the current comparison is between complete configurations and does not isolate command_run as the sole cause.
The proposed four-cell comparison is:
| Cell |
Configuration |
| A |
Tura with command_run |
| B |
Tura without command_run |
| C |
Original mini-swe-agent |
| D |
mini-swe-agent with the same command_run contract |
I would add a scheduler ablation inside that structure:
| Execution mode |
Model-turn batching |
Runtime parallelism |
| Primitive calls |
No |
No |
| Macro-serial |
Yes |
No |
| Macro-parallel, current classifier |
Yes |
Yes |
| Macro-parallel, explicit resources |
Yes |
Yes, resource-aware |
That separates:
- whether batching reduces provider turns and replayed context;
- whether parallelism reduces wall time;
- whether parallelism introduces contention or invalid downstream work;
- whether explicit resource and dependency contracts preserve the speedup.
The current benchmark evidence record also notes that the runtimes record commands differently, so one round is not necessarily a common atomic unit of work.
A useful report could therefore include three layers.
Provider layer
- model turns;
- uncached input;
- cached input;
- output and reasoning tokens;
- modeled cost;
- provider latency.
Runtime layer
- semantic actions;
- OS process invocations;
- parallel groups;
- wall time;
- peak concurrent processes;
- resource collisions;
- long-job wakeups or polling rounds.
Correctness and recovery layer
- verifier success;
- repeated-run variance;
- first failure;
- planned, executed, skipped, omitted, and cancelled commands;
- dependent work invalidated;
- partial results preserved;
- retries;
- duplicate side effects;
- replan reason;
- cost per verified completion.
For clarity, I would describe the reported test set as 60 sessions over 20 task IDs, with each task repeated three times, rather than 60 independent task definitions. The repetitions are useful evidence; the wording simply helps future readers interpret variance and generalization.
The five-point Direct-versus-High pass-rate difference is worth reporting, but it should probably be interpreted separately from the much larger round/token difference.
The large-scale On Randomness in Agentic Evals study reported that single-run SWE-bench-Verified pass@1 estimates could move by several percentage points depending on which run was selected. That does not invalidate Tura’s result; it supports showing distributions or intervals before treating a small pass-rate difference as a stable quality advantage.
Colab sanity-check details and limitations
The probe used a CPU-only Colab environment and cloned Tura at:
a0bdd1008e32b341850c71e2ad6d773bd118cd4a
It did not call a model or rerun the DeepSWE benchmark.
Repository behavior
Both focused Cargo tests passed:
| Behavior |
Result |
| Ordinary shell failure preserves later batch results |
Passed |
Failed apply_patch cancels later commands |
Passed |
Independent commands
Three commands each slept for the same short duration and shared no resource.
Across three repetitions:
- parallel: 3/3 succeeded each time;
- serial: 3/3 succeeded each time;
- mean parallel wall time was about 0.65 seconds;
- mean serial wall time was about 1.77 seconds.
This is the expected useful case for parallel execution.
Shared-lock commands
Three commands made no source-code changes but all required the same exclusive file lock.
Across five repetitions:
- parallel: 1/3 completed before the timeout each time;
- serial: 3/3 completed each time.
This shows only that a hidden exclusive resource can turn parallel waiting into timeout failures.
It does not show:
- that a public Cargo timeout trace had the same cause;
- that
command_run caused the reported benchmark difference;
- any token or pass-rate effect;
- that serial execution is generally preferable;
- behavior on every OS or repository.
Failed prerequisite
A separate synthetic pipeline compared:
failing prerequisite → downstream validation
Under continue-on-error, the downstream marker was created.
Under fail-fast, it was not.
Neither policy is universally correct:
- an independent diagnostic sibling may remain useful;
- a test requiring a successful build may be meaningless;
- cleanup may need to run regardless;
- an unexpected failure may require immediate replanning.
That is why typed dependency and failure semantics appear more useful than a global “continue” or “stop” switch.
One incremental implementation route
This could be introduced without immediately turning command_run into a general workflow engine.
Stage 1 — Separate batching from parallel execution
Add a mode that preserves a multi-command provider call while executing its commands serially.
That gives:
- a conservative fallback;
- a clean macro-serial benchmark control;
- continued round savings when resource independence is uncertain.
Stage 2 — Record typed outcomes
For example:
Succeeded
Failed
Errored
TimedOut
Skipped
Omitted
Cancelled
ApprovalRequired
Successful sibling results should remain available even when another command fails.
Stage 3 — Add result-sensitive dependencies
Start with a small subset:
depends_on:
- build.Succeeded
and:
on_failure:
siblings: continue
dependents: cancel
Stage 4 — Add resource groups
Begin with common development resources:
- Cargo targets;
- package-manager caches;
- ports;
- browser profiles;
- test databases;
- GPUs;
- external API pools.
Unknown commands can fall back to macro-serial execution.
Stage 5 — Add typed replan reasons
For example:
unexpected_output
dependency_failed
postcondition_failed
resource_contention
timeout
approval_required
ambiguous_target
long_job_completed
This makes boundary quality measurable.
Stage 6 — Move long-running processes into durable jobs
Use task IDs and event-driven wakeups rather than model polling.
Stage 7 — Learn from traces
Once traces record:
- why a macro stopped;
- which work became invalid;
- where resource collisions occurred;
- which commands the agent repeatedly serialized manually;
those observations can guide new static rules, operation-manual metadata, or eventually a learned boundary policy.
Each stage can retain macro-serial as a fallback.
So, for the immediate design question, my default route would be:
- keep macro execution as the provider-turn optimization;
- treat parallel execution as an optional scheduler optimization;
- return to the model on hypothesis, target, or semantic uncertainty;
- serialize or gate shared-resource and side-effectful work;
- express continuation using result states, not only step numbers;
- move long-running work into a durable job lifecycle;
- log why execution continued, stopped, skipped work, or replanned.
The core direction looks sound to me. The difficult part seems less like choosing a universal maximum macro size, and more like making the stop/replan boundary explicit, observable, and testable.