Run agent tool calls in parallel: 10 turns become 2 DAG levels

TL;DR

To run agent tool calls in parallel you have to stop asking the model for one action at a time: a planner that emits a dependency graph up front turns ten sequential round trips into two execution levels. The graph then becomes the place you attach a concurrency cap, a four-dimensional budget, and a critic that is not the same role that produced the work. The architecture is the easy part — error classification and knowing what it still fails to protect you from are not.

What is a DAG agent harness?

A DAG agent harness is an agent runtime where the model plans once, returning a directed acyclic graph of tool calls with explicit dependency edges, and a separate executor walks that graph instead of asking the model what to do next each turn.

The distinction that matters: in a normal loop the model holds the control flow. In a DAG harness the graph holds it, and the graph is a data structure you can validate, schedule, cap, cost, and trace before a single call executes.

Why the loop is the bottleneck, not the model

The canonical agent loop is a while statement: call the model, get one tool_use block, execute it, append the result, call the model again. It is simple and it is debuggable, and it serializes work that has no reason to be serial.

Take the running example from Building an Advanced Agentic Harness, the post that prompted this one: an agent that compares three cities on population, timezone, and a written summary, then aggregates the lot into a report. That is nine lookups, none of which needs any other one’s output, plus one aggregation that needs all nine.

In a one-action-per-turn loop that is ten sequential model round trips. As a graph it is two levels: nine nodes that fan out, one node that joins.

The cost is not only wall-clock. Every extra turn re-sends a transcript that keeps growing, and every extra turn is another chance for the model to revise a plan it already committed to. Prompt caching softens the token bill; it does nothing about the ten serialized network hops or the drift.

How to run agent tool calls in parallel

The change is small and structural: instead of asking the model “what is the next action?”, ask it once for the whole plan, and require the answer to be a DAG of tool calls with explicit dependency edges. Three things follow immediately.

You can validate before you spend anything. A structural pass over the returned JSON catches cycles, dangling dependencies, and unknown tool names before a single call executes. A plan that fails validation costs one model call — not nine tool invocations and a confused half-finished report.

You can find the join node by type, not by trusting an ID. The resolver in the source harness locates the aggregation step by looking for the tool type, and rejects any plan containing more than one aggregator. That is the general shape of the discipline: never trust the model’s structural claims implicitly, re-derive them from something you control.

You get an execution order for free. Repeatedly take every node whose dependencies are satisfied, run that whole set concurrently, wait, recompute. Each round is a level.

This level-synchronous walk is not the tightest possible scheduler — a fast node waits for its slow siblings before the next level starts. But it is trivial to reason about and trivial to trace, and for a graph two levels deep the difference is nil.

Cap the concurrency before you need to

A ready set of nine nodes launched at once is nine simultaneous API calls. That is a rate limit incident wearing a trench coat.

The fix is a semaphore in the executor, not an instruction in the planner prompt. The source harness pins MAX_CONCURRENT = 5, so a nine-wide level becomes two waves. Put the cap in code, because the planner will cheerfully emit a graph wider than your quota and you do not want correctness depending on the model’s restraint.

The other half of this is that most real tool code is synchronous — a database client, a requests call, somebody’s SDK. You do not need to rewrite it. asyncio.to_thread hands a blocking function to a worker thread and returns an awaitable, so sync tools drop into asyncio.gather beside the async ones. The concurrency model stays uniform; the tools stay untouched.

Typed tools are what make the graph safe

Every tool declares its arguments as a schema — Pydantic in the source, though the mechanism matters more than the library — and one declaration pays off three times.

Runtime validation runs before execution, so a hallucinated argument fails fast instead of halfway through a side effect. The same model serializes to JSON Schema, which is the exact shape the tool-calling APIs want, so the planner’s tool list and the executor’s validator cannot drift apart.

And the declaration is a natural home for a cost hint: the harness tags a population lookup at 0.1, a written city summary at 1.0, and the final aggregation at 2.0. Those hints let the budget see the future. A planner that emits four aggregations has emitted an 8.0 plan, and you can know that before the first call rather than after the invoice. Anthropic’s guidance on writing tools for agents makes the adjacent point about descriptions: the schema is not paperwork, it is the interface the model actually programs against.

Budget as one number

Track four resources — tokens, tool calls, wall-clock seconds, estimated dollars — and collapse them into a single pressure value: the maximum utilization across all four.

The max is the right operator because you stop when the first resource runs out. Averaging would let a plan at 99% of its token budget and 1% of its time budget report a comfortable 50%, which is precisely the moment it dies.

One scalar makes graceful degradation a ladder instead of a nested conditional:

Pressure	Behavior
below 0.7	Full pipeline — deterministic checks and the LLM judge
above 0.9	Skip the expensive judge, keep the deterministic checks
at 1.0	Halt and return partial results with the trace

Note what the middle rung does. It does not stop the run and it does not silently lower quality — it drops the most expensive optional stage and keeps the free one. Returning a partial report plus the trace that explains why it is partial beats a timeout, and it is the behavior most loops are missing.

Four error classes, four different recoveries

Retrying everything is the default failure mode of agent code, and it is wrong in three of the four cases. Classify first, then recover:

• Transient — rate limits, timeouts, 5xx. Exponential backoff with jitter. The call was fine; the world was busy.
• Tool misuse — schema validation failed, wrong argument type. Retrying identically loops forever. Feed the structured validation error back so the model can correct the call it got wrong.
• Missing information — the model invented an entity your tool cannot resolve. No amount of retrying conjures the city into existence. Re-plan, explicitly informed that the thing is unavailable.
• Policy violation — halt. This is the one class where the correct number of retries is zero, and where an automatic recovery path is a liability rather than a feature.

The bug this prevents is quiet and expensive: a validation error treated as transient, retried five times with backoff, burning budget on a call that was never going to succeed.

Verify in two tiers, and never let the worker grade itself

Split verification by cost. Deterministic checks are free — is every requested city present, is the JSON well-formed, did any node return empty. Run those first. The LLM judge is expensive, so spend it only on output that already survived the free tier.

The structural rule underneath is that the Critic is a separate role from the Worker. A generator asked to grade its own output measures its own confidence, not its correctness, and it will pass work a fresh reader would reject. Separate prompts, separate contracts, ideally separate model tiers. This is the same separation that makes an eval framework worth building rather than vibes-checking a transcript.

Roles generally are the quiet win here: a Planner that only emits DAGs, a Worker that only executes them, and a Critic that only evaluates. Each has one contract, so each can be tested alone and swapped alone — which is also what makes harness design, not model choice, the variable that moves your numbers.

What this still does not give you

The source post is honest about its own edges, and the honesty is worth repeating, because these are the gaps that bite in production:

• Memory is in-process. Working, episodic, and semantic tiers with a 4,000-character assembly budget and top-3 retrieval per tier — all of it evaporates on restart. Real deployments need a vector store behind it.
• Tool outputs are trusted as instructions. Anything a tool returns flows into the next prompt unsandboxed, which is the prompt-injection surface write-capable tools have to be designed against.
• Irreversible actions execute without approval. A DAG node that sends an email or moves money needs a human in the delegation path, not a critic reviewing it afterwards.
• Token accounting is estimated from character counts. Fine for relative pressure, useless for a real bill. Read the SDK’s usage metadata.
• There is no eval harness. The post ships architecture and traces, not benchmarks — it contains no success rates or latency comparisons, and it says so directly, deferring evaluation to a follow-up.

That last gap is the honest summary of the whole exercise. A DAG, a semaphore, and a pressure gauge give you an agent that is bounded and observable, and bounded-and-observable is genuinely the thing most agent code lacks. It is not the same as correct, and the distance between a demo that works and a system that keeps working is exactly where most agent projects stall.

Build the graph first anyway. You cannot evaluate a system whose failures you cannot see, and the trace that falls out of a DAG executor — step ID, parent ID, role, latency, tokens, cost, pressure snapshot, verdict — is the first artifact in this stack an eval could actually consume.

FAQ

Why can’t I just run tool calls in parallel inside the normal agent loop? Because the normal loop has no idea which calls are independent. It asks the model for the next action, executes it, and asks again — the dependency information never exists in a form your executor can read. Parallelism requires the model to commit to a plan up front, which is exactly what a DAG planner does: it returns the whole graph, and the executor reads dependencies off the edges rather than guessing.

What is a level-synchronous DAG walker? It is the simplest correct way to execute a dependency graph. You repeatedly find every node whose dependencies are already satisfied, launch all of them concurrently, wait for that batch to finish, then recompute the ready set. Each batch is one level. It is not the most aggressive scheduler possible, since a fast node waits for its slow siblings, but it is easy to reason about and easy to instrument.

How do I stop parallel execution from blowing up my rate limits? Cap concurrency with a semaphore rather than trusting the graph to stay small. A ready set of nine nodes launched at once is nine simultaneous API calls; a semaphore of five turns that into two waves. The cap belongs in the executor, not in the planner prompt, because the planner will happily emit a graph wider than your quota and you do not want correctness depending on the model’s restraint.

What is budget pressure and why collapse four dimensions into one number? Pressure is the maximum utilization across every resource you track — tokens, tool calls, wall-clock time, and estimated spend. Collapsing them to one number works because you die when the first resource runs out, so the max is the only figure that predicts a stop. It also gives you a single scalar to threshold on, which is what makes graceful degradation expressible in two lines instead of a nested conditional.

Should the same model that produced the output also grade it? No. Separate the Worker that generates from the Critic that evaluates, with different system prompts and ideally different model tiers. A generator grading its own work is measuring its own confidence, not its correctness, and it will confidently pass output that a fresh reader would reject. Run the free deterministic checks first and spend the expensive judge only on what survives them.

Does this architecture make an agent production-ready? It makes it observable and bounded, which is necessary but not sufficient. In-process memory, tool outputs trusted as instructions, irreversible actions without human approval, and token counts estimated from character lengths are all still open. Most importantly there is no evaluation harness here, so you can demonstrate the system works on a case — not that it works in general.

Sources

• Data4Sci, “Building an Advanced Agentic Harness” — the primary source for this post: the seven primitives, the nine-call city example, MAX_CONCURRENT = 5, the 0.1 / 1.0 / 2.0 cost hints, the four-dimensional budget and its 0.7 / 0.9 / 1.0 degradation thresholds, the four error classes, and the acknowledged limitations.
• Python documentation, asyncio coroutines and tasks — asyncio.gather, asyncio.to_thread, and the semaphore pattern behind the level-synchronous walker.
• Anthropic Engineering, Writing tools for agents — tool descriptions and response shaping as the interface the model programs against.