tool/execution

Concurrent batches, completion modes, atomic-attempt semantics, and output projection for tools that need more than a simple inline ToolResult::ok(...).

Concurrent Tool Batches

When the model emits multiple tool calls in one assistant turn, the runtime dispatches every call concurrently. Results are reported in the model's emission order even when calls finish in a different order.

Batch dispatch

Every tool's execute() may run concurrently with every other call from the same batch. The dispatcher collects calls as they complete, then restores source order for the reported results.

Resource ownership

A tool that owns an exclusive resource is responsible for coordinating access inside execute(). Batch dispatch does not infer resource conflicts from the tool name.

use lash::tools::ToolDefinition;

ToolDefinition::raw(
    "tool:write_file",
    "write_file",
    "Replace a file's contents.",
    input_schema,
    output_schema,
)

A batch containing several reads and writes starts all of them concurrently. If ordering matters for a resource, expose that ordering through the tool's own execution contract instead of relying on batch position.

Active vs. Deferred Completion

A tool's execute() returns a ToolResult, and the variant it returns chooses how the call completes. Most tools finish inline; a tool that waits on out-of-band work parks the call and lets an external resolver deliver the result later.

ToolResult::Done (active await)

The result is available inline and the runtime finalizes the call immediately. Build it with ToolResult::ok, ToolResult::err, ToolResult::failure, and friends.

ToolResult::Pending (deferred completion)

The tool has launched out-of-band work (a webhook, a human approval, another service) and the real outcome is delivered later against a completion key. The runtime parks the call on a durable wait and resumes it when the key resolves.

Deferred completion has one hard rule: before returning ToolResult::Pending, the tool must first obtain a completion key by calling call.context.completion_key().await. That key names the durable wait and is what the external resolver later uses to deliver the result. Returning Pending without having taken a key fails the call with the internal error pending_tool_missing_completion_key.

use lash::tools::PendingCompletion;

// Take the completion key BEFORE returning Pending, then hand it to whatever
// will deliver the result out-of-band — a webhook, a job queue, a human.
let key = match call.context.completion_key().await {
    Ok(key) => key,
    Err(err) => return ToolResult::err_fmt(format_args!("{err}")),
};
enqueue_external_work(key);

// Returning Pending without first taking the key fails the call with
// `pending_tool_missing_completion_key`.
ToolResult::pending(PendingCompletion::new())

PendingCompletion configures the wait: an optional deadline, an on_timeout behavior (ErrorAsResult feeds a timeout result back to the model; FailTurn fails the whole turn), and an on_cancel hint (CancelExternalWork by default, or Ignore to leave the out-of-band work running). Adjust it with PendingCompletion::new().with_deadline(..) and .fail_turn_on_timeout(). The remote protocol carries tool grants only; execute tools through a normal ToolProvider at the host boundary.

Atomic Tool Attempts

Attempts are atomic. In-attempt effects are opaque. Durable composition lives in the process layer. A prepared tool attempt is one journaled entry. Lash does not split the host code inside execute() into nested durable effects.

completed replay

After the tool-attempt outcome is recorded, replay returns that outcome and does not execute the tool or invoke an in-attempt provider again.

direct completion

context.direct_completions().complete(...) is an ordinary in-attempt operation. It uses the normal request planning, usage, trace, and bookkeeping paths, while the provider call runs locally under the open tool-attempt entry.

retry delay

A tool retry delay is a cancellable local sleep inside the atomic attempt. If the retry policy starts another attempt, all opaque work in that new attempt executes again.

composition

When work needs several independently durable effects, waits, or decisions, decompose it into process steps. Nested tool-batch dispatch is unavailable inside an attempt because it would create ambiguous nested durability.

Choose The Durable Boundary

Return ToolResult::ok(...) when all work belongs to one attempt. Use ToolResult::Pending when an external owner will eventually provide one final tool result. Use a Lashlang process when each operation must settle before the next begins:

process fulfill(order: Order) {
  quoted = await tools.quote({ order: order })?
  approved = await tools.request_approval({ quote: quoted })?
  receipt = await tools.charge({ approval: approved })?
  finish receipt
}

Each awaited tool call is its own atomic attempt and process state carries the durable composition. This keeps retry and crash behavior visible in the authored workflow instead of hiding it inside opaque host code.

Tool-Output Budgeting

Large tool outputs blow up provider context, model context windows, and persistent history all at once. ToolOutputBudgetPluginFactory sits in the runtime_plugin_stack() by default and projects oversized outputs into shorter forms at three different sites, so the dispatch result, the model's next request, and the session graph each see an appropriately budgeted view.

use lash::plugins::{
    ToolOutputBudgetConfig, ToolOutputBudgetPluginFactory, runtime_plugin_stack,
};

let config = ToolOutputBudgetConfig {
    limit: 32 * 1024, // default: 16 * 1024 bytes
    max_lines: 800,   // default: 400
    ..ToolOutputBudgetConfig::default()
};

let plugins = runtime_plugin_stack().configure(|plugins| {
    plugins.replace(Arc::new(ToolOutputBudgetPluginFactory::new(config)));
});

Defaults are 16 KiB and 400 lines per tool output (the size cap is bytes by default; ToolOutputBudgetMode::Tokens switches it to an approximate token count). Outputs under both caps pass through untouched; outputs over either cap keep a preview window wrapped in a "...N bytes truncated..." marker; the unit reads bytes (or tokens) when the size cap binds, lines when the line cap binds.

The plugin registers a tool-result projector that applies at three points:

RLM mode print observations go through the same projector: a Lashlang program that prints a 200 KiB blob sees the same trimmed view its next observation would carry. Replace the factory rather than appending; reg.tool_results() is exclusive (one projector per session). Plugins that need different projection logic should build on ToolOutputBudgetConfig rather than fight it.

Where Next

This page owns advanced execution behavior. Use the tools overview for registration, context capabilities, and registry state.

read on ·