Agent Loop¶

A reasoning agent must decide, each step, what to work on, what context to show the model, and when to stop. In most agent frameworks these decisions are made by the LLM itself --- it picks the next action, judges its own progress, and decides when to conclude. This makes the system unpredictable: the model may loop on dead ends, stop too early, or misjudge its own resource consumption.

Lethe externalizes all of these decisions. The agent loop is the orchestrator that selects tasks, composes context, queries the LLM, applies the resulting operations to the graph, and computes signals for the next iteration. The loop runs until the budget manager determines that work should stop --- either because the evidence has converged, resources are exhausted, or the LLM has signaled completion.

The loop coordinates all subsystems: context composition for evidence selection, DIN protocol for LLM communication, embedding algebra for signal computation, and the research engine for tool-driven web research. It is the composition root --- the only module that calls into every other package.

Iteration Cycle¶

Each iteration proceeds through ten steps:

flowchart TD
    S1["1. Ready Set<br/><small>pending tasks with resolved deps</small>"]
    S2["2. Task Selection<br/><small>gap-adaptive GA + depth + frequency</small>"]
    S2b["3. SIMILAR_TO Sync<br/><small>cross-parent edge maintenance</small>"]
    S3["4. Context Expansion<br/><small>PPR from seed nodes</small>"]
    S4["5. Context Scoring & Packing<br/><small>knapsack with MIG</small>"]
    S5["6. DIN-Read Serialization<br/><small>graph → text for LLM</small>"]
    S6["7. LLM Reasoning<br/><small>structured JSON response</small>"]
    S7["8. Parse & Validate<br/><small>Pydantic + repair loop</small>"]
    S8["9. Apply Operations<br/><small>graph mutations + tool calls</small>"]
    S9["10. Signal Computation<br/><small>embedding algebra → budget check</small>"]

    S1 --> S2 --> S2b --> S3 --> S4 --> S5 --> S6 --> S7 --> S8 --> S9
    S9 -->|"continue"| S1
    S9 -->|"terminate"| Done["Assembly"]

Step 1 --- Ready Set¶

The orchestrator queries Neo4j for all tasks in PENDING status whose dependencies (via DEPENDS_ON edges) are all in DONE status. These form the ready set --- the pool of tasks eligible for selection. ACTIVE tasks (currently being worked on) and FAILED tasks (marked as dead ends) are excluded: ACTIVE tasks will be retried only if the LLM emits a continue operation, and FAILED tasks are permanently immutable. This dependency-gated approach ensures the agent never works on a task whose prerequisites are incomplete.

Step 2 --- Task Selection¶

Each ready task is scored through a multi-factor pipeline (using signals from the previous iteration):

1. Warp --- The task embedding is warped toward unexplored goal dimensions via geodesic interpolation, controlled by budget pressure and the previous iteration's gap direction.

2. Goal alignment --- Cosine similarity of the warped embedding against the goal query embedding, mapped to [0,1] via normalize_cosine.

3. Evidence deficit --- Applied as 0.05 + 0.95 × deficit where deficit = 1 − SVD_coverage_topk. Tasks with insufficient evidence score higher. The 0.05 floor prevents zero-deficit degeneracy when all tasks are well-covered.

4. Ranking --- _rank_tasks() in loop/iterate.py combines these factors:

where \(\epsilon = 0.01\) (_TASK_GA_FLOOR) ensures diminishing returns applies even when all GAs are zero, \(w_{\text{depth}} = 1/(1+\text{depth})\) from BFS root distance, and \(n_{\text{productive}}\) only increments when the task produces at least one applied operation. The selected task is marked ACTIVE before context expansion.

Step 3 --- SIMILAR_TO Sync¶

Before PPR expansion, the SIMILAR_TO sync pipeline runs in three phases: prepare_similar_sync (read graph state), enrich_plan_with_nli (classify edge pairs via DeBERTa, outside any Neo4j lock), and commit_similar_sync (write enriched edges). A fourth phase, refine_strengths_nli, modulates evidence strength based on NLI corroboration — evidence corroborated by independent sources receives a smooth boost. This ensures the semantic graph reflects evidence ingested during the previous iteration, so that PPR can traverse freshly created cross-topic links with NLI-informed weights. The sync is intentionally placed after task selection (which does not depend on SIMILAR_TO) and before context expansion (which does).

Step 4 --- Context Expansion¶

Starting from the selected task, PPR (Personalized PageRank) expands outward through the graph to discover relevant context nodes. PPR is a biased random walk: it starts from a set of seed nodes (the selected task and its close neighbors), follows edges with probability proportional to cosine similarity, and periodically "teleports" back to the seeds. The result is a relevance distribution over all reachable nodes that naturally decays with graph distance. See Context Composition --- PPR Expansion for the full algorithm.

Step 5 --- Context Scoring and Packing¶

Expanded candidates are scored and packed into the token budget. The scoring function receives budget pressure \(p\) directly. Inside scoring, the GA weight shift uses \(p^2\):

This quadratic coupling means pressure stays low through most of the budget and accelerates the GA weight shift in context scoring near exhaustion. See Context Composition for the full pipeline.

Step 6 --- DIN-Read Serialization¶

The packed subgraph is serialized into DIN-Read format. Before serialization, the orchestrator constructs structured warn lines that give the LLM awareness of its context:

• Subtask status with qualitative coverage tags and truncated results
• Parent goal description (for subtasks)
• Dead-end siblings that the LLM should avoid repeating
• Sibling tasks working on other parts of the parent goal
• Zero-yield hint when previous searches returned no new evidence

For subtasks, the =GOAL line becomes YOUR TASK: {description} (Parent goal for context: {root_goal}), ensuring the LLM understands its narrower scope within the broader objective.

No numeric scores, budget percentages, or embedding distances appear in the serialized context. The LLM sees only task descriptions, evidence content, relationship lines (>SUB, >DEP), and tool availability.

Step 7 --- LLM Reasoning¶

The serialized context, system prompt, and JSON schema are sent to the reasoning LLM as a single-turn [system, user] message pair. The LLM responds with a JSON object containing DIN operations.

Thinking budget (two stages): The base allocation scales with the previous iteration's complexity score (iteration N uses complexity from iteration N-1, since the current iteration's signals have not been computed yet):

Then adjust_thinking_budget reduces the base under pressure:

Early iterations with high novelty receive budgets near max_tokens (12288). Late iterations under heavy pressure receive budgets near min_tokens (6144), encouraging terse synthesis over expansive exploration.

Failure recovery: If the LLM call fails entirely (HTTP error, timeout, or unrecoverable response error), the selected task is reverted to PENDING status and the iteration still counts as completed. The next iteration will re-select from the ready set --- possibly choosing the same task with fresh context, or a different one if the ranking has shifted.

Step 8 --- Parse and Validate¶

The LLM's JSON response is parsed and validated against Pydantic models. If validation fails, a repair loop formats the errors and resubmits with halved thinking budget:

flowchart LR
    Response["LLM JSON"] --> Parse["Pydantic<br/>Validate"]
    Parse -->|"valid"| Ops["Operations"]
    Parse -->|"errors"| Format["Format<br/>Error Prompt"]
    Format --> Retry["LLM Retry<br/><small>halved thinking</small>"]
    Retry --> Parse

Step 9 --- Apply Operations¶

Validated operations are applied to the graph within a write transaction. Tool calls (currently only web_search) execute in parallel via execute_calls_and_ingest after graph ops commit. The research engine runs its full loop for each call, and the resulting evidence nodes are ingested into the graph before proceeding to signal computation.

Embedding precomputation: Before applying operations, all embed_text fields are sent to the embedding model in batch. This amortizes embedding latency across the operation set rather than embedding one-by-one during graph writes.

Task deduplication: When a create operation is applied, the new task's embedding is compared against all existing tasks. If cosine similarity exceeds semantic_dedup_threshold (default 0.95), the creation is skipped and the new short ID is mapped to the existing task. This prevents the LLM from creating semantically identical subtasks across iterations.

Cycle detection: After all operations are applied, the orchestrator checks for cycles in SUBTASK_OF and DEPENDS_ON edges. If a cycle is detected, the entire write transaction is rolled back, and the operations are discarded.

Step 10 --- Signal Computation¶

compute_iteration_signals() runs the full embedding algebra pipeline (see Embedding Algebra). The resulting signals feed into the budget manager's termination check.

Budget Management¶

The iteration cycle above produces signals; the budget manager consumes them to answer the central question: should the agent keep working? It tracks resource consumption (iterations, tokens, wall clock) and convergence (from the embedding algebra), unifying three pressure sources into a single termination potential \(\psi\).

Resource Progress¶

Two resource dimensions contribute via bottleneck maximum:

Whichever resource is most depleted dominates. Tokens are tracked (record_tokens) but do not feed resource progress — with thinking-mode models, token consumption varies too much between iterations to give stable pressure. Token budgets operate via the thinking budget reduction path instead.

Accelerated Progress¶

Convergence accelerates perceived progress, creating urgency even when raw resources remain:

This means high convergence (the evidence subspace has stabilized) is treated as if resources were further depleted --- no point continuing when the answer is not improving.

Budget Pressure¶

The progress value feeds into the budget horizon function:

with \(\alpha = 4.0\) from TerminationConfig.budget_alpha. Pressure grows sub-linearly in the middle (plenty of room to explore) but super-linearly near exhaustion (aggressive synthesis). This curve shapes all downstream behavior --- context weights shift toward goal-aligned evidence, task warping intensifies, and exploration diminishes.

Termination Potential¶

The unified termination potential combines convergence signals with resource pressure:

where \(\text{effective\_pressure}\) is pressure urgency attenuated by a quality gate that ensures convergence quality must be present before pressure alone can trigger termination:

The quality gate smoothly attenuates pressure when convergence quality (\(\text{convergence} \times \text{plateau}\)) is low, preventing premature termination from time pressure alone. When quality is high (\(> 0.15\)), pressure operates at near-full strength. Other terms:

• conv_signal = convergence score \(\times\) plateau indicator \(\times\) iter_readiness
• iter_readiness = \(\text{soft\_gate}(\text{iterations}, \text{min\_iters} - 0.5,\ T{=}4)\) --- ramps from 0 to 1 as iterations approach the minimum threshold
• pressure_urgency = \(\text{clamp}_{01}(\max(\text{soft\_gate}(\text{pressure}, 0.85,\ T{=}8),\ \text{soft\_gate}(\text{resource\_progress}, 0.95,\ T{=}12)))\)

Termination occurs when \(\psi > \text{threshold}\) (default 0.82). The complement-product form with quality gating ensures that strong convergence triggers termination naturally, while pressure alone can only trigger termination when convergence quality is already moderate.

Hard Limits¶

A 2.0x OOM safety cap on wall-clock time provides a hard stop for catastrophic runaway. The root task reaching DONE status triggers immediate termination (the LLM explicitly marked the goal as complete). If no tasks remain in PENDING or ACTIVE status (no_ready_tasks), the loop exits --- preventing stalls when all tasks are completed or failed without the root being explicitly marked DONE.

Budget Check Positioning¶

The termination check runs at the start of each iteration, before task selection. This means the signals from the previous iteration (convergence, zero-yield count, root status) determine whether to proceed. The thinking budget is also adjusted at iteration start based on current pressure.

Zero-Yield Tracking¶

Consecutive iterations where tool calls return no new evidence boost the convergence score via an additive term: \(\text{boosted} = \text{clamp}_{01}(\text{score} + 0.08 \times n_{\text{consecutive}})\). This prevents infinite search loops. The boost is small per iteration but accumulates: after 3 consecutive zero-yield iterations the boost reaches 0.24, meaningfully accelerating termination pressure.

Session Dependencies¶

The SessionDeps dataclass carries all shared state through the iteration cycle:

Created at bootstrap. The session_pressure and time_budget_s fields are updated between iterations via dataclasses.replace(), propagating budget state to the research engine without mutating the original.

Further Reading¶

• Ousterhout, J. A Philosophy of Software Design. Yaknyam Press, 2018 --- The depth-as-leverage principle behind Lethe's module boundaries and single-interface control plane.
• BAVT. "Spend Less, Reason Better: Budget-Aware Value Tree Search for LLM Agents." 2026 --- Budget-conditioned exploration-to-exploitation via power-based scaling; related approach to Lethe's geodesic warp under budget pressure.
• Fang, Z. et al. "Inference-Time Budget Control for LLM Search Agents." 2026 --- Dual-budget (tool calls + output tokens) control via task-level value-of-information scoring. Closest recent peer to Lethe's approach of using budget pressure to affect what gets retrieved/packed rather than prompting "you have N iterations left."
• Wu, Y. et al. "ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents." 2026 --- Treats context management as a budget-constrained sequential decision: when to compress history vs. append observations. Lethe avoids growing chat history entirely (graph projection each iteration), solving the same tradeoff via submodular packing.
• Yao, S. et al. "ReAct: Synergizing Reasoning and Acting in Language Models." ICLR, 2023 --- The reasoning-action loop pattern that agent frameworks build on; Lethe replaces free-form ReAct with structured DIN operations.
• Wei, J. et al. "Chain-of-Thought Prompting Elicits Reasoning." NeurIPS, 2022 --- Foundation for thinking-mode LLM calls; Lethe uses llama.cpp thinking mode to separate reasoning trace from structured output.