Architecture¶

Lethe is organized as 11 packages under src/lethe/, with import boundaries enforced by the Makefile. The architecture follows a three-layer separation: the LLM reasons in natural language, the orchestrator computes in vector space, and I/O adapters handle external service communication.

Each package has a dedicated documentation page that covers its design decisions in depth:

loop/ --- Agent Loop (iteration cycle, budget management)
embed/ --- Embedding Algebra (signal pipeline, spectral methods)
context/ --- Context Composition (PPR, knapsack packing)
search/ --- Research Engine (discovery, streaming, convergence)
graph/ --- Graph Memory (Neo4j schema, evidence lifecycle)
din/ --- DIN Protocol (structured LLM communication)
llm/ --- Stateless HTTP client for the reasoning model (thinking extraction, repair)
config/ --- Configuration (complete parameter reference)

Package Map¶

src/lethe/
 |
 |-- main.py              Session lifecycle: bootstrap, iterate, finalize
 |-- server.py            HTTP adapter: Open Responses API with resumable streaming
 |-- chunk.py             Paragraph-group chunking + page embedding
 |
 |-- config/              Frozen configuration hierarchy
 |     +-- config.py      Single source of truth for all parameters
 |     +-- system_prompt   LLM instruction set (DIN protocol rules)
 |
 |-- graph/               Neo4j substrate
 |     |-- schema.py      Type vocabulary (TaskNode, EvidenceNode, Edge)
 |     |-- store.py       Sole mutation path (ACID transactions)
 |     |-- queries.py     Read-only graph queries + derivations
 |     |-- indexes.py     SIMILAR_TO edge synchronization
 |     +-- evidence_ingest.py  Chunk -> embed -> dedup -> persist
 |
 |-- din/                 LLM <-> orchestrator protocol
 |     |-- schema.py      Pydantic models for 8 JSON operation types
 |     |-- operations.py  Frozen dataclass operation types
 |     |-- parse_json.py  JSON extraction, validation, repair
 |     +-- read.py        Graph -> DIN-Read text serialization
 |
 |-- embed/               Embedding algebra (quantitative control plane)
 |     |-- signals.py     compute_iteration_signals() + EMATracker
 |     |-- static_ops.py  Pure vector operations (GA, cosine, centroid)
 |     |-- batch_ops.py   Set-level operations (novelty, gap projection)
 |     |-- spectral.py    SharedSVD, eigengap, Grassmannian convergence
 |     |-- smooth.py      C-infinity primitives (soft_gate, budget_horizon)
 |     +-- client.py      HTTP client for llama.cpp embedding server
 |
 |-- context/             Evidence selection for LLM prompts
 |     |-- __init__.py    compose_context() pipeline + GA/PPR scoring
 |     |-- knapsack.py    Greedy MIG packing + AdaGReS redundancy
 |     +-- expand.py      PPR-based candidate expansion
 |
 |-- loop/                Iteration coordinator
 |     |-- iterate.py     Composition root: select -> context -> LLM -> apply
 |     |-- apply_ops.py   DIN operation dispatch to graph mutations
 |     +-- budget_mgr.py  Token/iteration/wall-clock tracking + convergence
 |
 |-- llm/                 Reasoning model client
 |     |-- client.py      Stateless HTTP, thinking extraction, repair retries
 |     +-- embed_text.py  LLM projection of content to embed-optimized text
 |
 |-- compose/             Final answer assembly
 |     +-- assembly.py    DFS section collection + MMR ordering + LLM compose
 |
 |-- search/              Zero-LLM web research engine
 |     |-- engine.py      Research loop coordinator
 |     |-- provider.py    SearXNG metasearch adapter
 |     |-- fetch.py       CloakBrowser content extraction
 |     |-- schema.py      Research domain types
 |     +-- adapters/      Format-specific extraction (HTML, PDF, video)
 |
 |-- tools/               Tool registry + execution
 |     +-- executor.py    web_search dispatch + semantic query dedup
 |
 +-- drivers/             I/O adapters
       |-- browser.py     CloakBrowser CDP client
       |-- embed.py       Coalescing embed client (batched concurrent calls)
       +-- stt.py         whisper.cpp HTTP client

The package map above shows what each module contains. The dependency graph below shows how they connect --- which packages import which, and where the enforced boundaries prevent coupling.

Dependency Graph¶

The following diagram shows the import relationships between packages. Each arrow points from a dependent to its dependency.

flowchart TB
    SERVER["server.py"]
    MAIN["main.py"]
    LOOP["loop/iterate"]
    CTX["context/"]
    DIN["din/"]
    LLM["llm/"]
    EMBED["embed/"]
    GRAPH_SCHEMA["graph/schema"]
    GRAPH_STORE["graph/store"]
    GRAPH_QUERIES["graph/queries"]
    APPLY["loop/apply_ops"]
    TOOLS["tools/"]
    SEARCH["search/"]
    COMPOSE["compose/"]
    DRIVERS["drivers/"]
    BUDGET["loop/budget_mgr"]
    CONFIG["config/"]

    SERVER --> MAIN
    MAIN --> LOOP
    MAIN --> COMPOSE
    MAIN --> SEARCH
    MAIN --> DRIVERS
    LOOP --> CTX
    LOOP --> DIN
    LOOP --> LLM
    LOOP --> EMBED
    LOOP --> GRAPH_SCHEMA
    LOOP --> GRAPH_QUERIES
    LOOP --> APPLY
    LOOP --> BUDGET
    APPLY --> GRAPH_STORE
    APPLY --> TOOLS
    TOOLS --> SEARCH
    CTX --> GRAPH_SCHEMA
    CTX --> EMBED
    DIN --> GRAPH_SCHEMA
    SEARCH --> DRIVERS
    SEARCH --> EMBED
    MAIN --> GRAPH_STORE
    GRAPH_STORE --> GRAPH_SCHEMA
    GRAPH_QUERIES --> GRAPH_SCHEMA
    ALL["All packages"] -.-> CONFIG

    style EMBED fill:#e1f5fe
    style CTX fill:#e8f5e9
    style DIN fill:#fff3e0
    style SEARCH fill:#fce4ec
    style GRAPH_SCHEMA fill:#f3e5f5
    style GRAPH_STORE fill:#f3e5f5

Enforced Boundaries¶

The Makefile encodes import restrictions that prevent layer violations. These run on every make check:

Rule	Restriction	Rationale
P1	`search/` must not import `loop/` or `din/`	Research engine is structurally independent of the agent loop
P4	`din/` must not import `embed/`, `llm/`, or `graph.store`	Protocol layer is pure data transformation
P6	`tools/` must not import `graph.store`	Tools interact with the graph only through the orchestrator
P7	`llm/` must not import `graph/`	LLM client is a stateless HTTP adapter
P8	`embed/` must not import `graph.store`, `din`, `llm`, or `loop`	Embedding algebra is pure computation
P9	`context/` must not import `llm/`	Context composition is numeric, not LLM-driven
P11	`drivers/` must not import `loop/`, `tools/`, or `main`	I/O adapters are leaf dependencies

These boundaries encode the design: the LLM is a stateless function call, the orchestrator's numeric control plane (embed/, context/) is separated from its structural control plane (din/, loop/), and the research engine shares algebra but not orchestration with the agent loop.

Three-Layer Separation¶

flowchart TB
    subgraph L1["Layer 1: LLM (Decision Plane)"]
        LLM_CALL["Single-turn LLM call<br/>DIN-Read → JSON ops"]
    end

    subgraph L2["Layer 2: Orchestrator (Control Plane)"]
        TASK_SEL["Task selection<br/>GA × depth × diminishing"]
        CTX_COMP["Context composition<br/>PPR → MIG knapsack"]
        BUDGET_MGR["Budget management<br/>exponential horizon"]
        SIGNALS["Embedding algebra<br/>compute_iteration_signals()"]
        CONVERGENCE["Convergence detection<br/>3-signal weighted blend"]
    end

    subgraph L3["Layer 3: I/O Adapters"]
        BROWSER["BrowserClient<br/>CloakBrowser CDP"]
        EMBED_C["CoalescingEmbedClient<br/>batched embedding"]
        STT_C["STTClient<br/>whisper.cpp"]
    end

    LLM_CALL -->|DIN JSON ops| L2
    L2 -->|DIN-Read context| LLM_CALL
    L2 --> L3
    L3 --> L2

Layer 1: LLM (Decision Plane)¶

The LLM receives a serialized graph context (DIN-Read format) and emits structured JSON operations. It sees task descriptions, evidence content, relationship structure, and tool availability. It does not see:

Budget counters or iteration numbers
Embedding similarity scores or convergence metrics
Token budgets or pressure values
Any numeric control signals

This information asymmetry is deliberate. The LLM reasons about the problem domain --- which subtasks to create, which evidence to synthesize, when to search for more information. The orchestrator decides how much reasoning budget to allocate, which task to present, and when to stop.

Layer 2: Orchestrator (Control Plane)¶

The orchestrator operates in embedding space through vector operations organized into static, temporal, batch, and spectral categories:

Task selection --- goal alignment warped by gap direction, weighted by evidence deficit and graph depth (see Agent Loop)
Context composition --- PPR expansion, GA/PPR scoring, submodular knapsack packing under token budget (see Context Composition)
Budget management --- exponential horizon curve, convergence score from 3 weighted signals (GA, stability, recon)
Thinking budget --- adaptive token allocation from novelty signal
Termination --- continuous convergence detection blending goal alignment, centroid stability, and reconstruction coverage (see Budget Management)

Layer 3: I/O Adapters¶

The drivers/ package wraps external services behind narrow interfaces:

BrowserClient --- Playwright CDP connection to CloakBrowser with isolated contexts, PDF direct HTTP, and media interception
CoalescingEmbedClient --- batches concurrent embedding requests with debounce to reduce round trips to the GPU
STTClient --- whisper.cpp HTTP client for audio/video transcription

These adapters are the only modules that perform network I/O to non-LLM services.

Session Lifecycle¶

A complete session proceeds through three phases:

Bootstrap¶

Before the first iteration, the system establishes all dependencies and creates the initial graph state:

Service health --- connect to Neo4j, probe /health on both the LLM and embedding servers. If either is unreachable, the session fails immediately with a clear error rather than failing mid-research.
Graph reset --- clear any previous session's nodes and edges, ensuring a clean slate.
Root task --- create the root TaskNode with the user's goal as its description. Embed the goal in both document-prefix and query-prefix spaces (producing goal_doc_emb and goal_query_emb), which serve as the fixed reference points for all goal alignment computations.
Budget initialization --- configure the BudgetManager with iteration limits, token caps, and wall-clock bounds from TerminationConfig.
Research engine --- wire the ResearchEngine with its SearXNG provider, CloakBrowser client, and the session's goal embeddings.

Iteration Loop¶

Each iteration follows a fixed pipeline:

flowchart LR
    CHECK["Check termination<br/>ψ from prev signals"]
    READ["Tx1: Read graph<br/>ready_set"]
    SELECT["Select task<br/>rank by GA×depth×1/√(1+n)"]
    SYNC["Sync SIMILAR_TO"]
    EXPAND["PPR expansion<br/>cosine-weighted transitions"]
    SCORE["Score + Pack<br/>MIG + AdaGReS β"]
    SERIAL["DIN-Read<br/>U-shape zones"]
    LLM_C["LLM call<br/>adaptive thinking"]
    PARSE["Parse + Validate"]
    APPLY["Tx2: Apply ops<br/>+ tool calls inline"]
    SIGNALS["Compute signals<br/>embedding algebra"]

    CHECK -->|"continue"| READ --> SELECT --> SYNC --> EXPAND --> SCORE --> SERIAL --> LLM_C --> PARSE --> APPLY --> SIGNALS
    SIGNALS -->|"next iter"| CHECK
    CHECK -->|"terminate"| Done["Assembly"]

The diagram compresses the 10-step iteration cycle into its key phases: termination check, graph read, task selection with SIMILAR_TO sync, PPR expansion and knapsack packing, DIN serialization, LLM reasoning with adaptive thinking budget, parse and validate, graph mutation with inline tool calls, and signal computation. Two Neo4j transactions bracket the LLM call: Tx1 reads the graph state, Tx2 writes mutations --- no transaction is held while the LLM is thinking.

Assemble and Finalize¶

After the loop terminates, assemble_answer collects sections via DFS, orders them by MMR, groups by token budget, and composes the final answer through a dedicated LLM call. Then _finalize_graph marks remaining non-terminal tasks as DONE, fills missing result embeddings, and synchronizes SIMILAR_TO edges. Assembly precedes finalization so the composition LLM sees the graph in its working state, not the cleaned-up version.

DIN Protocol¶

The Directed Interaction Notation protocol governs LLM-orchestrator communication via 8 typed JSON operations (DIN-Write) and a flat line-oriented text format for graph serialization (DIN-Read). Key design constraint: no numeric scores, budget percentages, or embedding distances appear in the serialized context --- quantitative control happens entirely in the embedding algebra, not in the prompt. See the full specification for operation types, validation, and serialization format.

LLM Client¶

The llm/ package is the sole module that communicates with the reasoning model. It is a stateless HTTP client --- each call is an independent single-turn completion with no conversation history.

Thinking-mode extraction. Qwen's reasoning mode produces extended <think> blocks before the structured output. The client extracts thinking content from either the reasoning_content field (native API) or by parsing <think>...</think> tags from the response text. The thinking content is logged but never fed back to the model --- feeding it back would consume context window budget on self-referential reasoning rather than new evidence, and would make each call dependent on the previous call's internal monologue, breaking the stateless reproducibility guarantee.

Adaptive thinking budget. The number of reasoning tokens allocated scales with the previous iteration's novelty signal. Early iterations with high novelty receive budgets near max_tokens (28672). Late iterations under pressure receive budgets near min_tokens (8192), encouraging synthesis over exploration.

Structured parse and repair. The JSON response undergoes brace-matching extraction, Pydantic validation, and graph-reference checking. If parsing fails, a repair prompt containing the specific errors is sent with halved thinking budget and exponential backoff. Up to max_repair_attempts retries are attempted before the iteration is skipped.

Test seam. FixtureLLMClient satisfies the same LLMClient protocol, enabling deterministic tests without an LLM server.

Tools and Idempotency¶

The tools/ package manages tool registration, dispatch, and query deduplication. Currently, a single tool is implemented: web_search, which delegates to the research engine.

Tool definitions. Each tool definition specifies name, description, and argument schema. Tool names are included in the DIN-Read serialization via tool_hints.

Semantic query deduplication. The IdempotencyStore prevents redundant research by tracking past queries at two levels:

Exact match --- identical {task_id}:{tool_name}:{params_json} keys are skipped. The key is scoped to the executing task, not the parent.
Semantic match --- queries with cosine similarity above 0.92 to any previous query for the same task are skipped, preventing semantically identical but lexically different queries from triggering duplicate research

When a query passes dedup, the research engine is invoked with the session's current budget pressure and wall-clock time remaining. Results are returned as ToolResult objects containing chunked, embedded, and strength-scored evidence items ready for graph ingest.

Final Answer Assembly¶

The compose/ package assembles the final answer after the iteration loop terminates. The pipeline has four stages:

flowchart LR
    DFS["DFS Section<br/>Collection"]
    MMR["MMR<br/>Ordering"]
    GROUP["Budget<br/>Grouping"]
    LLM_C["LLM<br/>Composition"]

    DFS --> MMR --> GROUP --> LLM_C

DFS section collection --- a depth-first walk of the task tree collects content from two sources: DONE subtasks contribute their result_summary (plus an evidence appendix for data fidelity), while leaf tasks without a result contribute their top evidence directly (up to 20 items, 600 chars each). The traversal order matches the decomposition hierarchy, producing a natural outline.
MMR ordering --- Maximal Marginal Relevance (Carbonell and Goldstein, 1998) reorders sections by iteratively selecting the item that maximizes relevance to the goal minus similarity to the already-selected set. This diversity-aware reranking prevents the final answer from clustering related findings at the expense of breadth. DFS alone would follow the decomposition tree, which may group similar subtasks adjacently; MMR interleaves them. Sections with result_embedding (computed from completed task results) are preferred over raw evidence.
Budget grouping --- if the collected content exceeds a single LLM call's context budget, sections are partitioned into groups using the same group_by_budget() function as the context knapsack. Each group becomes a separate composition call, producing part N-of-M.
LLM composition --- a dedicated LLM call using complete_no_think() (no thinking mode, maximizing output tokens for composition) composes each group into prose. The system prompt explicitly instructs verbatim preservation of quoted evidence and source URLs. system_instructions (formatting guidance from the user) are injected only at this stage, never during the iteration loop.

Root shortcut. If the root task is already DONE with a result_summary (the LLM composed the answer during iteration), the assembly pipeline is skipped and that summary is returned directly.

Evidence appendix. For each DONE task with evidence, a [Source data] appendix (top 5 evidence excerpts by strength, 400 chars each) is appended to the task's result_summary before the composition LLM call. This ensures the assembly model sees raw data even when per-iteration composition paraphrased it. For leaf tasks without a result_summary (researched but not yet composed), the top 20 evidence items (600 chars each) are formatted directly as the section content.

Drivers¶

The drivers/ package wraps external service communication behind narrow interfaces:

BrowserClient (drivers/browser.py) --- manages CloakBrowser via Chrome DevTools Protocol. Each research session creates an isolated browser context (separate cookies, cache). PDF URLs are detected and fetched via direct HTTP rather than browser rendering, avoiding Chrome's PDF viewer trap. Media content (video, audio) is intercepted via CDP Network.responseReceived events when response body exceeds 50 KB, then extracted with av and transcribed through whisper.cpp.

CoalescingEmbedClient (drivers/embed.py) --- batches concurrent embedding requests with a 50ms debounce window. When multiple coroutines request embeddings simultaneously (common during evidence ingest and tool result processing), the coalescer groups them into a single HTTP batch call. Separate queues handle document-prefix and query-prefix requests, since mixing prefixes in one batch would corrupt results.

STTClient (drivers/stt.py) --- whisper.cpp HTTP client that sends WAV audio and returns transcribed text. Uses translate mode (output always in English). Failures are silently caught and logged rather than propagated: video transcription is an enrichment path, not a critical one. If whisper is slow or unavailable, the research engine continues with the page's text content. Surfacing STT errors to the agent loop would stall research for non-essential content.

Design Decisions¶

Single-turn stateless LLM calls. Lethe does not maintain conversation history with the LLM. Each iteration sends a fresh system prompt plus the current graph context. The graph itself is the memory --- the LLM reads it each turn through DIN-Read serialization. The alternative (multi-turn chat) would require managing growing context windows, determining what to evict, and tracking conversational state --- complexity that the graph already handles. The tradeoff: every call pays the full prompt cost, but each is independently reproducible and debuggable.

DIN protocol for structured output. The DIN protocol constrains LLM output to validated JSON operations. This was driven by a practical constraint: llama.cpp's thinking mode produces extended reasoning before the structured output, and the DIN protocol with Pydantic validation ensures parseable results despite the interleaved reasoning tokens. The alternative --- free-form text with heuristic parsing --- would require fragile extraction logic and produce ambiguous operations.

No separate vector store. Unlike RAG architectures that maintain a vector database alongside a document store, Lethe stores embeddings directly as node properties in Neo4j. The SIMILAR_TO edges serve the function of a nearest-neighbor index but within the graph topology, enabling PPR to traverse both structural and semantic links in a single computation. The tradeoff: Neo4j is not optimized for high-dimensional nearest-neighbor search, so the system relies on batch pairwise comparison with centroid pre-filtering rather than approximate methods like HNSW.

Frozen configuration hierarchy. Every numeric threshold lives in config/config.py as a frozen dataclass field with validation in __post_init__ (Configuration). No module defines its own magic numbers. Runtime overrides use LETHE_* environment variables. This makes the system's behavior fully auditable from a single file and prevents the "magic constant drift" common in research codebases where thresholds accumulate across scattered modules.

Two loops, one algebra. The agent loop and research engine share the same mathematical functions (SharedSVD, convergence_score, goal_alignment, soft_gate) with different configuration parameters. This symmetry reflects that both solve the same fundamental problem --- explore an information space until convergence --- at different scales and with different I/O. The agent loop explores a task graph; the research engine explores a GA-ranked URL queue. Both use \(\psi\) for termination, though at different thresholds (0.82 agent, 0.70 research).

LLM owns recovery and composition. The orchestrator does not implement algorithmic recovery strategies (retry with different parameters, escalate to a supervisor agent, classify failures). Instead, the LLM observes the graph state --- including dead-end siblings, zero-yield hints, and qualitative coverage tags --- and decides its own recovery through DIN operations: creating new subtasks, redirecting effort, or marking approaches as dead ends. This deliberate absence of algorithmic recovery code reflects a design conviction validated during development: heuristic detection by the orchestrator combined with LLM-directed repair outperformed prescriptive recovery pipelines.

CloakBrowser as sole fetcher. All web content passes through a single stealth Chromium instance rather than a tiered HTTP/browser fallback. The alternative (direct HTTP for simple pages, browser only for JavaScript-heavy pages) was rejected because it introduces a classification decision (which pages need JavaScript?) that is both unreliable and produces inconsistent content quality. A single rendering path eliminates this decision and ensures uniform bot-detection evasion.

Deliberate Exclusions¶

Several capabilities were evaluated and deliberately excluded from the initial design, applying a deletion test to each: if removing it concentrates complexity rather than spreading it, it was earning its keep; if not, it was cut.

Inter-session memory consolidation --- merging knowledge across sessions (RAPTOR-style hierarchical summaries, cross-session SIMILAR_TO edges) was deferred. Each session starts from a clean graph. The rationale: consolidation introduces a consistency problem (stale evidence from previous sessions conflicting with fresh research) that outweighs the retrieval benefit for single-query use cases.
Fan-out parallel task execution --- running multiple LLM calls in parallel on different tasks was rejected. llama.cpp processes one prompt at a time, so parallelism would require either multiple model instances (VRAM-prohibitive) or queuing that negates the latency benefit. The sequential iteration loop with inline research parallelism (multiple browser tabs, batched embeddings) provides sufficient throughput.
UCB1 / bandit-based task selection --- multi-armed bandit approaches were evaluated for task selection but removed. The embedding algebra's gap-adaptive warping achieves exploration-exploitation balance without maintaining per-task reward statistics, and the deletion test showed that UCB1 added interface complexity (reward updates, confidence bounds) without improving task selection quality.
Constrained decoding --- grammar-based token masking (XGrammar, Outlines) was considered for enforcing JSON structure during generation. The Pydantic validation + repair loop was preferred because llama.cpp's thinking mode separates reasoning from structured output, making post-hoc validation sufficient and avoiding the grammar compilation overhead.

Lethe draws on several lines of recent research: graph-native agent memory (GAAMA, HippoRAG), submodular context selection (S-RAG, AdaGReS), information foraging in agents (InForage), and budget-aware agent control (BAVT). The distinguishing architectural choice is using embedding algebra as a unified control plane across all subsystems --- not just for retrieval (as in typical RAG) but for convergence detection, task selection, budget management, and research navigation. See the overview for the full algorithm bibliography.