Memory Tooling — Open Decisions & Measurement Log

The living decision log for the memory stack: what is adopted, what is still open, what is rejected — with evidence and measured results per record.

Executive summary

Working with Large Codebases gives the current shape of the memory stack; this paper is the running log of what changes next, what is still an open call, and what we have already ruled out. Every entry carries evidence links and an empty Decision / Measured result slot that gets filled in when the call is made or the benchmark lands. This page is not a snapshot; it is the workspace's memory of how it makes memory decisions.

Benchmark integrity — what we trust

SWE-bench Verified was invalidated in February 2026: an independent audit found that 59.4% of the hardest failing cases were broken tests, not agent errors, which means every "SOTA on SWE-bench Verified" claim through that period is unsafe as a comparator. Our tracking now uses SWE-bench Pro (standardized-scaffold column) as the honest line — the standardized-scaffold column strips harness-specific advantages that made Verified numbers non-transferable in the first place. When a vendor cites Verified without pointing at Pro, that is a signal.

• SWE-bench Pro — standardized-scaffold leaderboard
• SWE-bench Verified — Feb 2026 audit (broken-test invalidation)

Incidents — live failure log

Three runaway ChunkHound DBs inside four days, sharing one upstream root-cause class (DuckDB HNSW churn) plus, in Incident #3, a CLI-path cap-bypass. Recorded so the reasoning behind the mitigations (and the D4 service-selection call) survives.

Incident #1 — 2026-07-09

Summary: 277 GB runaway ChunkHound DB during 46-repo onboarding. Write-heavy backfill × interrupted transactions × uncapped DuckDB HNSW churn.

Root cause: HNSW drop-recreate churn blocks DuckDB ART/row-group cleanup (duckdb#19468); the DB file never reclaims pages until compacted.

Mitigation shipped: Absolute database.path in the workspace .chunkhound.json scaffold, database.max_disk_usage_mb: 20000, fragmentation_threshold_pct: 30, bounded indexing knobs.

Upstream: chunkhound PR #339 (compaction, merged, unreleased post-5.1.0) will address the class; issue #340 (phased indexing + db_stop_pct) is the follow-up. Community tool it-bens/chunkhound-index-compactor exists as an interim.

Incident #2 — 2026-07-11

Summary: Second runaway DB — this time 412 GB inside nexartis-ozzydev, discovered during a parallel-wave run.

Root cause: Combined driver: (i) relative database.path resolved against CWD produced a repo-level DB that inherited none of the workspace cap; (ii) same DuckDB HNSW churn class as Incident #1; (iii) the pre-PR-#249 Voyage 429 retry death-loop amplified writes by looping on rate-limited batches.

Mitigation shipped: Absolute database.path everywhere, workspace-level 20 GB cap made mandatory in the scaffold, max_concurrent_batches lowered to 3 to stay well inside Voyage Tier 1 limits.

Upstream: Both incidents are known failure modes upstream and being actively fixed: PR #339 (compaction, post-5.1.0), issue #340 (phased indexing + db_stop_pct). PR #249 already shipped the Voyage 429 retry fix. See D4 for the mitigate-in-place vs migrate call.

Incident #3 — 2026-07-12

Summary: Third runaway DB — 202 GB accumulated in ~13 minutes during a throttled CLI re-index, with database.max_disk_usage_mb: 20000 configured. The cap was enforced on the daemon write path (stopped a parallel 24.9 GB daemon run cleanly) but NOT on the CLI bulk-indexing path; the CLI run terminated with SUCCESS despite the cap being blown by ~10×.

Root cause: The disk-cap enforcement lives on the daemon’s incremental write loop; the CLI bulk indexer takes a different code path that does not consult the same guard. Combined with the same DuckDB HNSW churn (duckdb#19468) as Incidents #1/#2, the CLI path is a cap bypass.

Mitigation shipped: Reclaimed ~863 GB across all three incidents. Disabled the chunkhound MCP in the workspace kilo.json until a release ships PR #339 (compaction) and the fix for the new upstream issue #365. Interim semantic layer = codesearch + serena; codesearch was healthy through all three incidents.

Upstream: Filed chunkhound/chunkhound#365 (CLI bulk path ignores max_disk_usage_mb; run reports SUCCESS with cap blown). D4 flipped to the service-selection call on the strength of this incident — the cap alone is no longer sufficient for on-by-default operation.

Adopt now — the low-effort queue (A1–A6)

These are obvious wins on published evidence, cheap enough to ship without a formal decision. Each earns its slot with a KPI delta in the benchmark harness.

1. A1 — Repo-map MCP layer (Low)

   Aider-style tree-sitter + PageRank map per repo, ~1k tokens. Drop-in MCP; measurably cuts "let me grep" round-trips and hallucinated imports across every reference implementation we surveyed. Complements ChunkHound (semantic) and Serena (symbol) with a cheap file-level scaffold the planner reads once per repo.

2. A2 — Embedding benchmark harness (100-query gold set from real agent traces) (Low)

   Score voyage-code-3 vs Qodo-Embed-1, jina-code-embeddings-1.5b, and SFR-Embedding-Code on our own queries. CoREB and CoIR both show code-specialised open-source challengers exist for our workload class; benching them beats faith. Also produces the honest baseline the whole decision log measures against.

3. A3 — Reranker A/B in the same harness (Low)

   rerank-2.5 vs Qwen3-Reranker-4B on the code-to-code slice. CoREB is explicit that all rerankers are task-asymmetric; rerank-2.5 is unbenchmarked on cross-repo C2C in our stack. Cheap to run once the A2 harness exists.

4. A4 — Verify ChunkHound AST chunking is structure-preserving on our tree (Trivial)

   ChunkHound advertises cAST-style chunking. Confirm no split-function pathology on our Svelte + TS + Python + Rust + Go mix — a 2–3% retrieval delta hides here per the cAST/AIRCoder findings, and we would rather see it in a fixture than in production.

5. A5 — codesearch git post-checkout hooks on Agent Manager worktree repos (Trivial)

   Already-documented codesearch feature; installs freshness on branch switches so the index does not lag the checked-out ref inside a scratch worktree. Complements the "worktree audits use native tools" rule with a smaller freshness window when the tool IS the right one.

6. A6 — Memory-review curation queue (Augment Memory Review pattern) (Low)

   Borrowed from Augment's Memory Review (2025-09-08): draft memories surface in a turn summary as "N pending" with per-item Approve / Edit / Discard. Our AGENTS.md self-maintenance loop is manual and end-of-session; a pending-items queue in the console is a better mechanism. Pair it with the mechanical trigger "same mistake twice → promote to a context file," which is automatable today because session_reports.wrong_tool_picks_json already exists in D1 — repeat detection can raise a queue item.

Open decisions (D1–D20)

Each decision below carries a problem statement, an options table, the evidence considered, the current recommendation, and a Decision / Measured result slot. That slot is the point of this page — when the decision is made or the benchmark comes back, it gets filled in here, not in a scratch document.

D1 — Cross-repo symbol intelligence

Problem: The one query class our stack cannot answer today: "who uses this SDK export across all 46 repos?" Serena is per-repo; codesearch fuses lexical + dense but has no cross-repo symbol graph; ChunkHound is semantic, not symbolic.

Evidence:

• Sourcegraph — cross-repository code navigation with SCIP
• GitHub — stack-graphs (arXiv 2211.01224)
• RANGER — repo KG + MCTS localisation

Recommendation: Pilot (b) CoreGraph for one week — near-zero cost. Commit to (a) SCIP-in-CI only if the pilot's answers are wrong often enough on our real "who calls this SDK export" queries to justify the CI investment.

Decision / Measured result: — (empty; to be filled when the decision lands or the benchmark result arrives)

D2 — Freshness architecture upgrade

Problem: Agent Manager worktrees are >90% similar to their parent; re-embedding everything on a fresh worktree is wasteful and, at workspace scale, the failure mode that produced the 277 GB runaway-DB incident. Cursor has a published answer (content-hash-keyed cache + Merkle diff + simhash worktree reuse); ChunkHound does not implement it yet.

Evidence:

• Cursor — secure codebase indexing (Merkle + simhash)
• 277 GB incident — Working with Large Codebases
• chunkhound PR #339 — compaction (merged, unreleased as of 2026-07-12; partially resolves D2)

Recommendation: File the upstream request; accept the cap meanwhile. Revisit if worktree spin-up cost becomes a measured bottleneck. Upstream compaction (PR #339) is confirmed merged post-5.1.0 — this decision partially resolves the moment the release ships.

Decision / Measured result: — (empty; to be filled when the decision lands or the benchmark result arrives)

D3 — codesearch multi-signal fusion

Problem: AIRCoder shows that adding import-graph-existence and structural-hierarchy signals to the existing BM25 + dense RRF stack yields +4–6 EM in ablations. codesearch fuses two of three signal classes; the structural signal is missing.

Evidence:

• AIRCoder — multi-signal RRF (ACL'26)
• CodeRAG-Bench (NAACL'25)

Recommendation: File the upstream request; revisit after D1 lands so a symbol graph can feed the fusion its structural signal.

Decision / Measured result: — (empty; to be filled when the decision lands or the benchmark result arrives)

D4 — Semantic layer — service selection

Problem: The prior mitigate-in-place-vs-migrate call reopened once three runaway-DB incidents (see Incident log) exposed a class of failures the disk cap alone could not contain. Cost is not the primary constraint; upstream feature requests are not the fast path (fork-and-improve at workspace scale first, propose a PR only when world-class after auditing related issues); documented-best commercial service is preferred where one fits. The question: which service or fork delivers a world-class semantic layer, measured at workspace scale.

Evidence:

• zilliztech/claude-context — hosted Milvus code-context MCP
• Zilliz Cloud — hosted Milvus
• Sourcebot — self-hosted regex + ask_codebase
• chunkhound PR #339 — compaction (6,819 lines / 53 files, actively merging)
• chunkhound issue #365 — CLI bulk path ignores max_disk_usage_mb
• duckdb issue #19468 — HNSW drop-recreate blocks ART/row-group cleanup

Recommendation: Primary = zilliztech/claude-context MCP + Zilliz Cloud (managed Milvus) + Voyage voyage-code-3 (native EMBEDDING_PROVIDER=VoyageAI; hybrid BM25+dense; AST chunking; Merkle incremental sync). Fallback = self-hosted Sourcebot + @sourcebot/mcp for regex + ask_codebase across the 46-repo workspace. Fork-chunkhound rejected; Greptile / Relace Repos rejected (GitHub-cloud-indexed); Augment Context Engine rerouted to D19 (public API and SDK now exist; real blocker is the string-only supported output shape — benchmark-only first, do not adopt as a router arm without numbers).

Decision / Measured result: Adopted 2026-07-12. Status: pending procurement of the Zilliz Cloud account. Primary = claude-context on Zilliz Cloud + voyage-code-3; fallback = self-hosted Sourcebot. chunkhound MCP is disabled in the workspace kilo.json until the primary lands. Signup and API-key wiring: see the Signup and setup links (D9) section; every key lands in the console Secrets panel per ADR-006. Amendment 2026-07-16 — upstream unblocked; scheduled bake-off. Both D4 pause conditions have shipped upstream: PR #339 (compaction) merged 2026-07-03 and released in v5.2.0 (2026-07-12, auto-compact via atomic swap on fragmentation_threshold_pct); the maintainer confirmed on chunkhound#365 that the CLI-path disk-cap bypass is fixed in the same version (not changelog-explicit — needs empirical verification). Local install 5.1.0 is behind v5.2.1. Action plan: (1) uv tool upgrade to v5.2.1, no MCP re-enable yet; (2) empirically verify #365 with one throttled CLI index run against a scratch DB under the 20 GB cap; (3) add a chunkhound expert to the ozzydev-search router behind enabled:false and bake off vs claude-context+zilliz on fixtures v1.4.0 (nDCG@10, selection F1, p95, cold/warm); (4) decide adopt/supplement/retire WITH NUMBERS. ChunkHound was never benchmarked in the IR harness — D4’s pause was operational-safety, not measured quality. What ChunkHound uniquely offers if it wins: workspace-local semantic (no Zilliz cluster dependency), live-watch incremental indexing (Zilliz arm is operator-run script today), code_research deep loop, git-diff semantic search (v5.2.0). Side-notes: rotate the plaintext Voyage key in .chunkhound.json through the Secrets panel (ADR-006) before re-enable; verify .svelte chunking directly in the bake-off (claude-context has the same gap, upstream #222). Bootstrap autostart stays opt-in; disk-cap + fragmentation settings stay mandatory. Amendment 2026-07-17 (dogfood session 2, homeport-release sprint) — D4 moves from paused to PARTIAL RE-ENABLE (read-only MCP). Upstream shipped both blockers: v5.2.0 (2026-07-12) automatic DuckDB compaction via atomic swap on database.fragmentation_threshold_pct (no indexing downtime) AND a new `chunkhound mcp --read-only` mode that opens the DB read-only and disables the indexer/watcher entirely; v5.2.1 is current. Issue #365 (max_disk_usage_mb not enforced during CLI bulk indexing — DB grew to 202 GB with a 20 GB cap configured, mirroring Incident #3) remains OPEN upstream. Measured on the 20260717-homeport workspace after upgrading 5.1.0 → 5.2.1: incremental reindex 23.55 s / 316 files / 0 errors; DB size 2.3 GB → 201 MB (compaction verified live); semantic search quality verified (top hit yanez-bls.ts at 0.812 for a BLS-verification query). New status: workspace kilo.json entry enabled with `--read-only`; indexing/watch remains explicit operator-run CLI only until #365 ships a fix. Amendment 2026-07-19 (dogfood session 3, 20260719-ocme workspace) — READ-ONLY DOGFOOD PROVEN and the registry default FLIPPED. Fresh-workspace measurement: `chunkhound index` discovered 751 files across 5 git repos; the DuckDB peaked ~657 MB during embedding then auto-compacted to 126 MB (0.6% of the 20 GB cap); the read-only MCP attached on VS Code window reload (query_ready true, watcher uninitialized as expected for --read-only) and a live semantic query returned the correct receipt-signing test (receipts-with-event.test.ts) at score 0.863. Executed the registry default flip: mcp/install.sh now scaffolds the chunkhound kilo.json entry enabled:true with `--read-only`, and bootstrap.sh §5c builds the one-time index by default (quiet, explicit `--max-disk-usage-gb 20`, opt out with --no-chunkhound-index) — superseding the earlier bootstrap-autostart-opt-in note now that compaction is proven live. The write/index path is never auto-run; chunkhound#365 (CLI bulk path ignores max_disk_usage_mb) remains the only gate on lifting the operator-run-CLI-only constraint. Lesson folded into the scripts: never index with --verbose (it emits a per-file/per-chunk DEBUG flood). Historical record above is preserved intact. Amendment 2026-07-27 — D4 option (e) rationale CORRECTED. The original "no public API" rejection is factually obsolete: Augment shipped a Context Engine MCP on 2026-02-06 (GA, local + remote variants) and an Experimental Context Engine SDK (@augmentcode/auggie-sdk / auggie-sdk). The REAL blocker is the supported OUTPUT SHAPE — `search()` returns a formatted string per Augment’s API reference ("Formatted string containing the search results, ready for LLM consumption"; `maxOutputLength` default 20 000 / max 80 000; no documented ranked `chunks[]`; no per-item relevance score; `ctxc search --raw` returns pre-summarization text, still not JSON). An undocumented `agents/codebase-retrieval-raw` endpoint returns `chunks[{text, path, charStart, charEnd, lineStart, lengthInLines, blobName, score, origin}]` structurally near-identical to our `ExpertHit`, but the CLI discards it and it is unsupported and liable to silent change. Cost ≈ $0.03–$0.06 per query (provider list + 40% service fee); Legacy Developer Plan availability is undocumented / unverified — adjacent Auggie features (`--show-cost`, `context stats`, cloud agents) are server-side feature-flagged off for it, so this needs an account probe. Adoption question moved to D19 (benchmark-only first; do not adopt as a router arm without numbers). D4’s own semantic-layer bakeoff (chunkhound vs claude-context+Zilliz, fixtures v1.4.0) still has an empty result slot — the Augment arm will be measured on the same fixtures so all three are comparable.

D5 — MCP transport hardening → world-class MCP controller/manager

Problem: stdio MCP fragility is a confirmed class-wide client bug (claude-code#43177): HTTP/SSE transports get auto-reconnect on the client side; stdio is explicitly excluded, and the one-line client fix is unmerged. In practice a stdio MCP that hiccups during a long wave stays broken until someone notices. codesearch sidesteps this by running as an HTTP daemon behind launchd; serena now does the same. Scope expanded: transport hardening is the foundation, and the deliverable is a world-class MCP controller/manager (supervision, health, restart, config) that becomes the substrate for cloud-deployed ozzydev ("My Cubicle Cube," a packaged dev-team-in-a-cubicle). Its architecture and data model feed the console D1 datastore (see status callout).

Evidence:

• claude-code#43177 — stdio MCP transports excluded from auto-reconnect (class-wide client bug)
• Serena — streamable-http transport docs
• mcp-proxy — stdio↔HTTP shim

Recommendation: Approved with expanded scope. Ship the world-class MCP controller/manager as the foundation for cloud-deployed ozzydev. Its supervision/health/restart/config plane is the substrate every MCP daemon plugs into; codesearch is the reference pattern; serena is done; chunkhound depends on D4.

Decision / Measured result: Approved 2026-07-12 with expanded scope. Serena runs supervised streamable-HTTP on 127.0.0.1:24283 (LaunchAgent com.nexartis.serena-serve); codesearch-serve on 127.0.0.1:39725 is the reference pattern; ChunkHound pending D4. Next deliverable: promote the launchd-behind-HTTP pattern into an explicit MCP controller/manager (supervision, health, restart, config) as the substrate for cloud ozzydev ("My Cubicle Cube"). Codesearch-serve LaunchAgent wedge (listener RSTs ~15s after start while process alive) root-caused 2026-07-16 as fd exhaustion: launchd inherits the macOS 256-fd soft limit and serve needs ~1000 fds for 60 registered repos (LMDB + tantivy FTS + FSEvents per repo; "Too many open files" errno 24 / EMFILE in serve.log is the artifact). Fix: SoftResourceLimits.NumberOfFiles=65536 in the plist (documented launchd key; install.sh scaffolds it and migrates pre-existing plists every run). Verified sustained-healthy under launchd with live MCP handshakes. CORRECTED DIAGNOSIS: the earlier TCC-on-~/Documents hypothesis was a real probe result but not the wedge cause — the RCA is EMFILE; the Full Disk Access grant would not have fixed it. Upstream finding for the controller/manager design: serve should fail LOUD on EMFILE in the accept loop instead of wedging silently.

D6 — voyage-context-4 evaluation

Problem: Voyage shipped voyage-context-4 on 2026-06-29: chunk-context-aware embeddings that ingest surrounding chunks at embedding time ("stop worrying about chunking"). If it holds up on the workspace workload, it could obsolete a chunk of the chunking tuning done for voyage-code-3 + cAST, and possibly reduce the freshness surface area D2 addresses.

Evidence:

• Voyage AI — voyage-context-4 announcement (2026-06-29)
• A2 — embedding benchmark harness (this page)

Recommendation: Approved as option (b) under the newest-first-evidence-checked principle: adopt voyage-context-4 as the default when the semantic layer returns (post D4). Bench remains scheduled as the evidence check; back off only if benchmarks show voyage-context-4 losing to voyage-code-3 on the workload.

Decision / Measured result: Approved 2026-07-12 as (b) — adopt voyage-context-4 as default when the semantic layer returns per D4. Newest-first, evidence-checked: bench is scheduled once the semantic layer is live; adoption backs off only if numbers say so.

D7 — RL-search subagents benchmark — WarpGrep v2 / Relace FAS vs codesearch group

Problem: The one query class our stack still cannot answer (D1: cross-repo symbol intelligence) is exactly the class the new RL-trained search subagents target. Morph WarpGrep v2 tops SWE-Bench Pro and advertises multi-repo + package + log search; Relace Fast Agentic Search claims 4× faster via parallel tool calls and now ships on OpenRouter. Whether either beats our codesearch group="nexartis" fusion on our own workload is unmeasured.

Evidence:

• Morph WarpGrep v2 — SWE-Bench Pro #1, multi-repo + package + log search
• Relace Fast Agentic Search — 4× faster via parallel tool calls, on OpenRouter
• D1 — cross-repo symbol intelligence (this page)

Recommendation: Bench WarpGrep v2 and Relace FAS against codesearch group on the D1 task set. Adopt the winner (or stay put) with numbers, not vibes.

Decision / Measured result: — (empty; to be filled when the decision lands or the benchmark result arrives)

D8 — Adopt the write-strategy rubric into AGENTS editing-routing

Problem: The editing-routing table in the workspace AGENTS.md currently lists tools by category (weave / ast-grep / morph / serena / native) but does not route per-task. The new Write Strategies paper introduces a rubric (a–h) mapping each change class to a primary + fallback tool, backed by SWE-Edit (arXiv 2604.26102) evidence that per-task routing is a Pareto win.

Evidence:

• Write Strategies for Coding Agents (this workspace)
• SWE-Edit (arXiv 2604.26102) — per-task edit-format routing is Pareto
• Aider — edit-format findings

Recommendation: (a) Adopt the rubric as the AGENTS editing-routing table.

Decision / Measured result: Approved and landed 2026-07-12 — the (a)–(h) rubric is the Write-strategy rubric block in the workspace AGENTS.md §1 (see /research/write-strategies). Cross-cutting rules (dry_run >3 files, no tool-mixing, git diff after batches) landed in the same section. Full rationale kept on the write-strategies page.

D9 — Commercial evaluation buy-list — Morph / Relace / Codemod

Problem: The write-strategies rubric leans on fast-apply and structural engines whose numbers are all vendor-reported. Two internal benchmarks (Morph-vs-Relace bakeoff; atomic-tool corruption bench) are cheap but not free; running them requires paid tiers of the vendor products.

Evidence:

• Write Strategies buy-list (this workspace)
• Morph — Fast Apply pricing + throughput claims
• Relace — pricing + Apply 3
• Codemod — BYOC / MCP
• Anthropic — advanced tool use (programmatic tool calling, free)

Recommendation: (a) Approve. Enable Programmatic Tool Calling immediately (free); procure Morph/Relace/Codemod tiers scoped to the two internal benchmarks; adopt the winners on numbers.

Decision / Measured result: Approved 2026-07-12 — proceed on Morph + Relace + Codemod + Anthropic Programmatic Tool Calling. Signup / setup path per vendor is in the "Signup and setup links (D9)" section. Every key lands in the console Secrets panel per ADR-006 — never in tracked files, never in shell history.

D10 — Parallel-waves repo architecture — one-per-harness vs single library with engine adapters

Problem: nexartis-opencode-parallel-waves already ships two harness engines (claude-code, opencode) and needs to grow to at least five (add kilo, aider, augment auggie). It is also becoming a KYM developer-program library — a starter template external developers use to orchestrate their own agent workstreams. The architecture question is how the harness diversity is represented: one repo per harness, one library with engine adapters, or a plugin-package model.

Evidence:

• nexartis-parallel-waves-agent-harness — scripts/engines/base.py (engine-adapter shape)
• Agentic Velocity — harness-level A/B (KYM bandit workstream)

Recommendation: (a) Single library + engine adapters. Use v2 as the base, absorb v1’s claude-code engine, then add aider + auggie + kilo. Retire v1. This is also the shape the KYM bandit workstream (harness-level A/B) needs — one CLI, N engines, receipts and votes as reward signal.

Decision / Measured result: Decided 2026-07-12 — (a) single library + engine adapters. Repo renamed nexartis-opencode-parallel-waves → nexartis-parallel-waves-agent-harness. v1 retires after claude-code parity is confirmed on v2. Two-tier adapter model: native adapters (per-engine) + generic ACP for anything speaking Agent Client Protocol. base.py follows the Terminal-Bench BaseAgent shape; mini-swe-agent is the control arm; the measured-result slot for the bakeoff lives on /research/harness-bandit-experiments.

D11 — KYM vs NANDA boundary — developer loop vs production publication

Problem: Working hypothesis under test: KYM (Know Your Model) is the developer optimization loop where variants are built, run, evaluated, and iterated; NANDA is the production publication layer where winning agents are exposed to the network and discoverable. If the boundary is clean, an end-to-end wizard in ozzydev looks like: select an agent batch → configure variance → publish to KYM → run/analyze → loop → publish winner to our NANDA node. If it is blurred in code, the wizard hits ambiguity and the code architecture needs a refactor.

Evidence:

• KYM (Know Your Model) — Nexartis internal agent registry
• NANDA — networked-agent discovery layer

Recommendation: (a) Test the boundary E2E in ozzydev with the wizard workflow. Verify code architecture keeps the KYM/NANDA split clean; refactor if blurred.

Decision / Measured result: — (empty; to be filled when the decision lands or the benchmark result arrives)

D12 — Letta / MemGPT for canonical-doc memory — REVISIT

Problem: We previously rejected Letta as an agent memory system (rejected list below). The real problem it solves, however, is not agent memory — it is managing a large canonical markdown corpus (DRY, organised, searchable, lifecycle). Letta’s Context Repositories is a git-backed document-management framework that may give us document conventions + management primitives we should adopt for the canonical-doc corpus, independent of the rejected agent-system claim.

Evidence:

• Letta — Context Repositories (git-backed memory)
• basic-memory MCP (AGPL) — query layer over markdown corpora
• Rejected list below — original Letta / MemGPT rejection

Recommendation: Adopt-pattern, not product. Transplant Letta MemFS conventions onto the canonical docs: YAML frontmatter with a MANDATORY description field (agents load-or-skip on it), a system/ folder for always-loaded content, memory-reorganization subagents run via worktrees (maps directly to the assess→refactor→compress loop), and a sleep-time "dream pass" at sprint end. Pilot basic-memory MCP (AGPL) as the query layer over the workspace docs corpus. Skip the full Letta runtime; skip mem0; skip khoj.

Decision / Measured result: Decided 2026-07-12. Adopt-pattern + pilot basic-memory MCP. Conventions to land: YAML frontmatter with mandatory description field; system/ folder for always-loaded content; memory-reorganization subagents via worktrees (assess→refactor→compress); sprint-end "dream pass". Full Letta runtime, mem0, and khoj skipped.

D13 — SCIP-in-CI + CoreGraph rubric — call-routing between exact and syntactic symbol layers

Problem: D1 (cross-repo symbol intelligence) had (a) SCIP-in-CI and (b) CoreGraph as competing options. The current direction: adopt SCIP as a package.json build step (freshness comes free with the existing build-before-commit gate — an instance of the general "index refresh as build step" pattern for any tool that supports it) AND pilot CoreGraph as an MCP. The open question is the call-routing rubric between them: which query class goes to SCIP (exact, type-precise, per-package), which to CoreGraph (syntactic, cross-repo traversal), and how agents pick.

Evidence:

• Sourcegraph — cross-repository code navigation with SCIP
• dora — SCIP → SQLite → MCP
• scip-query (PlunderStruck) — alternate SCIP query layer
• D1 — cross-repo symbol intelligence (this page)
• AGENTS §1 — search routing table

Recommendation: Adopt both. CoreGraph as the workspace-wide daemon (polyglot stack-graphs, zero-setup, live, cross-language config edges) plus dora per-repo (SCIP → SQLite → MCP), fed by scip-typescript as a package.json build step so freshness is free from the build-before-commit gate. Routing rubric: typed exact cross-repo refs / impact → SCIP (dora); live-edit nav + zero-setup + config / YAML cross-language → CoreGraph; single-repo symbol ops → serena; text / regex → codesearch. scip-query (PlunderStruck) is documented as an alternate SCIP query layer.

Decision / Measured result: Decided 2026-07-12. Both engines adopted with the routing rubric above; scip-typescript wired as a package.json build step per the general "index refresh as build step" pattern; scip-query held as an alternate query layer.

D14 — Ozzydev document workbench — canonical-doc lifecycle UX

Problem: Ozzydev’s next phase treats document editing / viewing / exporting as a core function: canonical docs, ADRs, and roadmap artefacts need a first-class read, approve, and export lifecycle inside the console. The research question is which 2026 packages give us the best rich-markdown reading UX AND beautiful PDF export — because the document corpus is a product surface, not a side effect of writing code.

Evidence:

• mdsvex — markdown-in-svelte
• @mistweaverco/mdsvex-shiki — Shiki highlighting for mdsvex
• pagefind — static full-text search
• Typst — modern typesetting
• Pandoc — universal document converter
• PRODUCT_ARCHITECTURE.md — ozzydev current architecture
• Voices Digest 2026-07 — reading-UX and export candidates as they surface

Recommendation: Reading UX = mdsvex + @mistweaverco/mdsvex-shiki + remark-toc + rehype-slug + mermaid + pagefind. PDF export = Pandoc → Typst pipeline (5–30 ms per doc, deterministic, no Chromium; one nexartis-brand.typ template) with Playwright page.pdf() as the exact-render escape hatch. Skip paged.js, WeasyPrint, and LaTeX.

Decision / Measured result: Decided 2026-07-12. Reading UX and PDF pipeline as above; nexartis-brand.typ is the single Typst template; Playwright page.pdf() reserved for the exact-render escape hatch.

D15 — Multi-orchestrator file coordination — worktree overlap detection

Problem: Parallel implementation waves run multiple orchestrators against overlapping repos in git worktrees. Silent cross-worktree edits to the same files produce merge grief that surfaces late (at the wave PR) instead of at dispatch time. The research question is which coordination primitive prevents the collision without introducing OT/CRDT complexity we do not need.

Evidence:

• worktree-mcp — file-overlap detection between worktrees

Recommendation: Adopt worktree-mcp. Add to the parallel-waves dispatcher so overlap is caught at dispatch, not at PR merge. Heavier options (coordinationhub — dev paused; worktree-mcp-manager — 62 tools) documented as rejected. Skip OT / CRDT.

Decision / Measured result: Decided 2026-07-12. Adopt worktree-mcp; land wire-up in the parallel-waves dispatcher; heavier alternatives rejected as noted.

D16 — nexartis-agent-scripts library — a curated skill monorepo

Problem: AGENTS §1 grants agents permission to write and execute scripts, but nothing curates them. Ad-hoc scripts evaporate at session end; the Voyager / CodeAct / Anthropic-PTC literature all point at a curated library as the compounding surface.

Evidence:

• Scripts as a First-Class Tool Category — architecture + literature

Recommendation: Adopt nexartis-agent-scripts as architected in /research/scripts-as-tools. First 5–10 skills mirror the routing rubrics already codified (search-fanout, write-patch-batch, verify-typecheck-then-test).

Decision / Measured result: Decided 2026-07-12. Adopt; architecture per /research/scripts-as-tools; the next revision lands first published skills + publish gate.

D17 — ozzydev-native-write-mcp — experiment program (approved direction)

Problem: The write-strategy rubric (D8) codifies routing between native edit / weave-patch / morph / ast-grep / serena, but there is no MCP that multiplexes writes across parallel worktrees and pairs each with a verify step to run the sprints of pure write experiments we need to tune the rubric.

Evidence:

• A Common Experimental Framework for Agent Tooling
• Write Strategies — the routing rubric (D8)
• ADR-007 — search MCP (canonical)

Recommendation: Approved direction; ship the write executor and pair it with the search executor under the same native-MCP program. The experimental framework (/research/experimental-framework) is the umbrella; core-workflow rule stands: experiments run out of band from user-visible work.

Decision / Measured result: Approved 2026-07-12; SHIPPED. Both native MCPs — ozzydev-write and ozzydev-search — are enabled in the workspace kilo.json and production-dogfooded (search MCP canonical spec: ADR-007). Experiments framed by /research/experimental-framework.

D18 — Workspace portability + workspace-local context (operator direction)

Problem: The 2026-07-16 workspace move (~/Documents → ~/Developer) required repairing ~160 absolute-path registrations across four machine-global surfaces: workspace kilo.json, ~/.codesearch/repos.json, ~/.serena/serena_config.yml, and the console D1 workspaces table. Operator direction: workspaces are periodically abandoned for clean restarts and moved at will, so the dev environment must survive relocation; it would be cleaner if all of a workspace’s context sat inside the workspace itself rather than in machine-global registries that track every workspace ever created.

Evidence:

• codesearch — CODESEARCH_REPOS_CONFIG + CODESEARCH_RELOCATE_MAX_DEPTH (binary env-var surface, verified 2026-07-16)
• D5 — MCP controller/manager (supervision substrate the per-workspace daemons plug into)

Recommendation: Adopt (a) now (shipped), design (b) into the D5 controller/manager: one supervised serve per workspace, workspace-local repos.json via CODESEARCH_REPOS_CONFIG, per-workspace port allocation, and a workspace manifest the console registers/retires atomically. A workspace becomes a self-contained, movable unit of context.

Decision / Measured result: (a) shipped 2026-07-16 (install.sh kilo.json scaffold + fd-limit plist migration; move executed and verified: 592-test validate green, live adapter smokes green, D1 workspace row re-registered, zero stale path refs). (b) approved direction — design pending with D5.

D19 — Augment Context Engine: adopt as a retrieval arm, or benchmark-only?

Problem: Augment's Context Engine is plausibly best-in-class at code retrieval, and Augment's own thesis is that downstream code-writing quality is downstream of retrieval quality. We have never measured it against our local stack (codesearch / serena / chunkhound / claude-context+Zilliz). Operator has approved indexing all workspace repos on Augment's tenant, so the measurement is now unblocked. Supersedes D4 option (e)'s original rejection rationale — public API and SDK now exist; the blocker is the supported output shape, not access.

Evidence:

• Augment — Context Engine MCP (local + remote variants, GA 2026-02-06)
• Augment — @augmentcode/auggie-sdk (Experimental)
• Augment — vendor downstream PR-quality benchmark (300 Elasticsearch PRs × 3 prompts = 900 attempts, five subjective dimensions, vendor-run, unreplicated: Cursor+Opus 4.5 +71%, Claude Code+Opus 4.5 +80%, Cursor+Composer-1 +30%)
• MoE search router — arm design and hazards (do not duplicate here)
• D4 — semantic-layer bakeoff fixtures v1.4.0 (this page; Augment arm to run on the same fixtures)

Recommendation: (a) benchmark-only first, explicitly gated on the retrieval bakeoff landing numbers. Revisit (b) only if Augment measurably beats our fused router by a margin that survives the hazards documented on /research/moe-search-router. No vendor retrieval-only IR numbers exist — their published lift is a downstream PR-quality benchmark (900 attempts, five subjective dimensions, vendor-run, unreplicated). Our own honest baseline to compare against: selection micro-F1 0.824 / macro 0.861, IR nDCG@10 0.821, MRR@10 0.857, Recall@10 0.830 on n=36 scored queries. Local MCP variant gives index parity by construction (real-time working-tree index); remote gives default-branch only. Arm design lives on /research/moe-search-router — cross-reference, do not duplicate. Supersedes D4 option (e)'s rejection rationale; D4's own semantic-layer bakeoff (chunkhound vs claude-context+Zilliz, fixtures v1.4.0) still has an empty result slot — measure the Augment arm on the same fixtures so all three are comparable. AMENDMENT 2026-07-31: gating condition RESOLVED with numbers. The bakeoff ran (Phase-1 pilot s13 + Phase-2 calibration batch calibration-20x3, prereg lock 4f3cce5, sonnet4.5 pinned); the Phase-2 de-saturation gate FAILED (three of four pre-stated criteria fail — all arms 100% resolve, per-task composites byte-identical across arms with 0.0 pp arm spread and sd 0.1646 identical; only criterion 4 at 8/20 divergent-output tasks hit threshold). Operator ruling: accept parity as the headline finding; Phase-2 full-n (~302–430 kcr) CANCELLED. No adoption trigger fired. See result field.

Decision / Measured result: MEASURED 2026-07-31 — parity. Calibration batch calibration-20x3 (20 hard-stratum tasks × 3 arms × 1 seed, sonnet4.5, prereg lock 4f3cce5): 60 runs, 13,675 cr, zero crashes. Retrieval stack does not change resolve/composite under sonnet4.5 on this deck at pilot+calibration scale (~56 kcr total spend); Phase-2 full-n cancelled. Exploratory (non-registered) cost signal: retrieval arms ~27% cheaper than arm-a (a=277 / b=202 / c=205 cr/run mean). Decision outcome: neither (a) adopt nor (b) adopt-as-router-arm fires — Augment Context Engine is NOT adopted as a retrieval arm on this deck under this model. Option (c) inverted integration remains available; option (d) reject-on-architecture is unchanged by parity. Evidence: nexartis-auggie-harness/probes/bakeoff/analysis/calibration-20x3/CALIBRATION-RESULT.md @ commit bb8b333 (+ ruling 8cc56b7).

D20 — Index staleness semantics: HEAD-pointer vs content-aware?

Problem: Two+ workspaces on one machine hold the SAME feature branch concurrently (ADR-014 era). Every commit from the peer workspace advanced HEAD and fired `expert_index_stale` in this workspace — even when the delta touched zero indexed content (measured: a one-line `.codesearchignore` commit false-dirtied a whole repo mid-session). On a shared branch the HEAD-pointer check forces constant reindexing of indexes whose content is still correct. Companion finding from the same probes: the hint-store GET cost 1.3–5.4 s per search (remote D1 round trips), often exceeding expert fan-out.

Evidence:

• ADR-014 — multi-workspace DX: same-branch parallelism, content-aware index staleness, repo inventory
• Latency evidence: router probe timings (hint_fetch_ms 1.3–5.4 s) — this page, telemetry substrate

Recommendation: (a) as ruled by the operator 2026-08-01 (option-picker Q1). Also shipped in the same ruling: hint-store in-process TTL memo (`hintCacheTtlMs`, default 300 s, 0 disables; errors never cached; config_version in the key self-invalidates on bump) and the repo-inventory schema (migration 0006: `repos` + `workspace_repos` + `GET /api/repos/inventory/cross` — one-PR-per-repo visibility across workspaces).

Decision / Measured result: DECIDED + SHIPPED 2026-08-01 — content-aware downgrade live in the codesearch expert (23 new tests; live smoke: aged sidecar → clean 31-item search with zero expert errors → sidecar auto-restamped to live HEAD at call time); hint memo live (10 new tests, hit <50 ms vs miss >80 ms at the seam); 0006 applied to shared D1 and cross endpoint verified returning TWO workspaces' checkouts of nexartis-ozzydev on the same branch. `expert_index_stale` now means real content staleness. Root validate EXIT 0 (console 417, search-mcp 349+2skip).

Signup and setup links (D9)

Vendor-by-vendor onboarding paths for the D9-approved buy-list. Reminder: every key lands in the console Secrets panel per ADR-006 — never in tracked files, never in shell history.

• Morph — Pricing + plan selection at morphllm.com/pricing. Team creation + API key at the Morph dashboard. Enterprise contact via the calendar link on the site. WarpGrep Pro is under the same dashboard.
  • morphllm.com/pricing
  • Morph dashboard
• Relace — Sign up at relace.ai (Get Started → app.relace.ai), create a team + API key. Also available via OpenRouter as relace/relace-apply-3 for OpenAI-compatible clients.
  • relace.ai
  • app.relace.ai
  • OpenRouter relace/relace-apply-3
• Codemod — BYOC / enterprise onboarding via contact / demo form at codemod.com. OSS MCP is free per docs.codemod.com/model-context-protocol.
  • codemod.com
  • Codemod MCP docs (OSS, free)
• Anthropic Programmatic Tool Calling — Free. Enable in the harness config wherever the harness supports it. No signup; no key. Reference the advanced-tool-use engineering post for the enablement flag.
  • Anthropic — advanced tool use
• Zilliz Cloud (claude-context primary — D4) — Create an organization at cloud.zilliz.com, provision a serverless cluster in the closest region, mint an API key, and paste it into the console Secrets panel (/system → API Keys). The key is verified live and applied into the workspace claude-context config; the vector store lives in Zilliz, not on the developer machine.
  • cloud.zilliz.com
  • Zilliz Cloud — get started
• zilliztech/claude-context (semantic-layer MCP — D4) — Install the MCP server via `npm i -g @zilliz/claude-context-mcp` (or clone zilliztech/claude-context and follow the repo README for the source-install path). The workspace kilo.json entry points at the local MCP; the Zilliz API key wires in from the Secrets panel — never in tracked files.
  • @zilliz/claude-context-mcp on npm
  • zilliztech/claude-context repo (README + install)
• Sourcebot (semantic-layer fallback — D4) — Follow docs.sourcebot.dev to deploy Sourcebot locally (docker compose is the shipped path). Point it at the workspace repos and register the MCP endpoint in kilo.json. Sourcebot is the durable fallback when Zilliz is offline or credentials are unavailable.
  • docs.sourcebot.dev
  • Sourcebot — docker deploy guide

Rejected — with evidence

These options came up in the survey and were ruled out. The point of naming them here is so the reasoning survives the moment: when a new voice re-proposes any of them, this section is the answer.

• Full GraphRAG pipeline (offline KG build + graph-only retrieval) — RAGSearch (arXiv 2604.09666) shows agentic search over strong hybrid retrieval closes most of the gap that motivates a full GraphRAG build. Offline graph cost fits stable corpora; our PR-wave velocity produces continuous churn. High cost, low delta on our workload class.
• Unified 46-repo embedding space for codesearch — BM25 statistics are per-corpus by definition; merging all repos into one lexical index blurs the signal. No published evidence that a unified space beats per-repo + RRF on cross-repo retrieval; SembleX validates the per-repo + fusion architecture independently.
• Pre-emptive voyage-code-3 switch — voyage-4-large is Voyage's general flagship, but they ship no voyage-code-4. No head-to-head shows a dominator on our workload class. Bench first (A2); do not switch on brand refresh.
• Adopting Letta / MemGPT as a memory system — Markdown-memories-in-git — the pattern Serena, Claude Code, and Letta all converged on, and what we already run — is the only approach with durability evidence for engineering work. Sleep-time consolidation has no external benchmark. Borrow the defrag idea when we need it; do not port the framework.

How this page works

Living decision log. When an adopt-now item ships, it moves out of A1–A6 and into a measurement row with the KPI delta. When an open decision lands, the recommendation is archived and the result slot records the actual call plus the numbers that justified it. When new benchmarks or SOTA papers arrive, they land in the sources list and any decision they affect is reopened in place. The version at the top bumps on every material change.