A Code-Graph Expert for the MoE Search Router

Every fixture we run says the same thing: our stack answers "what is this" and "who calls this" well and answers "how does this reach that, and what breaks if I change it" badly. A code-graph expert — an LLM-free tree-sitter build of a typed, persistent code graph with EXTRACTED / INFERRED / AMBIGUOUS edge provenance, path queries, blast-radius traversal, god-node detection, and ADR-as-node — is the shape that plugs into the ADR-007 seam without changing it. Research-only: a measured evaluation, not an adoption.

The capability gap — what our stack cannot answer today

Every fixture we run confirms it: serena and codesearch answer "what is this" and "who calls this" cleanly; grep + git log -S answer "where does the string appear" and "when did it land". Nothing in the pool answers path, blast-radius, or architecture-shape questions natively — we re-derive them by hand every time.

• Path queries — "How does the harness router reach the verify ladder?"
  Status quo: No status-quo tool answers this without a human hand-chaining imports across two MCPs. codesearch/serena/grep return the endpoints; the connection between them is on the reader.
• Blast-radius traversal — "What would break if I change the ExpertId union / receipt schema / D1 harness_runs row?"
  Status quo: serena find_referencing_symbols gives one-hop references per file — the answer we typically want. For the deeper "who then propagates from those" we chain calls manually. Adequate on small blast fronts, tedious on large ones.
• Architecture maps (god nodes / communities) — "What are the biggest architectural hubs in ozzydev?"
  Status quo: No tool. We infer it from grep counts (getDb, resolveRuntime, Db, apiErrorResponse — the ones on-call people already know) but the workspace has never had a first-class ranker.
• ADR-as-linked-entity — "Which code cites ADR-009?"
  Status quo: grep finds it in 77 places with full context — the same answer graphify gives with LESS recall (19 nodes). ADR-as-entity is elegant on paper but grep already wins.
• Provenance-aware review surfaces — "Show me the edges the extractor is uncertain about (AMBIGUOUS / INFERRED)."
  Status quo: No tool. This is the flip side of a graph: the graph can DECLARE its own confidence — grep cannot.

Source-level verification of Graphify's claims

Every load-bearing claim was checked against /site-packages/graphify/ — the actual installed code — not the README. Vendor summaries do not become our findings.

• LLM-free for code — verified
  graphify/cli.py emits "Re-extracting code files … (no LLM needed)" on the update path; the extract command with --code-only skips every non-code file with no LLM extraction. The label step (community NAMING) is the only LLM call in the code path and is skippable with --no-cluster / --no-label. graphify/extract.py has a language dispatch table that runs pure tree-sitter walks for every code extension.
• EXTRACTED / INFERRED / AMBIGUOUS provenance tags — verified, with a concrete scoring model
  graphify/extractors/engine.py and resolution.py set "confidence": "EXTRACTED" on syntactic edges (real call/import/method), "confidence": "INFERRED" on derived edges (cross-inheritance uses, indirect calls where the callee name is unresolved), and analyze.py + export.py document AMBIGUOUS on truly unresolved references. export.py maps them to numeric scores {EXTRACTED: 1.0, INFERRED: 0.5, AMBIGUOUS: 0.2}. analyze.py sorts review priority AMBIGUOUS > INFERRED > EXTRACTED — highest uncertainty first, which is the correct discipline.
• Tree-sitter across ~40 languages — verified with a caveat
  28 named language extractors in graphify/extractors/, plus 30 tree-sitter grammars installed as dependencies (typescript, python, rust, javascript, java, c, cpp, ruby, kotlin, swift, php, go, scala, lua, bash, powershell, elixir, julia, groovy, c-sharp, verilog, fortran, zig, objc, json). Svelte/Vue/Astro are handled by a hybrid extract_svelte / extract_vue / extract_astro path in extract.py that parses the <script> block via JS AST and rescues template imports with regex — a pragmatic-but-fragile shim, not a first-class Svelte grammar. .sql needs the optional tree_sitter_sql extra (missing in the default install).
• Persistent state = a single graph.json + a mtime/ast-hash manifest; no server, no key required for the code path
  graphify-out/graph.json holds nodes + edges + hyperedges as one JSON blob (2.9 MB for ozzydev, 397 files, 2745 nodes, 6133 edges). graphify-out/manifest.json is a per-file {mtime, ast_hash} cache that supports incremental re-extraction. graphify-out/.graphify_root is a bare pointer to the ABSOLUTE workspace path — the incremental cache and the god-node/path/query commands read the graph from a workspace-anchored directory. No daemon, no port, no API key required unless you opt into --backend for the doc/media semantic pass or the community-naming LLM step.
• Ships an MCP server (query_graph / shortest_path / get_pr_impact / triage_prs) — verified as CLI-callable; not evaluated in this pass
  graphify/serve.py + graphify/mcp_ingest.py + graphify/prs.py exist and the CLI has a serve command; we did not spawn it in this evaluation because the CLI already exposes the same primitives we needed (explain / path / affected / god-nodes / query / diagnose multigraph) and stdio MCP surfaces would only add spawn discipline without changing the fit-to-seam analysis. Adopting graphify as our seam expert would use its Python primitives via a thin adapter, not its stdio MCP surface (see §Integration cost).
• Self-reported benchmarks (LOCOMO recall@10 0.497) — NOT verified
  We did not re-run their benchmark on their corpus; we ran ours on ours. Every load-bearing number in this page is from graphify-eval/fixtures/*, our own artifacts under /var/folders/lc/…/graphify-eval/. The upstream LOCOMO number is noted as their claim, not our finding.

Evaluation environment

• Package — graphifyy 0.9.29 (PyPI, installed 2026-07-29 via uv 0.11.28). CLI entry point: graphify. Import path: python -m graphify (the Python module is named graphify, PyPI name is graphifyy).
• Python — CPython 3.12.13 (uv-managed venv, isolated in the scratch dir).
• Scratch directory — /var/folders/lc/…/T/kilo/graphify-eval/ — pre-approved external directory; nothing from this evaluation lives inside any tracked repo.
• Repos evaluated — nexartis-ozzydev @ feat/harness-router (397 code files → 2745 nodes / 6133 edges, extract wall time 4.10s cold) and showrunner-link @ feat/… (506 code files → 2625 nodes / 4804 edges, 3.01s cold). Both extracted with --code-only --no-cluster to keep the run LLM-free.
• Read-only against source — graphify extract --out <scratch> writes graph.json + manifest.json + a .graphify_root pointer under the scratch dir; find nexartis-ozzydev -name "graphify*" -o -name ".graphify*" returns nothing after the run. It does NOT insist on writing in-repo when --out is set. Its opt-in "install" subcommands (kilo install / claude install / codex install / …) DO write into the repo — we did not invoke any of them.
• Persisted artifacts — graphify-eval/out/<repo>/graphify-out/graph.json (raw graph), manifest.json (incremental cache), .graphify_root; graphify-eval/fixtures/graphify/q1-def.txt … q10-diag.txt (raw graphify CLI outputs); graphify-eval/fixtures/status-quo/summary.md (status-quo baseline outcomes); graphify-eval/logs/*.log (extract logs). Every load-bearing number in this page traces to one of these files.

Fixture table (n=10)

Ten questions drawn from this codebase, five in categories our stack already covers well (Q1–Q4, Q8) and five in categories our stack does not (Q5–Q7, Q9–Q10). Ground truth was determined by reading the code, not by trusting either tool. Latencies are wall-clock cold-run on this laptop — a first approximation, not a controlled benchmark.

1. Q1 — Where is SearchRouter defined? (symbol · definition)
   graphify: explain "SearchRouter" — Node router.ts L140, degree 12, 12 typed connections listed with per-edge provenance. 309 ms.
   status quo: serena find_symbol("SearchRouter") — router.ts:139-430 with kind=Class. ~150 ms.
   Winner: status-quo — serena returns the exact symbol range; graphify returns the node with a line but not the range. serena is the right tool for symbol definitions.
2. Q2 — Who imports ExpertId? (symbol · usages)
   graphify: affected "ExpertId" --depth 2 — 39 references across 7 files, per-edge relation tag (imports / imports_from / references). 106 ms.
   status quo: serena find_referencing_symbols("ExpertId") — comprehensive with line-context snippets. ~800 ms.
   Winner: status-quo — Both find the same reference set; serena adds inline code snippets. Ties on recall, serena wins on legibility.
3. Q3 — How does expert fan-out handle timeouts? (conceptual · semantic)
   graphify: query "how does expert fan-out handle timeouts" --budget 1000 — 255 nodes truncated to 30 by budget; the answer (per_expert_ms from config, wrapped in withDeadline() per expert) is not in the returned nodes at all. 196 ms.
   status quo: grep for deadlineMs / perExpertBudgetMs / per_expert_ms — 53 exact matches across every expert file, config.ts, and router.ts. The answer is legible from the grep alone. ~500 ms.
   Winner: status-quo — BFS from an NL question over a code graph does not surface a conceptual answer — the graph has no representation of "timeouts as a concept", only the mechanical edges. codesearch/ozzydev-search would answer this more cleanly with a real semantic expert; grep is enough here.
4. Q4 — Where is emitSearchReceipt used? (identifier · exact)
   graphify: query "emitSearchReceipt" — BFS returns nodes around the identifier, some noise. ~174 ms.
   status quo: grep "emitSearchReceipt" — 13 exact matches with line numbers. ~200 ms.
   Winner: status-quo — Identifier lookup is what codesearch (regex) and grep do best.
5. Q5 — How does the harness router reach the verify ladder? (path query) (path · connection)
   graphify: path "HarnessRouter" "LadderResult" — the reported "shortest path (4 hops)" routes through ref_node_crypto (a stdlib import shared by many files). Structurally correct as BFS, semantically meaningless. 182 ms.
   status quo: No status-quo tool answers this cleanly — a human chains index.ts → router.ts → verify.ts imports.
   Winner: neither — a wash — This is the crown-jewel promise of a code graph, and it is where the fixture stings the hardest: graphify’s BFS "shortest path" is hijacked by high-degree nodes (node:crypto, Db, resolveRuntime — the very "god nodes" the tool itself surfaces). Path queries in a codebase need a weight function that penalises structural super-connectors; graphify does not carry one in the OSS distribution. Fixable — likely in a fork — but not in the current tool.
6. Q6 — Blast-radius: what breaks if I change ExpertId? (transitive) (path · blast-radius)
   graphify: affected "ExpertId" --depth 3 — depth-limited reverse traversal, ~40 unique symbols across imports / references / imports_from / method edges. 108 ms.
   status quo: serena find_referencing_symbols("ExpertId") — one-hop reference set, ~30 references. Chaining to depth 3 by hand is tedious.
   Winner: graphify — modest — The graph answer is the same universe of files at depth 1 but graphify natively goes to depth 3 without prompting. The one place where the graph gives a bounded, useful answer today.
7. Q7 — What are the biggest architectural hubs (god nodes) in ozzydev? (architecture · centrality)
   graphify: god-nodes --top 15 — Db 63 · resolveRuntime 59 · apiErrorResponse 57 · renderConsoleError 55 · getDb 52 · requireLocalRuntime 51 · ConsoleError 42 · newId 33 · runBatch 33 · getThisMachine 33 · … 125 ms.
   status quo: No status-quo tool answers this. Best available: grep counts, informally.
   Winner: graphify — unique — This is the one answer where the graph produces something no other tool in our stack does, and the answers pattern-match to what an on-call engineer would name. Same-file counterparts exist in the serve.py MCP surface (get_pr_impact / triage_prs) but were not needed here.
8. Q8 — Which code cites ADR-009? (ADR-as-entity)
   graphify: query "ADR-009" — 19 nodes touching an ADR-0009 entity node backed by console/src/lib/server/db/schema.ts. 192 ms.
   status quo: grep "ADR-009" — 77 matches across code, docs, tests, ADR files, roadmap. Full context, line numbers. ~200 ms.
   Winner: status-quo — grep catches every mention; the graph catches only what it lifted to a first-class ADR entity. Interesting-in-principle idea, worse-in-practice on the actual question.
9. Q9 — What implements the Expert interface? (symbol · implementations)
   graphify: query "Expert interface implementations codesearch serena semantic context7" --budget 1200 — 250 BFS nodes, truncated to 33. No dedicated implements-filter primitive in the CLI. 212 ms.
   status quo: grep "implements Expert" — 5 clean hits: CodesearchExpert, SerenaExpert, ZillizSemanticExpert, Context7Expert, GitLogSExpert. ~200 ms.
   Winner: status-quo — The graph HAS the implements edge (9 of them in the ozzydev graph, per relation histogram) but the query CLI exposes it via NL-BFS with a token budget — a graph-primitive that ought to be O(1) is turned into a paginated traversal by the query surface. Fixable at the adapter layer if we ever ingested.
10. Q10 — Diagnose multigraph (unresolved/collapsed edges — provenance surface) (graph-native)
    graphify: diagnose multigraph --json — one JSON: 2745 nodes / 6133 raw edges / 741 dangling / 29 same-endpoint collapses / 0 self-loops. Provenance histogram from the raw graph: EXTRACTED 6106 / INFERRED 27 / AMBIGUOUS 0. 237 ms.
    status quo: No status-quo equivalent.
    Winner: graphify — unique — Graph-native diagnostic. This is where the confidence tags earn their keep — the tool can report its own uncertainty, which is what makes review over a code graph feasible at all. On our extraction the AMBIGUOUS count was zero and INFERRED was only 27, which is either a very clean corpus or a stingy classifier — we would need to compare across repos to know.

Fit against the ADR-007 expert seam

The seam in mcp/servers/ozzydev-native-search-mcp/src/experts/types.ts is small: an Expert with an id in a string-union and one call(req) method returning ExpertHit[]. Every existing implementation (CodesearchExpert, SerenaExpert, ZillizSemanticExpert, Context7Expert, GitLogSExpert) is a class with that shape and its vendor-specific state in config. Fitting a code-graph expert means the same shape and the same discipline: the ID in the type union stays generic ("graph"), the file naming the vendor lives in experts/, the endpoint and any binary path go in config.json. Zero seam changes.

• Type union in experts/types.ts — Add `"graph"` to Expert.id — a GENERIC seam name, per workspace §2. The file implementing it may be graph-graphify.ts (vendor identity is data, not a type). The existing zilliz-semantic.ts precedent applies: file names the vendor, expert.id is generic ("semantic"), config carries the endpoint.
• ExpertRequest / ExpertResult shape — The shape (question, scope, deadlineMs, subQuery → { hits: ExpertHit[] }) is enough. graphify’s node/edge output would map onto ExpertHit with path, line, and a native_score derived from confidence_score (EXTRACTED 1.0 / INFERRED 0.5 / AMBIGUOUS 0.2). Zero shape changes.
• Classifier / gate — Add a new prior column for the "graph" expert per question class. Path-query and blast-radius do not currently have their own classes — we would extend classifier.ts with a new class "path-query" (or reuse cross-repo-impact) and give the graph expert prior weight in it, with a soft floor across typed-symbol / cross-repo-impact so it fires as a corroborator.
• Fusion — RRF already handles heterogeneous ranked lists. A graph "path" or "affected" answer maps to a small ranked list of files with per-file scores; RRF fuses that alongside serena and codesearch results normally.
• Adapter runtime — A Python CLI shelled out from a TS adapter is the wrong shape long-term (spawn cost, JSON parsing). graphify’s graph.json is a pure data artifact — the correct adapter is a TS reader that loads graph.json once at expert boot and answers all four primitives (explain / path / affected / god-nodes) in-process. That is a ~500 LOC TS adapter, not a fork; the CLI stays only as an out-of-band indexer.
• Persistence + freshness — graph.json is machine-local, ~3 MB per repo, one file per repo, gitignored (an entry we would add). Freshness is a mtime/ast-hash manifest; the same LaunchAgent that runs codesearch could run `graphify update` on the same watch triggers. This is the SAME machine-local, path-keyed pattern as our semantic Zilliz collections — and inherits the SAME orphan hazard on a workspace-directory move: .graphify_root is an absolute-path pointer.
• Receipts — ADR-009 does not require expert-side changes; the router-level receipt already carries per-expert latency, error class, and result count. A "graph" expert would appear in expert_registry alongside codesearch/serena/semantic/context7 with no receipt-schema change.

Integration cost — the honest bill

• ~500 LOC TS graph.json reader adapter (loads once, answers explain/path/affected/god-nodes in-process). We already reject spawning per-request Python.
• One new question class OR reuse of cross-repo-impact + new gate row for "graph". Small classifier delta.
• A LaunchAgent + .codesearchignore-style .graphifyignore per repo for the incremental cache (mirrors existing codesearch/chunkhound plumbing).
• One-shot ~4s cold extract per repo (measured: 397 files ozzydev, 506 files showrunner-link — sub-4s each). Incremental updates via mtime/ast-hash on watch events are near-free.
• A path-query weight function IN THE ADAPTER to penalise structural super-connectors (node:crypto, Db) — the Q5 finding. Without this the graph’s crown-jewel primitive is not usable.
• One `.graphify_root` gitignore entry + a workspace-move handler that rebuilds the incremental manifest, mirroring the Zilliz orphan-detection pattern in console/lib/memory/.

Adoption hazards

• Path-keyed orphan hazard. .graphify_root stores an absolute workspace path. A workspace-directory move breaks the incremental cache and any consumer reading the graph via a stale ref — the exact orphan pattern our semantic Zilliz collections have. Mitigation: reuse the console /memory Repos-tab orphan detection surface.
• Path-query god-node hijack. BFS shortest-path over an unweighted code graph routes through node:crypto, Db, resolveRuntime() — structurally correct, semantically meaningless. Fix requires an edge-weight scheme (PageRank damping, degree penalty) at query time.
• NL-BFS over a graph is not semantic search. Q3 showed that a natural-language question fanned out as BFS start-nodes returns a truncated node dump, not an answer. The graph expert should be gated on class = path-query / cross-repo-impact / architecture-shape, not on generic conceptual queries.
• Svelte support is a hybrid shim. No tree-sitter-svelte grammar; the tool parses <script> via JS AST and rescues template imports with regex. Works for our SvelteKit repos; a Svelte-heavy refactor might trip edge cases.
• In-repo write surfaces we would never invoke. The tool ships opt-in "install" subcommands (kilo install, claude install, codex install, cursor install, …) that write CLAUDE.md / AGENTS.md / .kilo/ plugins into the target repo. None run without an explicit subcommand — we simply never call them.

Decision

Four candidate outcomes weighed against the AGENTS.md conventions (adopt proven libraries where the decision is clear · no defense-in-depth · no hidden behavior · newest-first, evidence-checked · no vendor name in a type / registry key / enum).

1. Adopt now — REJECT. Six of ten fixtures are won cleanly by our existing tools; two graphify-unique wins (god-nodes + multigraph diagnostic) do not on their own justify a new expert with its own indexer, per-repo state file, and gitignore entry. The one modest win (Q6 depth-3 blast-radius) is a serena limitation, not a graphify novelty. Adopting today would violate the "adopt proven libraries where the decision is CLEAR" convention — the decision is not clear.
2. Adopt behind a flagged research seam — REJECT. A flagged seam that is not measured is a defense-in-depth pattern; a flagged seam that IS measured wants a bakeoff fixture at repo scale (n=50–100 real fixtures) before it makes sense to spend implementation days on the adapter. Building a hidden path first is the wrong order.
3. Re-evaluate at a stated trigger — RECOMMENDED. Two triggers, either sufficient: (a) graphify or a peer ships a weight-aware path primitive that survives Q5 without hopping through node:crypto — the Q5 result is the single blocking finding; (b) we accumulate a bakeoff fixture of ≥50 real path-query / blast-radius / architecture-hub questions from live sessions (the deliberative-search v0.2 fixture-harvest pattern) and the graph-answers-beat-status-quo rate on that fixture crosses 40%. Meanwhile the code-graph slot in the ADR-007 §Expert pool table stays reserved with "graph (unimplemented — see /research/code-graph-expert)" so newcomers see the design intent without seeing the vendor.
4. Reject outright — PARTIALLY YES for graphify-as-adopted-expert on THIS measurement. NOT for the CATEGORY: the fit analysis (§seam) shows a code-graph expert would plug into ADR-007 in a few hundred LOC without changing the seam — that is a design result independent of the specific tool. Rejecting the category would forfeit a real gap (path queries + god-nodes).

Strongest counter-argument to the recommendation

The recommendation could be wrong for four reasons, in decreasing order of force. Named honestly so a future re-open can weigh them.

1. The strongest counter is that fixture design biases the result: we picked our own fixtures against our own status-quo tools that we know how to use. A code graph is a different SHAPE of answer — traversal + centrality + provenance — that our fixtures do not fully surface. The tenth diagnostic-multigraph fixture is the tell: the graph can report its own uncertainty; grep cannot. A fixture that emphasises unknown-unknowns (review surfaces, "what am I missing", drift audits) plausibly tilts to the graph.
2. Second, we tested the current OSS distribution’s CLI. The upstream MCP surface (query_graph / shortest_path / get_pr_impact / triage_prs) plus their PR-triage dashboard is a product-shape we didn’t re-implement — it may answer "which PRs conflict on god-node overlap" better than our existing worktree overlap tool, and that is a different value proposition (PR triage, not agent search).
3. Third, the path-hijack finding (Q5 via node:crypto) is a KNOWN class of failure with a KNOWN fix (edge-weight penalties on structural super-connectors, à la PageRank damping). A 200-LOC adapter change fixes it. Rejecting adoption on a fixable finding conflates "shipped fix cost" with "conceptual defect".
4. Fourth, the newest-first convention says default to the newest tool for baselines. graphify is 0.9.29 in July 2026 — barely six months old. Re-measuring at 1.x with a fresh fixture is the honest position; the current recommendation embeds that.

Open questions

• How does graphify’s path-query fare with a super-connector penalty in place? — The Q5 hijack via node:crypto is the single blocking finding. A fork or a downstream adapter that damps high-degree nodes on path traversal would answer this — worth ~1 day of scratch work if the trigger fires.
• How does the graph corpus overlap with our existing typed-graph plans (SCIP / dora / CoreGraph in the ADR-007 §Expert pool)? — ADR-007 already reserves a scip-dora slot ("typed cross-file symbol edges") and a coregraph slot ("polyglot graph — cross-language symbol + import edges"). A code-graph expert overlaps both — is it the SAME slot, one of them, or a third? Depends on typed-vs-heuristic edge quality on the same fixtures.
• Does the PR-triage surface (get_pr_impact / triage_prs / community-overlap ranking) beat our current worktree overlap tool? — Different customer (orchestrator, not subagent search). Not in scope for the ADR-007 seam; worth its own evaluation if we grow PR-review orchestration into a first-class Ozzy Dev surface.
• What is the AMBIGUOUS-edge rate on a messier repo? — ozzydev is unusually clean (AMBIGUOUS=0, INFERRED=27 out of 6133 edges). A large TS+Svelte app with dynamic imports and framework magic will produce a very different confidence distribution — and the provenance-tag value proposition is proportional to that distribution.
• If we ever ingest, which questions merit their OWN classifier class? — Today ADR-007 has typed-symbol / cross-repo-impact / conceptual / identifier / docs / history / unknown. path-query and blast-radius do not have their own classes. Adding them is the right modelling but changes the classifier-fixture surface — v1.4.0 measurements would need to be re-run.

Positioning

1. Vendor-neutral. The category is "code-graph expert"; the vendor is a measured instance. Slug and page title carry the category; graphify is the arm label inside.
2. ADR-007 stays the seam. No new type, no new registry key. If we ever ingest, the Expert.id union gains the string "graph" — never a vendor name.
3. No defense-in-depth. Rejecting adoption today is a LOUD reject with a stated re-eval trigger — not a silent "maybe later".
4. Newest-first, evidence-checked. The current version measured cleanly on ozzydev’s clean corpus; the reject-with-trigger is exactly the evidence-checked backoff the convention asks for.
5. Adopt proven libraries where the decision is CLEAR. Here it is not clear. The convention itself tells us to wait.

What we could not measure

• Cross-corpus AMBIGUOUS-edge rate — ozzydev is unusually clean (AMBIGUOUS=0). One clean repo is not evidence about the provenance-tag value across the fleet.
• The upstream MCP surface (query_graph, shortest_path, get_pr_impact, triage_prs) — CLI parity was sufficient for the seam evaluation; MCP-shape ergonomics would matter only if we ingested.
• The PR-triage dashboard and community-labelling workflow (LLM step) — orthogonal to the search-expert question; would need its own evaluation as an orchestration surface.
• A controlled latency benchmark — reported wall-clocks are one cold run per fixture on a warm process, not a distribution.
• Router-scoped fixtures where ozzydev-search is the baseline rather than codesearch + serena directly — the router had a schema mismatch in this session (query vs question) that another workstream is repairing. All status-quo baselines were run via serena + native grep, and named as such in the fixture table.