graphify: 100% edge accuracy, 73% query accuracy, 1.7x more tokens than grep

graphify turns any codebase into a queryable knowledge graph using tree-sitter AST parsing, with no embeddings, no LLM and no vector store. Its README claims you can "query instead of grepping". We ran it on our own codebase (bestofgit: 406 TypeScript/Python/SQL files) and measured two things: whether the graph it builds is correct, and whether querying it beats plain grep.

1.Setup

Installed graphifyy (the PyPI package) in a fresh Python 3.11 venv. Ran graphify extract . --code-only --no-cluster to build the graph from source files only. Result: 2,302 nodes and 6,575 edges, built in ~30 seconds on MacBook Pro M1, 8 GiB, macOS 26.3.

2.Edge accuracy: 20/20 (100%)

We sampled 20 edges uniformly at random from the 6,575-edge graph and checked each one against the actual source code. Every edge was correct: imports resolve to real import statements, calls resolve to real call sites, and contains relationships point to real definitions. The graph graphify builds is structurally accurate.

A sample of 20 edges out of 6,575 that all check out is not proof that every edge is correct. It is evidence that the parser is accurate on what we checked, which is all a cheap sample can buy.

3.Query accuracy: graphify 11/15 (73%) vs grep 13/15 (87%)

We prepared 15 questions about the bestofgit codebase with known gold answers, split into three types. Each question was answered twice: once via graphify query (BFS, a breadth-first walk of the graph), and once via plain grep/read_file with no graph.

graphify tied on direct lookups, which only need the file that imports a module. It lost on multi-file questions that require reading across several files to synthesize an answer, such as "what is the error handling pattern in the worker?". The graph gives you the map, but you still need to read the territory.

4.Latency: graphify 9.2x faster

Speed is where graphify wins clearly. Average latency per question is 322 ms for graphify against 2,978 ms for grep, a 9.2x speedup. graphify traverses a pre-built graph instead of scanning the filesystem every time. For a codebase that changes rarely, that tradeoff is attractive: build once, query fast.

This is the result of one experiment on one codebase. It is not a claim that graphify is 9.2x faster than grep everywhere.

5.Token cost: graphify uses 1.7x more tokens

graphify returns a BFS traversal subgraph (50-250 nodes); grep returns a file list plus 2-3 files. On 15 questions graphify averages 1,811 tokens per answer and grep 1,050, which is 1.7x more.

6.What graphify got wrong

Four of the fifteen questions were answered wrong, and each miss points at a different limit of the index:

1. fetchRepoStars: the file is not in the index, so the query never sees it.
2. Error handling: the graph finds related nodes, but cannot summarize the try/catch pattern.
3. Clock usage: it finds the definition of Clock, but not every place that uses it.
4. Environment variables: the answer needs .env, which is not code and is not indexed.

Common thread: graphify does well on direct lookups but struggles when the answer requires reading across many files or non-code files.

Decision

Use graphify when

You need fast lookup on a codebase that changes rarely: build once, then query in about 320 ms.

• Which file imports a module
• What a function depends on
• Cross-references in a large codebase

Skip graphify when

The answer needs reading, not locating.

• Synthesizing an answer across many files
• Understanding logic and error handling
• Reading non-code files (config, .env)
• Keeping token cost low: graphify uses 1.7x more tokens per answer

Verdict

For daily dev work on a codebase you know, grep is still more accurate and cheaper in tokens. graphify is a useful index layer, not a replacement.

Machine: MacBook Pro M1, 8 GiB, macOS 26.3. Date: 2026-10-10. Evidence: docs/analysis/2026-10-10-graphify-vs-grep.jsonl (20 edge rows + 15 query rows + metadata).