The Parser Map: Every Tool in Your Python Monorepo

There are at least six different tools parsing things in your Python monorepo right now, and none of them agree on what they’re looking at. Some read your .py files. Some read your pyproject.toml. Some read BUILD files that were generated by other tools on this list. They all produce different outputs because they’re asking completely different questions.

This post is the mental model for how they all fit together — the map, not a deep dive into any single territory.

The spectrum

Think of code parsing as a spectrum. On the left: “what characters are in this file?” On the right: “what does this program do?”

Every parsing tool sits somewhere on that spectrum, and where it sits determines what it’s built for, what it’s good at, and where it breaks down.

  Lightweight                                                    Deep understanding
  ─────────────────────────────────────────────────────────────────────────────────>

  Tree-sitter     Black/isort    Gazelle       Ruff            Pyright
  "what syntax    "what does     "what does    "does this      "what does this
   is here?"      this code      this file     code follow     code mean in
                  look like?"    import?"      the rules?"     the type system?"

  CST             AST (format    Import        AST (lint       Full semantic AST
  (concrete       only — no      extraction    rules, unused   (type narrowing,
  syntax tree)    semantics)     (patterns)    vars, control   flow analysis,
                                               flow checks)    cross-file)

The further right you go, the more the tool needs to understand about Python — not just its syntax, but its semantics. And the more it understands, the more expensive it is to build and run.

But here’s the thing — this spectrum is only Track 1. These tools all read your .py files. There are two other tracks running in parallel that don’t read source code at all.

Tree-sitter: the fast, dumb one

Tree-sitter is an incremental parsing library. It reads source code and produces a concrete syntax tree — a CST. Every token, every parenthesis, every whitespace boundary, represented as a tree node.

The key word is concrete. An AST (abstract syntax tree) throws away syntactic noise — it doesn’t care whether you wrote (x + y) or x + y because they mean the same thing. A CST keeps everything. It’s a lossless representation of exactly what you typed.

Why would you want that? Editors.

If you’re doing syntax highlighting, you need to know exactly where every token starts and ends. If you’re doing bracket matching, you need the actual brackets. If you’re doing code folding, you need the concrete structure of the file — where blocks start, where they end, and what nesting level they’re at.

Tree-sitter’s sweet spot:

• VS Code — syntax highlighting, bracket matching, some code navigation
• Neovim — nvim-treesitter does highlighting, text objects, folds
• GitHub — code navigation and syntax highlighting on the web

What tree-sitter does NOT do: understand what the code means. It can tell you there’s an import statement at line 7. It cannot tell you what that import resolves to, whether it’s valid, or what type foo.bar() returns. It’s a syntax tool, not a semantic one.

The tradeoff is speed. Tree-sitter is fast — fast enough to reparse on every keystroke in an editor. That speed comes from not trying to understand anything. Parse the syntax, hand off the tree, move on.

Gazelle: the pragmatic one

Here’s a thing that surprised me: Gazelle does NOT use tree-sitter. I assumed it did — it’s parsing Python files, tree-sitter parses Python files, seems like a natural fit. Nope.

Gazelle has custom parsers per language, and the Python parser is… let’s say targeted. It’s written in Go, and it doesn’t build a full AST. It pattern-matches import statements.

That’s it. It scans your Python files looking for import foo and from foo import bar lines. Regex-style line scanning. It finds the imports, maps them to Bazel targets via gazelle_python.yaml, and generates BUILD files.

This is a brilliant design decision. Gazelle doesn’t need to understand your code. It doesn’t need to know about type annotations, class hierarchies, or control flow. It needs to answer exactly one question: “what does this file import?” And for that question, a regex-level scanner is enough 95% of the time.

The other 5% is where it falls apart.

# Gazelle sees this:
import torch                              # ✅ found it

# Gazelle does NOT see these:
importlib.import_module("torch")          # ❌ dynamic import
if sys.platform == "linux":
    import triton                          # ❌ conditional import (it'll find it,
                                           #    but doesn't know it's conditional)
__import__("some_module")                  # ❌ runtime import

If you’ve ever wondered why Gazelle generates wrong deps for files with metaprogramming, or why it can’t handle conditional imports — this is why. It’s not parsing Python. It’s scanning for patterns that look like imports. When your code does anything clever with its imports, the pattern matcher misses it.

For reference, Gazelle’s Go parser is much more sophisticated — it uses Go’s stdlib go/parser, which produces a full AST. Go earned the real parser because Go’s import syntax is simple and the stdlib parser exists. Python got the quick-and-dirty version because Python’s import system is a fractal of complexity and building a full parser in Go for it would be a project unto itself.

Pyright: the one that actually understands Python

Pyright sits at the far right of the spectrum. It doesn’t just parse your code — it understands it. That sentence deserves unpacking, because the list of things Pyright tracks is long and each one adds a layer of “understanding” that simpler parsers don’t have.

Types — the most basic layer. x: int = 5 tells Pyright that x is an integer. Call x.upper() and Pyright knows that’s wrong — ints don’t have .upper(). Tree-sitter sees x.upper() as valid syntax. Gazelle doesn’t even look at it. Pyright is the only one that says “hey, that’s a bug.”

Control flow — Pyright tracks which path your code takes. If you have if x is None: return, then after that line, Pyright knows x is not None anymore. It follows every branch, every early return, every raise. It’s simulating the runtime execution order without actually running anything.

Narrowing — this is control flow applied to types. Classic example:

def process(x: str | int):
    if isinstance(x, str):
        print(x.upper())   # Pyright knows x is str here — .upper() is valid
    else:
        print(x + 1)       # Pyright knows x is int here — arithmetic is valid

After the isinstance check, Pyright narrows the type from str | int to just str inside that branch. It’s not guessing — it’s tracking how conditionals constrain what a variable can be.

Overloads — same function name, different behavior depending on argument types:

@overload
def fetch(id: int) -> User: ...
@overload
def fetch(id: str) -> list[User]: ...

Call fetch(42) and Pyright knows you get a User. Call fetch("admin") and you get a list[User]. Same function, different return types. Pyright picks the right overload based on what you pass in.

Generics — types with a slot for “fill in the blank later.” list[int] is a list of integers. list[str] is a list of strings. The list itself is generic — it works with any type. When you write def first(items: list[T]) -> T, Pyright tracks that T is whatever type the list contains. Pass list[int], get int back.

Protocols — Python’s version of “if it walks like a duck and quacks like a duck.” Instead of checking inheritance (class Dog(Animal)), protocols check shape:

class Quacker(Protocol):
    def quack(self) -> str: ...

# Any object with a .quack() method satisfies this — no inheritance needed

Pyright checks whether your object has the right methods and attributes, regardless of what class it inherits from. This is structural typing — match the shape, not the family tree.

All of this together is what people mean by “the whole type system.” Pyright is running a simulation of Python’s type-level behavior in your editor — every assignment, every branch, every function call, every generic parameter — tracking what types flow where and flagging when something doesn’t fit.

And no, it doesn’t use tree-sitter either. Pyright has a hand-written recursive descent parser in TypeScript that produces a full Python AST.

Why not tree-sitter? Because tree-sitter gives you a CST — a syntax tree. Pyright needs a semantic tree. It needs to know that after if isinstance(x, str):, the variable x is narrowed to type str inside that branch. It needs to track assignments through control flow. It needs to resolve imports, follow them into other files, and build a cross-file understanding of your type relationships.

A CST doesn’t give you any of that. You’d have to build all the semantic analysis on top of the CST anyway, and at that point you might as well write your own parser that produces exactly the tree structure you need. Which is what Pyright did.

This is the most expensive kind of parsing. Pyright is essentially running a simplified version of the Python interpreter in your editor — not executing code, but simulating the type-level behavior of every expression.

The payoff: Pyright catches bugs that no other tool can see. Type mismatches, unreachable code, invalid attribute access, wrong argument types. It’s not reading syntax — it’s reading meaning.

Black and isort: the reformatters

Between tree-sitter and Gazelle on the spectrum, there’s a class of tools that parse your code into an AST, rearrange it, and print it back out — without caring what any of it means.

Black parses Python into an AST, applies formatting rules (indentation, line length, trailing commas), and writes the reformatted code back. It doesn’t know what your variables are. It doesn’t know if your function is correct. It just knows that this if statement needs a newline here and this dict literal should be broken across lines.

isort is even more targeted — it only cares about your import block. It parses the imports, sorts them into groups (stdlib, third-party, local), and rewrites them. Everything below the import block? Invisible.

Both tools use Python’s built-in ast module (or a fork of it) for parsing. They need just enough understanding to know where the boundaries are — where one statement ends and another begins, where an expression wraps to the next line. But they have zero interest in types, correctness, or semantics.

This is why Black can format syntactically valid but completely broken code without complaint. It doesn’t know your code is broken. It just knows the indentation is wrong.

Sidebar: Python’s ast module — Python ships with its own AST parser in the standard library (import ast). It produces an abstract syntax tree: every expression, statement, and block as a tree node, with syntactic noise stripped away. Black, isort, and plenty of custom tooling use it. It’s the “default parser” when you need to understand Python structure but don’t need types. If you’ve ever written a codemod script or a custom linter, you probably used ast.

Ruff: the fast linter

Ruff sits between Gazelle and Pyright on the spectrum. It parses Python into a full AST (using its own parser written in Rust — that’s why it’s fast enough to replace both flake8 and isort in a single pass) and applies lint rules.

What Ruff can see that Gazelle can’t:

• Unused imports — import os at the top but os never used? Ruff catches it. Gazelle can’t — it just extracts imports, it doesn’t check usage.
• Unreachable code — a return before the last line of a function? Ruff follows the control flow enough to flag it.
• Shadowed variables — defining list = [1, 2, 3] that shadows the builtin? Ruff sees the AST and knows.
• Style violations — all the stuff flake8 used to do, but faster.

What Ruff can’t see that Pyright can:

• Type mismatches — x: int = "hello" is valid syntax. Ruff won’t flag it. Pyright will.
• Invalid attribute access — x.upper() on an int. Ruff doesn’t track types, so it can’t know.
• Cross-file type relationships — Ruff processes files independently. Pyright builds a project-wide type graph.

Ruff’s sweet spot is structural correctness without type knowledge. It knows the shape of your code but not the meaning. Fast enough to run on every save, smart enough to catch the obvious stuff, and deliberately stops before entering Pyright’s territory.

The practical result: Ruff and Pyright are complementary, not competing. Ruff catches the fast stuff (style, unused code, simple bugs). Pyright catches the deep stuff (type safety, semantic correctness). Most monorepos run both.

Track 2: Package resolution

Everything above lives on the “source code parsing” track. But there’s a whole parallel world of tools that never read your .py files at all.

uv, pip, and poetry operate on a different input entirely:

uv is a constraint solver（约束求解器）. You say “I need torch>=2.0 and numpy>=1.24.” uv goes to PyPI, reads the metadata for every version of torch and numpy, and finds a set of versions where all requirements are satisfied simultaneously. torch 2.1.0 needs numpy>=1.23,<2.0 — does that overlap with your numpy>=1.24? If yes, proceed. If no, conflict.

This is a fundamentally different problem from “what does this file import?” Gazelle answers the code-level question (file A imports module B → add dependency). uv answers the package-level question (package X requires package Y version Z → resolve the graph).

They interact at the boundary: Gazelle determines which Bazel targets depend on which. uv determines which PyPI packages to install. But they’re solving different constraint problems on different inputs.

pip does the same job as uv, just slower and with a less sophisticated resolver. poetry adds its own lockfile format and virtual environment management. The point is: none of them read your source code. They read metadata about your source code.

Track 3: Build graph analysis

The third track doesn’t parse Python or package metadata. It parses BUILD files and the dependency graph that Bazel computed from them.

bazel-diff answers: “Given BUILD files at commit A and commit B, which targets changed?” It’s a graph diffing tool. It takes two snapshots of the Bazel target graph and computes the delta — which targets were added, removed, or modified, and which downstream targets are affected.

bazel query / bazel cquery let you ask questions about the target graph: “what depends on //lib/mai_config?”, “what are the transitive deps of this test?”, “show me every target tagged torch.” These are graph traversal queries on the build system’s data structure — they don’t read Python source code at all.

Buildifier is the BUILD file equivalent of Black — it reformats .bzl and BUILD files. It parses Starlark (Bazel’s configuration language), applies style rules, and writes them back. It has no idea what the BUILD file does in terms of building software. It just knows the formatting is wrong.

The interaction between tracks: Gazelle (Track 1) generates BUILD files by parsing source code. bazel-diff (Track 3) analyzes those BUILD files to detect changes. uv (Track 2) installs the packages that the BUILD files reference. Each track feeds into the others, but they parse completely different inputs.

The full map

Here’s how it all fits together:

TRACK 1: SOURCE CODE PARSING (reads .py files)
──────────────────────────────────────────────────────────────────────────>
Tree-sitter   Black/isort   Gazelle      Ruff         Pyright
syntax        formatting    imports      lint rules   full type system
CST           AST (shape)   patterns     AST (rules)  semantic AST

TRACK 2: PACKAGE RESOLUTION (reads pyproject.toml, lockfiles, registries)
──────────────────────────────────────────────────────────────────────────>
pip               poetry              uv
simple resolver   lockfile + venvs    fast constraint solver

TRACK 3: BUILD GRAPH (reads BUILD files, target graph)
──────────────────────────────────────────────────────────────────────────>
Buildifier        bazel query         bazel-diff
format BUILD      graph traversal     graph diffing
files             "what depends       "what changed
                   on what?"           between commits?"

            ┌──────────────────────────────────┐
            │  Gazelle connects Track 1 → 3    │
            │  (reads .py → generates BUILD)   │
            │                                  │
            │  uv connects Track 2 → runtime   │
            │  (resolves packages → installs)  │
            │                                  │
            │  Pyright uses Track 2 output     │
            │  (installed packages → type info)│
            └──────────────────────────────────┘

None of these tools are wrong. They’re all operating on different inputs, asking different questions, at different depths. The confusion happens because they all involve the word “dependencies” — but code-level deps (what does this file import?), package-level deps (what PyPI packages do you need?), and build-level deps (what targets depend on what?) are three separate problems.

How they remember: caching strategies

Every tool on the map faces the same question: “I did work last time. Can I skip doing it again?” The answer reveals as much about a tool’s design as its parsing depth does. Each caching strategy is a direct consequence of what the tool parses and how its results go stale.

Tree-sitter: reuse the tree, re-parse the edit

Tree-sitter’s caching is the most elegant on the list because it doesn’t feel like caching at all — it’s incremental parsing. The old syntax tree IS the cache.

When you type a character, tree-sitter doesn’t re-parse the whole file. You tell it “bytes 247 through 247 are now bytes 247 through 248” (one character inserted). It walks the old tree, finds which subtrees overlap with the edit, and marks them dirty. Everything else — the entire rest of the file — gets reused by bumping a reference count on the old subtree nodes.

  Before edit:        After edit:
  ┌──────────┐        ┌──────────┐
  │ function │        │ function │  ← reused (refcount++)
  ├──────────┤        ├──────────┤
  │ if-block │        │ if-block │  ← DIRTY: re-parsed
  ├──────────┤        ├──────────┤
  │ return   │        │ return   │  ← reused (refcount++)
  └──────────┘        └──────────┘

The new tree structurally shares most of its nodes with the old tree. A keystroke in a function body does O(log n) work, not O(n). That’s why tree-sitter can reparse on every keystroke — it barely reparses anything.

All in-memory. No disk cache. When the editor process dies, the tree is gone. Makes sense — why persist a syntax tree that takes milliseconds to rebuild?

Black: “was this file already formatted?”

Black’s question is binary: is this file already formatted, yes or no? So its cache is a lookup table: file path → (mtime, size, sha256).

On the next run, Black checks each file against the cache:

1. mtime matches? → skip. No hash needed. (Fast path.)
2. mtime differs but hash matches? → skip. File was touched but not changed.
3. Hash differs? → reformat.

The cache lives on disk at ~/.cache/black/<version>/cache.<mode>.pickle. Key detail: it’s versioned per Black version AND per formatting mode. Change your line length? New cache key. Upgrade Black? New cache directory. The old entries don’t get invalidated — they just become unreachable.

isort has no cache at all. Every run re-reads and re-sorts every file. It’s fast enough that nobody noticed.

Ruff: per-file results, content-addressed

Ruff caches lint results — the actual diagnostics — keyed per file. The cache lives at .ruff_cache/<version>/<settings_hash>/, where the settings hash covers every lint rule, target Python version, line length, everything. Change one setting → different directory → cold cache.

Per file, the cache key is (last_modified_time, permissions). If those match, Ruff returns the cached diagnostics without re-parsing or re-linting.

This is why ruff check on an unchanged codebase is near-instant, even on thousands of files — it’s doing timestamp comparisons, not AST parsing. The first run pays the full cost. Every subsequent run on unchanged files is a stat syscall per file.

Stale entries get pruned automatically after 30 days of not being seen.

Gazelle: no cache, by design

Gazelle has zero cross-run caching. Every invocation scans every directory, re-reads every .py file, re-extracts every import, and regenerates BUILD files.

Within a single run it deduplicates filesystem reads (if two goroutines need the same directory, only one reads from disk). But nothing persists to disk between runs.

Why no cache? Because Gazelle’s job is to see the current filesystem. Caching old directory listings would be dangerous — you’d miss newly created files. And the actual work (scanning for import patterns) is cheap enough that re-doing it takes a few seconds on a large monorepo. The cost doesn’t justify the cache complexity.

Pyright: the dependency graph in memory

Pyright’s caching is the most sophisticated because it has the most to lose from re-doing work. Full semantic analysis of a large codebase takes minutes. Re-analyzing on every keystroke would be unusable.

The solution: incremental analysis with cross-file dependency tracking.

Each source file tracks three independently-invalidatable phases:

1. Parse — tokenize + build AST
2. Bind — build scopes, symbol tables, control flow graph
3. Check — evaluate types, generate diagnostics

When you edit file A:

• Pyright re-parses A (phase 1)
• It checks if the content hash actually changed (you might have hit save without changes). If not → stop. No cascade.
• If content changed → re-bind and re-check A (phases 2-3)
• Then: walk the importedBy reverse dependency graph. Every file that imports A gets phases 2-3 invalidated → re-check those too

This cascade can be expensive in a tightly-coupled codebase. Change __init__.py in a core library → every file that imports from it gets re-analyzed. Change builtins.pyi → every single file gets re-analyzed (builtins are implicitly imported everywhere).

All in-memory, in the long-running LSP server. No disk cache. Restart Pyright and it re-analyzes from scratch.

Change pruning happens naturally: if re-analyzing file B (because it imports A) produces identical diagnostics, the editor doesn’t get a redundant update. The diagnostic version counter handles this.

uv: the clone trick

uv’s speed advantage over pip isn’t just faster resolution (though it is faster). The killer feature is how it installs from cache.

Both pip and uv cache downloaded wheels on disk. The difference is what happens next:

On macOS, clonefile() is essentially free — it tells the filesystem “this is the same data, don’t actually copy bytes, just point at it.” A 500-package install becomes 500 syscalls that each take microseconds. pip copies every file byte-by-byte.

Cache invalidation varies by source:

• PyPI packages: HTTP cache headers (ETags, Last-Modified)
• Git dependencies: Keyed by resolved commit hash — immutable, never stale
• Local directories: mtime of pyproject.toml (you can customize this via tool.uv.cache-keys)

Bazel: content-addressed everything

Bazel has the most rigorous caching model of any tool on this list because it was designed as a caching system that happens to build software.

Every build action (compile, link, test, etc.) has a cache key computed from:

• The command line
• Content hashes of all input files (NOT timestamps — never timestamps)
• Environment variables
• Declared output names

The results go into two stores:

1. Action Cache (AC): action_key_hash → output metadata (which outputs were produced, their hashes)
2. Content-Addressable Store (CAS): content_hash → file blob (the actual files, deduplicated)

Change a source file → its content hash changes → the action that used it has a different key → cache miss → rebuild. But only that action and its transitive dependents. Everything else still hits cache.

Change pruning is the clever bit: if rebuilding an intermediate target produces a bitwise-identical output (e.g., you changed a comment in C++ — the .o file is the same), Bazel detects this and does not invalidate downstream actions. The hash didn’t change, so the cache entry is still valid. The rebuild ripple stops.

Remote cache (--remote_cache) puts the same AC + CAS on a shared server. One developer builds something → the action result + output blobs go to the server → another developer (or CI) running the same action gets a cache hit. Same content hash → same result.

bazel query benefits indirectly: the Bazel server keeps a long-running in-memory graph (Skyframe). First query loads and parses all BUILD files (expensive). Subsequent queries reuse the graph with incremental invalidation — only re-parsing BUILD files that changed on disk. Kill the server → the graph is gone → next query starts cold.

The caching spectrum

Just like parsing depth, caching strategy follows the tool’s core tradeoff:

The pattern: tools that run once and exit (Black, Ruff, Gazelle) cache to disk or not at all. Tools that stay alive in your editor (Tree-sitter, Pyright) cache in memory with incremental invalidation. Tools that need to share results across machines (Bazel, uv) use content-addressed stores.

And the most revealing detail: Bazel never uses timestamps. Every other tool (Black, Ruff, uv for local dirs) uses mtime as a fast check because hashing file contents is expensive. Bazel hashes everything because in a distributed build system, timestamps are unreliable — different machines, different clocks, different filesystem semantics. Content hashes are the only ground truth. It’s slower per file, but it’s correct across machines, which is the whole point of a remote cache.

Why this matters for Bazel migration

If you’re migrating a Python monorepo to Bazel, you’re living with all three tracks simultaneously, and understanding both parsing depth and caching strategy explains most of the friction.

Gazelle breaks on clever code — because it’s a pattern matcher, not a parser. Every importlib.import_module(), every try: import foo except: import bar, every plugin loader that discovers modules at runtime — Gazelle can’t see these. You’ll be writing # gazelle:resolve and # gazelle:ignore directives by hand for every edge case. This isn’t a bug in Gazelle. It’s the intentional tradeoff of a tool that chose speed and simplicity over completeness.

Pyright and Gazelle disagree about deps — because they see different things. Pyright follows TYPE_CHECKING imports (the ones inside if TYPE_CHECKING: blocks). Gazelle finds them too but might map them differently. Pyright resolves re-exports through __init__.py chains. Gazelle sees the direct import and maps it to whatever gazelle_python.yaml says. Same file, different dependency conclusions.

Tree-sitter is invisible — and that’s the point. It’s running in your editor, making your syntax highlighting work, and you never think about it. Until you’re debugging a Gazelle-generated BUILD file and you notice that your editor highlights the import statement just fine (tree-sitter sees it) but Gazelle missed it (pattern matcher didn’t). The syntax is valid. The import extraction failed. Different tools, different depths.

The pattern: every tool parses only what it needs

None of these tools are wrong. They’re all making the same engineering tradeoff: parse only as deep as your use case requires.

This is a general principle in tooling: the depth of parsing is a function of the question being asked. If you only need syntax, don’t build semantics. If you only need imports, don’t build a full AST. If you only need package versions, don’t read source code at all. Each layer of understanding has a cost — in complexity, in speed, in maintenance burden — and good tools stop at the layer they need.

The confusion comes from the fact that they often use the same words. “Dependency” means three different things depending on which track you’re on. “Parsing” means everything from regex pattern matching to full type system simulation. “Understands your code” ranges from “can highlight it” to “can prove it’s type-safe.”

Your editor sees syntax. Your formatter sees shape. Your build system sees imports. Your linter sees patterns. Your type checker sees meaning. Your package manager sees versions. Your build graph sees targets. They’re all correct. They’re all incomplete. And when you’re debugging a build failure that your editor says shouldn’t exist, the answer is usually: the tool that found the problem is operating on a different track — or parsing deeper on the same one — than the tool that missed it.