The previous post teased this at a high level: with a complete dependency graph, you can scope pyright to only the packages affected by a change. Five minutes drops to seconds. Sounds great. Ship it.
This post is the implementation story. Including the part where I made it worse before I made it better.
The starting point
Our monorepo has 106 Python packages. Pyright checks them all on every PR — 108 execution environments in a single pyrightconfig.json, running on an 8-core CI runner. Every PR. Every time. About five minutes.
I’ve written about this before. The math: ~100 PRs per week, 50 weeks a year, ~417 dev-hours annually of engineers watching a type checker re-verify 105 packages they didn’t touch. That’s a full-time engineer’s year. Just… waiting.
The obvious fix: only check the packages that changed and everything that depends on them. You need a dependency graph for that. And as of last week, we have one.
Why not Bazel’s graph?
Bazel has rdeps(). It’s beautiful. You ask “what depends on fortknox?” and get back exactly the right targets. Twenty lines of bash replaces a thousand lines of Python.
One problem: Bazel only covers the packages that have real BUILD files. We’re at about 54 out of 106. The stubs I added for graph analysis (PR #23805) have tags = ["manual"] — they show up in bazel query, but they don’t have pyright targets. They don’t have any targets that do anything. They’re ghosts.
Yipu’s PR #22032 already adds pyright_targets — a tool that does BFS on the Bazel dep graph to scope pyright to affected packages. The approach is right. But it can only see the half of the monorepo that’s been migrated.
So for scoped pyright, we need a graph that covers all 106 packages. The uv workspace dependency graph does this. Every package declares its dependencies in pyproject.toml. uv resolves them into uv.lock. The graph is already there — sitting in the lockfile — covering every single workspace member. No migration required.
The pipeline
Five steps. Each one is boring. The composition is the interesting part.
Step 1: git diff → changed files.
git diff --name-only origin/main...HEAD
Standard stuff. Same thing find_changed does, same thing every CI system does. You get back a list of paths.
Step 2: File paths → workspace packages.
This is a longest-prefix match against workspace member paths. The root pyproject.toml declares members like lib/fortknox, mai_config, rocket. A changed file at lib/fortknox/src/fortknox/core.py maps to the fortknox package. Straightforward — sort workspace paths by length, take the longest match.
Step 3: Reverse BFS on the dep graph → transitively affected packages.
This is the one that matters. Changed fortknox directly. But common_utils depends on fortknox, and mai_job depends on common_utils, and… you trace the chain. BFS from the changed packages, following reverse dependency edges. Everything you reach is “affected.”
The graph comes from ci/workspace_graph.py, which parses uv.lock and the workspace pyproject.toml files. It already existed — the CI test selector uses it. I’m reusing infrastructure, not building new infrastructure.
Step 4: Affected packages → pyright execution environments.
Map the affected canonical package names back to the root field in pyrightconfig.json’s executionEnvironments. Package fortknox maps to environment root lib/fortknox. Some packages are entirely excluded from pyright (like lib/bus with its C++ — pyright can’t do anything with it), so those get filtered out.
Step 5: Run pyright with a scoped config.
Create a temporary pyrightconfig.json that only includes the affected execution environments. Pass it to pyright with -p. Done.
At least, that’s what I thought.
The bug
I built all five steps. I ran a test: simulate changing one file in ci, a true leaf package. Zero reverse dependencies. One execution environment out of 108.
Expected: pyright runs on 1 environment. Fast. Maybe 5 seconds.
What actually happened: pyright ran slower than the full check and reported errors in packages I hadn’t asked it to check.
I sat there for a while. Stared at the output. Went back and re-read my code. The temp config had exactly one execution environment. That was correct. I had filtered out 107 of 108 envs. That was correct. Pyright was reading the temp config — I verified the -p flag was pointing at the right file. That was correct.
Everything was correct and everything was wrong.
Pyright’s default behavior
Here’s the thing about pyright that is not obvious until it ruins your afternoon.
Pyright has execution environments. Each one claims a set of files via its root directory. Files in that root get checked under that environment’s settings. Good. Makes sense.
But what about files that aren’t in any execution environment’s root?
Pyright checks them anyway. Under the top-level config. Every .py file in the project root that isn’t claimed by an execution environment gets type-checked under the default settings.
Read that again. If you have 108 execution environments and you remove 107 of them, those 107 packages don’t get skipped. They get promoted. They move from their per-package environment (with its carefully tuned severity overrides) into the root-level default check. Pyright now scans them all — under the strictest settings — as unclaimed files.
Fewer environments = more files checked. The exact opposite of what you’d expect.
It’s like removing rooms from a house’s thermostat system. You don’t save energy. The rooms don’t stop existing. They just all start getting heated by the hallway thermostat, which is set to 80°F because nobody configured it carefully.
The fix
One line. Add "include" to the temporary config:
# CRITICAL: Set `include` to only the affected env roots. Without this,
# pyright checks ALL files in the project root that aren't claimed by an
# execution environment — meaning removing 107 envs causes those 107
# packages to be checked under the default (root-level) config, which is
# actually worse than checking everything.
config["include"] = list(env_roots)
The include directive tells pyright: “Only look at files in these directories. Everything else? Not your problem.” Now when we filter to 1 execution environment, pyright only sees the files in that one directory. The other 107 packages aren’t unclaimed — they’re invisible.
Before the fix: fewer envs → pyright scans more files → slower. After the fix: fewer envs → pyright scans fewer files → faster.
The kind of bug where the code does exactly what you told it to, and what you told it to is wrong, because the tool’s behavior is not what you assumed.
The benchmarks
Local machine, pyright 1.1.408, 6 threads. Full baseline: 82.5 seconds.
| Scenario | Envs checked | Time | Speedup |
|---|---|---|---|
True leaf (ci) |
1 / 108 | 2.2s | 37.5x |
Mid-level (mai_job) |
10 / 108 | 15.5s | 5.3x |
Leaf with fans (fortknox) |
36 / 108 | 34.0s | 2.4x |
Core config (mai_config) |
67 / 108 | 60.6s | 1.4x |
The time model is roughly linear with the number of environments, plus about 2 seconds of fixed startup overhead. Which makes sense — pyright starts up, loads the config, then does work proportional to how many files it needs to check.
CI runners are about 3.6x slower than my local machine. So that 2.2-second leaf case becomes roughly 8 seconds in CI. Still a 37x improvement over the ~300-second full run.
The shape of the graph
The benchmarks are nice, but the real question is: what do typical PRs look like? Because if every PR touches mai_nano (44 reverse deps) and scoping only saves 10%, this is a science project, not an optimization.
Here’s what the graph actually looks like:
- 106 total workspace packages
- 31 are true leaves — zero reverse dependencies. Maximum benefit. 37x speedup territory.
- ~60% of packages have fewer than 15 reverse deps — consistently >50% savings
- High-fan roots:
mai_nano(44 rdeps),conversation(40),blobfile(35),mai_config(24)
Most real PRs touch 1-3 packages. The median package has fewer than 15 rdeps. That means most PRs land in the 50-90% reduction range. The core-library PRs that touch everything are the exception, not the norm — and even those get some benefit (1.4x for mai_config).
The “no Python changes” case is the silent winner. Someone edits a README, a YAML file, a Dockerfile. Current system: pyright runs for 5 minutes. Scoped system: detects zero Python changes, skips entirely. That’s not a speedup multiplier — it’s going from 5 minutes to zero.
The tool
bazel/tools/scoped_pyright.py — about 780 lines (PR #24017). It’s a CLI with four commands:
analyze — dry run. Show what would be checked without running pyright:
uv run --package ci python bazel/tools/scoped_pyright.py analyze \
--simulate lib/fortknox/src/fortknox/core.py
run — analyze then execute pyright on the scoped config.
shadow — the paranoia mode. Run scoped pyright and full pyright, compare results. If the scoped run misses errors that the full run catches, something is wrong with the graph or the mapping. This is how you build trust in the tool before putting it in the critical path.
scenarios — batch-run predefined scenarios (leaf packages, core libs, markdown-only, etc.) and output a comparison table. For data collection and benchmarking.
The dependency graph comes from ci/workspace_graph.py, which already existed for the CI test selector. I didn’t write a graph library. I reused the one we had. The new code is just the pyright-specific parts: parsing pyrightconfig.json, mapping packages to execution environments, generating the scoped config, and running pyright with it.
Shadow CI
The tool has a shadow workflow — a CI job that runs scoped pyright alongside the full run and compares results. This serves two purposes:
-
Correctness validation. Does scoped pyright catch the same errors as full pyright? If it misses something, the shadow run catches the divergence before any developer is affected.
-
Data collection. Real-world scope ratios, actual speedups, edge cases where scoping breaks down. You need data from real PRs, not simulated scenarios, to know if this actually works in practice.
The shadow phase runs in parallel with the existing full pyright job. No risk. No behavioral change for developers. Just data accumulating quietly in the background until we’re confident enough to swap.
What this connects to
This is part of a series, and the pieces fit together:
When Homegrown Tools Hit the Ceiling described the moment we realized find_changed was topping out — still useful, but increasingly a liability for the things it couldn’t see.
Your Monorepo Has Two Dependency Graphs mapped out how find_changed and Bazel see different graphs from the same codebase.
The Dep Graph Is the Product was about getting to 100% Bazel graph coverage with manual stubs — making the graph complete enough to be useful.
What a Full Dep Graph Actually Unlocks surveyed all the things a complete graph enables. Scoped pyright was one bullet point in that post.
This post is the deep dive on that bullet point. The part where you stop saying “we could scope pyright with the dep graph” and start saying “here’s the code, here’s the bug I hit, here’s the benchmark, here’s the shadow CI collecting data.”
The pattern repeats: a complete dependency graph is infrastructure. Once you have it, the applications are straightforward — file-to-package mapping, reverse BFS, filter, run. The graph is the hard part. Everything else is plumbing.
Except when the plumbing has a pyright quirk that inverts your assumptions and makes the scoped run slower than the full run. Then the plumbing is the hard part too.
This is the fifth post in a series about dependency graphs in a Python monorepo. Previous posts: When Homegrown Tools Hit the Ceiling, Your Monorepo Has Two Dependency Graphs (and They Disagree), The Dep Graph Is the Product, What a Full Dep Graph Actually Unlocks.