Two Engineers, One Dependency Graph: Parallel CI Work

2026/03/07

BUILDbazelcidep-graphtesting

Yipu and I sit on the same six-person squad. We show up in the same standups. Our names appear in the same Slack channels. If you read our individual status updates, you’d think we work on completely different things. He’s optimizing test execution. I’m migrating the build system. Two parallel tracks, unrelated.

Except they’re not. They’re the same track, seen from different altitudes.

It took me a while to see this. Not because the connection is subtle – it’s not, once you look – but because we each describe our work in the vocabulary of our own layer. He talks about shards and flake rates and find_changed latency. I talk about BUILD files and torch toolchains and Gazelle resolves. Different words. Same dependency graph underneath.

What we each build

Here’s the rough split.

Yipu builds the test discovery and execution layer. His tool, find_changed, answers the question every CI system needs answered: which tests should run for this PR? It does dependency analysis on the changed files, traces the impact through the codebase, and spits out a test matrix. He also handles the aftermath – flaky test auto-disable, auto-revert, CPU test sharding (took that from 13.8 minutes down to 5.6), pyright speedups, runner metrics into Datadog. The whole lifecycle of “something changed, what do we test, how fast can we test it, what do we do when it breaks.”

I build the build system and dependency layer. Bazel migration, the torch toolchain that unblocked 30+ packages, Docker image optimization, dep graph analysis, CI failure investigation automation. The whole lifecycle of “how do we know what depends on what, and how do we build it correctly.”

And then there’s Sriram, whose work I keep bumping into from a third angle. He’s building the orchestration layer – a central test planner that decides which tests to run for a given code change, plus content-based image hashing so CI skips Docker rebuilds when nothing actually changed. If Yipu’s layer answers “what changed and what kind of tests are these?” and my layer answers “what depends on what?”, Sriram’s layer answers “okay, given all that – what do we actually schedule?”

His layer sits on top of both of ours. My layer feeds into Yipu’s. Yipu’s feeds into Sriram’s. None of us planned it that way. It’s just what happens when three people attack the same fundamental problem – “make CI fast, correct, and trustworthy” – from different altitudes.

Where the layers touch

The intersections are specific and concrete. Not abstract “we both care about CI” hand-waving. Actual code touching actual code.

Torch tagging. I built the system torch toolchain – local_runtime_repo plus py_venv_test – that lets Bazel tests import torch from the Docker image. That unblocked 30 packages. But now you need to know which tests actually need torch, because torch tests run on different infrastructure (GPU runners, Docker images with CUDA). Yipu built _check_needs_torch – a detection mechanism that scans test files and figures out whether they need torch at runtime. My toolchain says “torch is available here.” His detection says “this test needs it there.” One without the other is half a solution.

INP001 and package boundaries. This one’s subtle and important. Yipu enforced INP001 across the monorepo – the lint rule that requires every Python package to have an __init__.py. Sounds like housekeeping. It’s not. Without __init__.py, Python’s import system doesn’t recognize a directory as a package. Which means find_changed’s dependency analysis has blind spots – it can’t trace imports through directories that aren’t proper packages. And Bazel has the same problem. Gazelle needs package boundaries to generate correct BUILD files. One lint rule enforcement, two dependency systems made more accurate.

Dependency analysis convergence. find_changed does its own dependency analysis – parsing imports, tracing through the file graph, consulting uv.lock for version changes. I wrote about this in Your Monorepo Has Two Dependency Graphs. Bazel does dependency analysis too, but at a different granularity and with different guarantees. The interesting thing is that both systems get better from each other’s work. When I fix a Bazel dep declaration, that’s a signal about a real dependency. When Yipu’s find_changed catches something Bazel missed (non-Python deps, runtime config loading), that’s a signal about a gap in the build graph.

Datadog. We’re both feeding metrics into the same observability system. Yipu sends test execution times, flake rates, runner utilization. I send image build times, CI pipeline durations. When someone asks “why is CI slow today?” the answer lives in both our dashboards. It’s one system. We just instrument different parts of it.

Test categorization. This one’s developing in real time. Yipu’s _check_needs_torch detects torch imports in test files. Sriram is building AST-based test categorization that’s broader – parsing test files to automatically tag them as GPU, CPU, integration, whatever. They’re both trying to answer the same question: “what resources does this test need?” Sriram’s version feeds into his central planner. Yipu’s version feeds into find_changed. They’ve been collaborating on integrating the two approaches, which makes sense – you don’t want two independent systems classifying the same tests.

Image build skipping. Sriram’s content-based image hashing complements the Docker layer optimization I did with uv-stable.lock. I made individual builds faster by restructuring what gets cached. He makes it so you skip the build entirely when nothing meaningful changed. Different layers of the same “build less” strategy. Neither of us coordinated this. We just both stared at “why is this build running?” and found different parts of the answer.

The idea we keep circling

There’s a conversation Yipu and I have had maybe four times now, in slightly different forms. It goes something like this:

find_changed traces file-level imports to figure out what’s affected by a PR. It’s good at this. But it’s heuristic-based – AST scanning, lockfile diffing, manual annotations for non-Python deps. It works, but it’s a best-effort reconstruction of the dependency graph from source files.

Bazel has the dependency graph. It’s not reconstructed – it’s declared. Every target knows exactly what it depends on, transitively, down to the file level. If you change one file, Bazel can tell you every target in the entire monorepo that’s affected. Not by scanning imports – by reading the actual build graph.

So: what if find_changed used Bazel’s graph instead of building its own?

We’re not there yet. Bazel covers maybe 25% of the monorepo. But the convergence point is clear. In a world where Bazel covers everything, find_changed becomes a thin wrapper: “query Bazel for everything that transitively depends on the changed files, run those tests.” No AST scanning. No manual annotations. No heuristic blind spots. Just the graph.

Here’s the thing that makes me think this architecture is right: Sriram is already building the orchestration layer that assumes it. His central test planner sits above both find_changed and Bazel – it takes signals from both and decides what to schedule. He’s not waiting for Bazel to cover everything. He’s building the planner now, with the current messy hybrid of heuristic analysis and declared deps. But the design clearly gets simpler as Bazel coverage grows. The planner’s “what to test?” question gets trivially answered by the build graph once the build graph is complete.

Three people independently converging on the same architecture is a decent signal that the architecture is right.

The vision is deceptively simple to state: CI knows exactly what to test, builds only what changed, and self-heals when tests flake. From “run everything and hope” to “run exactly what matters and know why.”

We’re maybe 20% of the way there. But 20% already unblocked 30 packages and cut CPU test time in half.

The language thing

Here’s where this post takes a turn.

Yipu and I are both Chinese native speakers on a team where the working language is English. This matters more than you’d think for how we collaborate.

Technical discussion has this property where the concepts are precise but the words are slippery. “Dependency” means four different things depending on context. “Hermetic” means something specific in Bazel and something vague everywhere else. “Flaky” is a judgment call dressed up as a category. When you’re an ESL speaker, these ambiguities don’t slide past you – they snag. You stop and think “wait, which kind of dependency?” in a way that native speakers often skip.

When Yipu and I pair-debug something, we switch languages mid-sentence. Not because we can’t say it in English – we can, and we do, all the time – but because Chinese buys us speed in certain moments. The “oh wait, 那个是不是因为它没有声明 transitive dep?” moments. The shorthand where you skip the scaffolding and go straight to the thought. You’d think this would be confusing, mixing languages. It’s the opposite. It’s like having two gears – English for precision and documentation, Chinese for velocity and intuition.

There’s a specific kind of cognitive load that comes from generating English in real time under pressure. Meeting prep is a whole thing. You rehearse phrases. You pre-load responses to likely questions. You think in Chinese, translate to English, check that the translation didn’t lose anything, and deliver – all while someone’s waiting for you to respond. It’s exhausting in a way that’s invisible to people who don’t do it.

Having a teammate who shares that load changes the dynamics. Not because we speak Chinese in meetings – we don’t, really, beyond the occasional sidebar. But because in the private space between meetings, where the actual thinking happens, we can drop the translation layer and move faster. The ideas get sharper when you can think in the language that’s closer to how your brain actually works.

There’s an irony here that I think about sometimes. We’re building dependency resolution tools. Our whole job is untangling “what depends on what” – tracing import chains, fixing missing declarations, making implicit relationships explicit. And the hardest dependency to resolve on any team isn’t technical. It’s communication. The implicit assumptions, the context that doesn’t survive a Slack message, the thing someone said that you parsed three different ways before asking for clarification.

Language is the dependency nobody puts in a BUILD file.

The shape of the collaboration

From the outside, our work looks like two independent streams that occasionally cross. From the inside, it feels more like one system with two maintainers – or three, really, once you count Sriram’s orchestration layer. Yipu fixes the test flake auto-disable and that changes which signals my torch migration needs to handle. I land the system torch toolchain and that changes which tests his find_changed needs to route to GPU runners. Sriram builds content-based image hashing and that changes how often my Docker optimizations even matter. He enforces INP001 and my Bazel migrations get cleaner. I publish dep graph analysis and he gets data about which packages are most connected.

We don’t coordinate most of this explicitly. There’s no shared project plan with all our names on it. The coordination happens because we’re all staring at the same codebase, the same dependency graph, the same CI pipeline – and when one person improves their layer, the other layers get easier.

I think this is what good DevEx work actually looks like. Not three people working on the same feature. Three people working on different layers of the same system, where the improvements compound. His test execution gets faster. My build system gets more accurate. Sriram’s planner gets smarter about what to schedule. All of those make the others’ jobs easier. A feedback loop that nobody designed but that works anyway, because the underlying problem is one problem.

Run exactly what matters. Know why. Fix it when it breaks.

We’re not there yet. But we’re closer than we were last month, and the path is getting clearer from both ends.