Bazel Migration: The Four Levels of Readiness

TL;DR: I batch-migrated 22 “ready” packages to Bazel in one afternoon. 17 of them build. Zero are safe to merge. “Ready” in the migration tracker means “no dependency blockers.” What it doesn’t mean is “actually shippable.” The gap between those two things is where monorepo migrations go to die.

This is Part 2 of the Bazel migration series. Part 1 covered the technical blockers — torch, CUDA, the config package that depends on a 68MB GPU framework. This post is about what happens after the technical blockers are solved: the organizational blockers that nobody warns you about.

Previously, on “Why Is This Config Package Importing torch”

Quick recap. We have a 95-package Python monorepo. Eight packages were already on Bazel. Twenty-three were marked “Ready” — no dependency blockers, just waiting for someone to run Gazelle. Fifty-nine were stuck behind GPU/native deps, mostly because a config package transitively drags in torch.

Part 1 ended with a plan. Step 1 was: “Harvest the 23 ready packages. Just… do it.”

So I did.

“I’ll Just Do All of Them at Once”

The plan was simple: take every package the migration tracker marked as “Ready,” run Gazelle on all of them, fix whatever breaks, ship it by end of day. Twenty-two packages (one had been done between posts). One afternoon. How hard could it be.

I spun up a Claude Code engineer agent, pointed it at the migration status file, and said “go.” 263 tool calls and 45 minutes later:

• 135 files changed
• 123 new BUILD.bazel files generated
• Gazelle ran on every package
• python_root directives, visibility declarations, # keep comments — all in place

The infrastructure is all there. It’s beautiful. It’s also, as I was about to discover, not shippable.

The Mechanical Success: 17 out of 22

Seventeen packages build. bazel build //pkg/... — green checkmarks, no drama. For pure Python packages with well-behaved PyPI deps, the rules_py + Gazelle pipeline genuinely works. You point the tool at a directory, it reads your imports, generates BUILD files, and the builds just… succeed.

This is the part that works exactly as advertised. Treasure it.

The Five That Broke (And Why They Broke)

Five packages failed. And the failure modes are instructive, because they’re not bugs in our code or our Gazelle config. They’re bugs in the ecosystem.

1 & 2: mai_config + mai_logger — The Upstream Packaging Bug

These are the two most strategic packages in the entire migration. mai_config alone transitively unblocks ~50 downstream packages. It’s the gateway. The one I spent all of Part 1 talking about.

And it’s blocked by… antlr4-python3-runtime having a broken SOURCES.txt in its sdist.

Here’s what happens: rules_py builds wheels from source inside a sandbox. antlr4-python3-runtime ships an sdist where SOURCES.txt references files that don’t exist. The wheel build fails. Not because of anything we did. Not because of Bazel. Because a third-party package has a packaging bug that pip install silently ignores but a sandboxed build does not.

This is the kind of failure that makes you stare at the ceiling for a while. The two packages that would single-handedly move the migration scoreboard the most are blocked by someone else’s SOURCES.txt.

3: lib/audio_segmentation — Same Class, Different Package

Same story, different victim. The julius package can’t build a wheel from its sdist inside the sandbox. Another upstream packaging issue. Another package that works fine with pip install and explodes in any hermetic build environment.

Pattern recognition kicking in: sdist quality is the hidden tax of hermetic builds. pip is forgiving. Bazel is not. Every sdist that was “good enough for pip” is a potential landmine.

4: caas_server — The Circular Dependency Ouroboros

This one’s different. Gazelle generates the BUILD files fine. They look correct. They even parse correctly. And then bazel build fails because:

docker_api depends on containers
containers depends on utils
utils depends on docker_api

Circular dependencies. The code has subpackages that import each other in a loop. Python doesn’t care — it’ll figure it out at runtime through deferred module loading. Bazel cares very much. A build graph with cycles is not a build graph. It’s modern art.

This isn’t a Bazel problem. It’s a code architecture problem that predates Bazel by years. Bazel just made it visible.

5: lib/tau2 — Same Ouroboros, Different Snake

Same pattern. agent imports utils imports data_model imports agent. Python shrugs. Bazel refuses.

The Blocker Taxonomy

Three of the five failures are upstream packages we don’t control. Two are code architecture problems that require refactoring code owned by other teams. None of them are solvable in this PR.

The Realization: Even the 17 Working Ones Aren’t Shippable

Here’s where the afternoon gets interesting.

I’ve got 17 packages that build. A 4500-line diff. A PR draft ready to go. And then I start thinking about what happens after I click “Create Pull Request.”

And I realize I can’t ship any of it.

Problem 1: Nobody Knows These BUILD Files Exist

Twenty-two teams now have BUILD.bazel files in their packages. Files they didn’t ask for, didn’t know were coming, and won’t know how to maintain. When someone on the coding_qa team adds a new Python file next Tuesday, they won’t update the BUILD file. Why would they? They don’t know it’s there.

Bazel will break. Silently, in CI, on someone else’s PR. And they’ll have no idea why.

Problem 2: No CI Enforcement

Without a “run Gazelle, check BUILD files are up-to-date” CI step, BUILD files drift from reality the moment someone adds or removes a .py file. But adding that CI step means every PR now needs to run Gazelle as part of the check suite. Which adds latency. And requires everyone to understand why CI is telling them their BUILD file is stale.

The lane markings don’t exist yet, and nobody agreed to paint them.

Problem 3: Gazelle Overwrites Manual Fixes

We added # keep comments to preserve manual dependency adjustments. We added gazelle:ignore directives to skip files with circular deps. We manually merged targets to work around import issues.

The next time someone runs bazel run //bazel:gazelle, all of those fixes are at risk. The # keep comment is supposed to prevent this. In practice, it’s a fragile gentleman’s agreement between you and a code generator.

Problem 4: .bazelignore Is Global

Removing packages from .bazelignore means bazel build //... and bazel query //... now see 22 more packages. Including the 5 that don’t build. Anyone who runs //... — which is how most people interact with Bazel — gets failures from packages they’ve never heard of.

Problem 5: We Touched Other Teams’ Files

We had to add # gazelle:ignore comments to 7 Python files in packages owned by other teams. coding_qa, rocket_lib, excel_evaluation. Those teams will see surprise diffs in their files. Diffs they didn’t author, for a system they may not know exists.

“Why is there a # gazelle:ignore comment in my Python file?” is not a question you want 7 teams asking simultaneously.

Problem 6: The 4500-Line PR

Nobody reviews a 4500-line PR properly. It’s too big to reason about. It either gets rubber-stamped by someone who trusts you, or rejected on principle by someone who doesn’t. Neither outcome is “reviewed.”

Problem 7: No Runbook

When a BUILD file breaks after someone’s code change, what do they do? There is no documentation. No “Bazel for package owners” guide. No runbook. No Slack channel to ask in. Just a broken CI and a BUILD.bazel file that feels like someone else’s problem.

Every single one of these is an organizational problem, not a technical one. The technology works. The process doesn’t exist.

The Four Levels of Migration Readiness

Here’s the taxonomy I wish I’d had before starting:

Level 1: Dependency-Ready

All workspace dependencies are migrated. No blockers in the dependency graph.

This is what the migration tracker measures. It’s necessary. It is not sufficient. It’s the equivalent of checking that a road is paved.

22 of our packages are here.

Level 2: Build-Ready

Gazelle generates valid BUILD files. bazel build //pkg/... passes.

This eliminates broken sdists, circular deps, and other issues that only surface when you actually try to build. You know the road is paved and you’ve driven on it.

17 of our packages are here.

Level 3: Test-Ready

bazel test //pkg/... passes, matching existing pytest/uv behavior.

This is where the real work happens — test environment differences, missing test data, import path mismatches, all the fix patterns that come from “builds” and “tests” being different verbs. The road is paved, driven on, and the speed limit signs are posted.

Some packages are probably here. We haven’t systematically verified.

Level 4: Ship-Ready

CI enforcement exists. Teams are notified. Runbook is written. BUILD file maintenance has an owner. The process is sustainable.

This is the one that matters. This is when you can actually merge the PR and not create a slow-motion disaster. The road is paved, driven on, has speed limit signs, and someone is maintaining it.

Zero packages are here.

The migration tracker measures Level 1. Part 1 of this series was about getting more packages to Level 1. The entire batch migration exercise got us to Level 2.

The gap between Level 2 and Level 4 is where migrations stall. And it’s not a gap you can close with an afternoon and a code generator.

What “Ship-Ready” Actually Requires

Here’s the checklist. None of these are technically hard. All of them require coordination, communication, and consensus — which is why they’re harder than the technical work.

1. Gazelle Idempotency CI Check

Run Gazelle on every PR. If BUILD files change, fail the check. This is the “lane markings.” Without it, BUILD files are decorative.

# The check that makes BUILD files real
- name: Check BUILD files are up-to-date
  run: |
    bazel run //bazel:gazelle
    git diff --exit-code '*.bazel' '*.bzl'

2. Package Owner Notification

Slack each team: “Your package now has Bazel build files. Here’s what that means for you. Here’s the 3-minute version of what to do when CI says your BUILD file is stale.”

Not optional. Not “we’ll tell them later.” Before merge.

3. Runbook in MIGRATE.md

A section titled “When Your BUILD File Breaks” that covers:

• What a stale BUILD file looks like in CI
• How to regenerate it (bazel run //bazel:gazelle -- path/to/your/package)
• When to add # keep comments and when not to
• Who to ask for help

4. Ship in Waves, Not Batch

Start with zero-dependency leaf packages: chz, codeshield, log_dedupe. Get feedback from those teams. Fix the rough edges. Then expand.

A 4500-line PR is a statement. A 200-line PR is a conversation.

5. Fix the Five Blockers Separately

The antlr4 sdist issue is upstream — file a bug, or build the wheel externally and pin it. The circular deps need code refactors, which means conversations with the teams that own that code. Neither belongs in a batch migration PR.

The Uncomfortable Insight

Part 1 ended with: “The thing that makes this migration hard isn’t Bazel. It’s that the Python AI/ML ecosystem was built around a different assumption.”

That was about the technical gap — pip vs. hermetic builds, system-installed CUDA vs. resolved dependencies.

Part 2’s version is: The thing that makes this migration hard isn’t generating BUILD files. It’s everything that has to be true about your organization before those BUILD files are safe to merge.

The code generator did its job. 123 BUILD files, 45 minutes. The tooling is there. What’s not there is the organizational scaffolding: the CI checks, the documentation, the team communication, the maintenance ownership.

In a monorepo, “I generated the files” is maybe 30% of shipping a migration. The other 70% is making sure the files stay correct after you stop looking at them. That’s a people problem, not a code problem.

The Scoreboard, Updated

Seventeen out of twenty-two is a great batting average for a code generator. Zero out of twenty-two is a sobering batting average for a migration.

What’s Next

The BUILD files aren’t going anywhere. They’re on a branch, they’re tested, and they’re waiting. The work now is:

1. Write the runbook
2. Build the CI check
3. Pick 3-4 leaf packages, notify those teams, ship a small PR
4. Get feedback, iterate
5. Expand wave by wave

The highway is paved. Now we need the speed limit signs, the lane markings, and the on-ramps. And someone has to tell the drivers the road is open.

Field notes from migrating a 95-package Python monorepo to Bazel. The 22-package batch, the 17/22 build rate, the upstream sdist failures, and the “zero packages ship-ready” realization are all from a real codebase. The 4500-line PR that nobody should review is still sitting on a branch.

Previously: 62% of Our Packages Are Blocked by torch — the technical blockers. This post is the sequel nobody asked for: the organizational blockers.