A few weeks ago I wrote about why Bazel can’t compile your Python C extensions and the spectrum of hermeticity — from “fully sandboxed” to “leave the kitchen door open and hope the grocery store is still there.” That post ended with us at Level 2: leaking the host compiler via --action_env=PATH. A TODO in the code. Tech debt we knew about.
This is the follow-up. PR #22578 closes that TODO. No more host compiler leaks. Bazel now has its own C++ compiler, downloaded and managed like any other dependency.
Here’s how it works, and why the interesting part isn’t the toolchain itself — it’s the patch.
The kitchen analogy, revisited
If you haven’t read the previous post, here’s the short version.
Bazel builds things in a sandbox. The sandbox contains only what you explicitly declare as inputs. This is great for reproducibility — same inputs, same outputs, every time. But when a Python package ships as source code (an “sdist”) instead of a pre-compiled wheel, the build script tries to call gcc or clang. The sandbox doesn’t have a compiler. Build fails.
Think of it like a kitchen. Bazel’s philosophy is: every ingredient must be in the kitchen before you start cooking. The recipe lists what it needs, the kitchen is stocked accordingly, and the result is the same no matter which kitchen you’re in.
Ryan’s earlier fix (--action_env=PATH) left the kitchen door open so the cook could run to whatever grocery store happened to be nearby. Works — but your recipe now depends on which store is nearby. Different machine, different compiler, different build output.
Our fix: stock the kitchen. Put a specific compiler on the shelf, same version every time, and close the door.
Step 1: give Bazel its own compiler
toolchains_llvm is the off-the-shelf solution for this. It’s a Bazel module that downloads a pre-built LLVM/Clang compiler — you pick the version, it handles the rest. Auto-detects your OS and CPU architecture, downloads the right binary, and registers it as Bazel’s C/C++ toolchain.
In MODULE.bazel:
bazel_dep(name = "toolchains_llvm", version = "1.6.0")
llvm = use_extension("@toolchains_llvm//toolchain/extensions:llvm.bzl", "llvm")
llvm.toolchain(llvm_version = "19.1.6")
use_repo(llvm, "llvm_toolchain", "llvm_toolchain_llvm")
register_toolchains("@llvm_toolchain//:all")
That’s it for the toolchain itself. Bazel now downloads Clang 19.1.6 for whatever platform it’s running on — Linux x86_64, macOS ARM64, whatever. Same compiler, everywhere. Hermetic.
But here’s the thing. Having a compiler available in Bazel doesn’t automatically mean Python sdist builds use it. The compiler is registered as a cc_toolchain, which Bazel’s native C++ rules know about. But sdist compilation doesn’t go through Bazel’s native C++ rules. It goes through aspect_rules_py’s sdist_build rule — a special action that runs pip wheel inside the sandbox to compile source distributions.
And sdist_build didn’t know about cc_toolchain.
Step 2: the patch nobody wrote yet
This is the part I find genuinely interesting.
aspect_rules_py is the Bazel ruleset that handles Python dependency management — downloading wheels, compiling sdists, setting up virtual environments. It has a rule called sdist_build that takes a source distribution and compiles it into a wheel inside Bazel’s sandbox.
The authors of this rule knew it needed C++ toolchain support. They literally left a comment in the code:
# FIXME: Add in a cc toolchain here
The FIXME had been sitting there. Nobody had implemented it. So we did.
The patch is 82 lines. Here’s what it does, step by step.
Discovering the toolchain
First, the patched rule needs to find the registered C++ toolchain. Bazel has a standard API for this:
cc_toolchain_info = find_cc_toolchain(ctx, mandatory = False)
The mandatory=False is important. Not every sdist needs a C compiler — pure Python sdists exist, and plenty of sdists only have trivial C extensions that might already be handled. If no toolchain is registered, this returns None and the build proceeds without one. No breakage.
Extracting the compiler paths
When a toolchain is found, we need the actual paths to clang and clang++. Bazel doesn’t just hand you a file path — it uses a feature configuration system that abstracts over different toolchain implementations:
feature_configuration = cc_common.configure_features(
ctx = ctx,
cc_toolchain = cc_toolchain_info,
)
cc_path = cc_common.get_tool_for_action(
feature_configuration = feature_configuration,
action_name = "c-compile",
)
cxx_path = cc_common.get_tool_for_action(
feature_configuration = feature_configuration,
action_name = "c++-compile",
)
cc_common is Bazel’s built-in API for working with C++ toolchains. configure_features sets up the feature configuration for the toolchain (things like optimization levels, debug symbols, sanitizers). get_tool_for_action extracts the path to the specific binary used for a given action — in this case, the C compiler and C++ compiler.
This is the right way to do it. Hardcoding /usr/bin/clang would work on one machine. This works everywhere — because the toolchain abstraction already handled platform detection.
Wiring it into the build action
Now we pass those compiler paths to the sdist build as environment variables:
env = {}
if cc_path:
env["CC"] = cc_path
if cxx_path:
env["CXX"] = cxx_path
When pip wheel (or setuptools, or whatever build backend the sdist uses) runs inside the sandbox, it checks $CC and $CXX to find the compiler. This is standard behavior — every Python build backend respects these environment variables. By setting them, we’re telling the build script: “use this compiler, not whatever’s on the system PATH.”
Making the compiler visible in the sandbox
Setting $CC to a path isn’t enough. The sandbox only contains declared inputs. If the compiler binary isn’t in the sandbox, the path points to nothing.
toolchain_inputs = cc_toolchain_info.all_files if cc_toolchain_info else depset()
This adds the toolchain’s files — the compiler binary, the linker, the standard library headers, everything — to the build action’s inputs. Now when Bazel creates the sandbox, the compiler is physically present at the path we specified.
This is the detail that makes or breaks it. Without this line, $CC points to a path that exists outside the sandbox but not inside it. The build script would get a FileNotFoundError — which, delightfully, would look exactly like the original “no compiler found” error.
The fallback
And if no cc_toolchain is registered? The env dict stays empty, the toolchain inputs are an empty depset, and the build action runs exactly as before. Pure Python sdists don’t notice anything. Sdists that need a compiler and don’t have one still fail — but they were already failing before the patch. No regression.
Step 3: telling Bazel what fasttext needs
There’s one more piece. fasttext doesn’t just need a compiler — it needs build-time Python dependencies. Its setup.py imports pybind11 (the C++/Python bridge), and the build process needs setuptools and wheel.
In a normal pip install, pip handles this automatically — it reads the package’s build requirements, installs them in a temporary environment, and proceeds. In Bazel, you have to declare this explicitly.
pip_annotations.toml:
[fasttext.sdist_build]
deps = ["@pypi//pybind11", "@pypi//setuptools", "@pypi//wheel"]
This tells Bazel: “when building the fasttext sdist, make sure pybind11, setuptools, and wheel are available.” Without this, the build fails at import pybind11 before it even gets to the C++ compilation.
What we removed
With the hermetic toolchain in place, Ryan’s workaround goes away:
--action_env=PATH— gone. The host PATH no longer leaks into builds.--action_env=CC/--action_env=CXX— gone. The compiler comes from the toolchain, not the host.use_default_shell_env = True— gone. Build actions don’t get the host’s shell environment.
And fasttext comes off the exclusion list in find_bazel_ready_packages.py. It’s a real Bazel package now — hermetically compiled, reproducibly built, cacheable like everything else.
Why this matters beyond fasttext
fasttext is the only sdist we’re compiling today. So you could argue this was overkill — why set up a whole toolchain for one package?
Three reasons.
It unblocks more packages. fasttext was a dependency blocker for ~3 packages in our monorepo. Removing it from the exclusion list cascades: those packages can now be Bazel-migrated. Small number, but the pattern matters — every blocker you remove shrinks the “can’t migrate yet” list.
It’s infrastructure, not a one-off fix. The next time someone adds an sdist dependency with C extensions, it just works. No new --action_env hacks, no “why does my build fail in CI but work locally” debugging sessions. The toolchain is there. The patch knows how to use it.
It patches upstream’s FIXME. The sdist_build rule in aspect_rules_py needed this. The authors knew it. We’re carrying a local patch for now, but this is the kind of change that can go upstream — it’s generic, it’s optional (mandatory=False), and it solves a real problem that anyone compiling sdists in Bazel hits.
The kitchen is stocked. The door is closed. The recipe works the same everywhere.