Bazel CC Toolchain for Python C Extensions

2026/02/27

BUILDbazeldependencies⭐

Most Python packages install in under a second. You download a .whl file, unzip it, and you’re done. No compiler. No headers. No drama.

Then you add fasttext to your dependencies and everything explodes.

The error will say something about a missing C++ compiler. Or gcc not found. Or — my personal favorite — just a raw FileNotFoundError on /usr/bin/cc. In a normal pip install, you’d fix this by installing build tools. In Bazel, it’s a different story, because Bazel’s whole thing is that nothing from your machine is supposed to leak into the build.

This is the story of wheels, sdists, hermetic sandboxes, and the cc_toolchain — Bazel’s answer to a question that most Python developers never had to ask.

Two ways to ship a Python package

PyPI — the place pip install pulls from — serves packages in two formats. Understanding the difference is the whole game.

Wheels: the prebuilt binary

A wheel (.whl) is a ZIP file with a funny extension. Inside: Python code, maybe some pre-compiled .so or .pyd files, and metadata. That’s it. Installation is literally “unzip to the right directory.” No compilation, no build step, no compiler needed.

When a package maintainer publishes a wheel, they compile the C extensions ahead of time and ship the result. They do this separately for each platform — numpy has wheels for Linux x86_64, Linux aarch64, macOS arm64, macOS x86_64, Windows. Each one contains binaries compiled for that specific platform.

numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.whl
numpy-1.26.4-cp312-cp312-macosx_14_0_arm64.whl
numpy-1.26.4-cp312-cp312-win_amd64.whl

That filename encodes everything: package version, Python version, platform, architecture. pip reads it, picks the right wheel for your system, downloads it, unzips it. Done.

This is the happy path. Most popular packages ship wheels. When you pip install numpy and it takes 2 seconds, that’s a wheel.

Sdists: the source code tarball

An sdist (source distribution, .tar.gz) is the raw source code of the package. If the package has C extensions — meaning it wraps C or C++ code for performance — the sdist contains that C/C++ source, plus a build script (usually setup.py or pyproject.toml with a build backend like setuptools or meson).

Installing an sdist means building the package from source on your machine. pip downloads the tarball, extracts it, runs the build script, which invokes a C compiler, which compiles the extensions, which produces .so files, which get installed alongside the Python code.

This requires:

Why some packages only ship sdists

You might think: why wouldn’t every package ship wheels? It’s strictly better for users.

A few reasons:

fasttext is a classic example. It wraps Facebook’s C++ library. The maintainers ship sdists and occasionally pre-built wheels for a subset of platforms. If your platform isn’t covered — or if the latest version hasn’t built wheels yet — pip falls back to the sdist, and now you need a compiler.

In a normal development environment, this is a speedbump. Install build-essential, try again, move on with your life.

In Bazel, it’s a wall.

Bazel’s hermetic sandbox

Here’s the thing about Bazel: it doesn’t trust your machine. By design.

When Bazel builds something, it creates a sandbox — an isolated directory containing only the declared inputs for that build action. Your source files, your declared dependencies, and nothing else. No /usr/bin. No $PATH. No system headers. No compiler (unless you explicitly gave it one).

This is hermeticity. The build’s output depends only on its declared inputs, not on whatever happens to be installed on the machine running it. Same inputs → same outputs, whether it’s your laptop, your coworker’s laptop, or a CI runner in Azure.

For pure Python packages (wheels), this is perfect. Bazel downloads the wheel, verifies its hash, unpacks it into the sandbox. No compilation needed. The binary artifacts are already in the wheel.

For sdists, the sandbox is a problem. The build script says “call gcc.” Bazel’s sandbox says “what’s a gcc?”

error: command 'gcc' failed: No such file or directory

The compiler isn’t in the sandbox because nobody declared it as an input. Bazel is working exactly as intended. Your sdist just doesn’t know it’s in a hermetic world.

The escape hatches

When people first hit this, there are two immediate reactions: the quick fix and the right fix.

--action_env=PATH: poking a hole

The fastest way to make it work:

bazel build //my_package:target --action_env=PATH

This tells Bazel: “for every build action, pass the host’s PATH environment variable into the sandbox.” Now the build script can find gcc because /usr/bin is on PATH again.

It works. It also defeats the purpose of hermeticity.

With --action_env=PATH, your build depends on whatever compiler happens to be installed on the host. Your laptop has gcc 13. CI has gcc 12. Your coworker is on macOS with Apple Clang. Same bazel build command, potentially different build outputs. You’ve re-introduced “works on my machine.”

Bazel will also bust caches more aggressively when environment variables change, since the env is now part of the action’s cache key. Different PATH = different cache key = rebuild from scratch.

There’s also use_default_shell_env = True, which you can set on individual build actions instead of globally. Same idea, smaller blast radius — only the actions that need the host environment get it. But it’s still a hole.

cc_toolchain: the hermetic compiler

The “right” answer in Bazel-land is a cc_toolchain. Instead of relying on whatever compiler the host has, you declare a specific compiler version as a Bazel dependency. Bazel downloads it, checksums it, and makes it available in the sandbox — just like it does with wheels.

Conceptually, it’s simple: “I need clang 17. Here’s where to download it for each platform. Use this for all C/C++ compilation.”

In practice, a cc_toolchain is one of the more complex things in Bazel. You’re telling the build system everything about the compiler: where the binaries are, where the headers are, what flags to use, how to link, which sysroot to target. It’s a lot of configuration.

cc_toolchain(
    name = "my_toolchain",
    toolchain_identifier = "clang-17",
    compiler_files = ":clang_compiler_files",
    linker_files = ":clang_linker_files",
    ar_files = ":clang_ar_files",
    dwp_files = ":empty",
    strip_files = ":empty",
    objcopy_files = ":empty",
    all_files = ":all_clang_files",
    # ... more configuration
)

You also need a cc_toolchain_config that specifies flags, include paths, linker behavior, and a bunch of other stuff that’s specific to your target platform. It’s the kind of thing where you either write 200 lines of Starlark or you use someone else’s.

toolchains_llvm: someone else’s 200 lines

toolchains_llvm is the ready-made solution that most teams reach for. It provides a hermetic LLVM/Clang toolchain that Bazel downloads and manages. Your MODULE.bazel gets something like:

llvm = use_extension("@toolchains_llvm//toolchain/extensions:llvm.bzl", "llvm")
llvm.toolchain(
    llvm_version = "17.0.6",
)
use_repo(llvm, "llvm_toolchain")
register_toolchains("@llvm_toolchain//:all")

Bazel downloads LLVM 17.0.6 for your platform, sets it up as the C/C++ toolchain, and now every build action that needs a compiler gets this specific version. Hermetic. Reproducible. The same compiler on every machine.

For compiling Python sdists with C extensions, this is usually all you need. The sdist’s build script calls gcc or cc, the cc_toolchain provides it via the sandbox, and the compilation proceeds with a known, consistent compiler.

The spectrum

Here’s where it gets nuanced. “Hermetic” isn’t binary. It’s a spectrum, and every team draws the line somewhere different.

Level 1: Fully hermetic

Everything — compiler, sysroot, headers, libraries — is downloaded and managed by Bazel. Nothing from the host leaks in. This is the ideal. Google operates here (mostly). It’s achievable for pure C/C++ and pure Python projects.

For Python with C extensions: set up toolchains_llvm, and sdists compile inside the sandbox with a hermetic compiler. Done. This is tractable for packages like fasttext or pyyaml with C extensions.

Level 2: --action_env leaks

The host compiler leaks into the sandbox via environment variables. Not ideal, but it works, and sometimes it’s the pragmatic choice when you have 3 sdists to compile and setting up a full toolchain isn’t worth the investment yet.

This is where we are right now for fasttext in our monorepo. We added --action_env=PATH with a TODO to set up a proper cc_toolchain. The TODO exists because we know it’s tech debt. The --action_env exists because we had a migration to ship.

Level 3: System-installed dependencies

Some dependencies are too heavy or too weird for Bazel to manage. The canonical example in our world: torch.

torch is a 2+ GB package with CUDA bindings compiled for specific GPU architectures. It’s system-installed inside our Docker images. It’s not in uv.lock on Linux. Bazel doesn’t know about it. We handle this with override mechanisms that tell Bazel “trust me, torch is available at runtime, don’t try to fetch it yourself.”

This is the least hermetic option. Your build graph has a hole in it — a dependency that Bazel can’t verify, can’t cache properly, can’t reproduce on a different machine. But sometimes the alternative is worse: downloading 2 GB of CUDA wheels in every CI job, or maintaining a parallel installation of torch that might drift from the Docker image’s version.

Where teams draw the line

Dependency type Typical approach
Pure Python packages Fully hermetic (wheels via @pypi//)
Python + small C extensions Hermetic with cc_toolchain
Python + CUDA/GPU deps System-provided, override mechanism
Heavy ML frameworks (torch, TF) System-provided or platform-specific wheels

The pattern: the heavier and more platform-specific the native code, the harder full hermeticity becomes. A cc_toolchain gets you a C compiler. It doesn’t get you CUDA 12.4, cuDNN 9.1, and a GPU driver — those are runtime dependencies that Bazel’s build-time hermeticity model wasn’t designed for.

The two cc_toolchain problems

Here’s the distinction that took me a while to internalize: “set up a cc_toolchain for compiling Python sdists” and “set up a cc_toolchain for compiling CUDA kernels” are wildly different problems wearing the same name.

Problem 1: Compiling sdists (tractable)

A package like fasttext needs a C++ compiler that can handle standard C++11/14/17 code. The build script runs g++ -O2 -shared -fPIC fasttext.cc -o fasttext.so. That’s it. A hermetic Clang from toolchains_llvm handles this just fine.

What you need:

What you don’t need:

This is a solved problem. Set up toolchains_llvm, register the toolchain, and your sdists compile hermetically. An afternoon of work, maybe a day if your sysroot needs fiddling.

Problem 2: CUDA kernels (hard)

CUDA compilation needs nvcc (NVIDIA’s compiler), CUDA headers, cuDNN, sometimes NCCL — and the versions need to match your GPU driver. A cc_toolchain can theoretically provide all of this, but now you’re managing NVIDIA’s entire toolchain as a Bazel dependency.

Some teams do this. rules_cuda exists. But it’s significantly more work than toolchains_llvm, and the version matrix (CUDA version × driver version × GPU architecture × Python version) is a combinatorial nightmare.

Most teams — including ours — draw the line here. System-installed CUDA in the Docker image. The cc_toolchain handles standard C++ compilation. The GPU stuff lives outside Bazel’s hermetic boundary.

It’s not pure. But purity is expensive, and the marginal benefit of hermetic CUDA compilation is low when your CUDA versions are already pinned in your Docker images.

What we actually did

Back to the real world. We hit this when migrating fasttext to Bazel in our monorepo. The build failed because the sdist needed g++ and the sandbox didn’t have it.

We chose the pragmatic path:

# In our Bazel configuration
# TODO(devex): Set up toolchains_llvm for hermetic C++ compilation
--action_env=PATH

One line. Builds work. The compiler leaks in from the host. It’s not hermetic. We know that. The TODO is real — we’ll set up toolchains_llvm when we have more sdists to compile, at which point the investment pays for itself.

For now, fasttext is the only sdist we’re compiling. One non-hermetic dependency among ~300 hermetic ones. The blast radius is small.

The torch situation is separate and harder — that’s the system-provided dependency problem where cc_toolchain doesn’t help, because the issue isn’t compilation, it’s resolution.

Two different problems. Two different solutions. The cc_toolchain is the answer to “how do I compile C extensions in Bazel’s sandbox.” It’s not the answer to “how do I use a 2 GB system-installed GPU framework in Bazel’s dependency graph.” Knowing which problem you’re actually facing saves you a lot of yak-shaving.

The takeaway

If you’re migrating Python to Bazel and everything works until one package doesn’t — check whether it’s a wheel or an sdist. If it’s an sdist with C extensions, you’ve just encountered the hermeticity boundary.

Your options, in order of increasing investment:

  1. Find a wheel. Check if a prebuilt wheel exists for your platform. Most of the time, it does.
  2. --action_env=PATH. Leak the host compiler. Fast, dirty, works. Put a TODO in the code.
  3. toolchains_llvm. Hermetic C/C++ compiler. A day of setup. The right answer if you have more than one or two sdists.
  4. Custom cc_toolchain. Full control. Write it yourself if toolchains_llvm doesn’t fit your needs. Prepare to learn more about Starlark than you planned.

For CUDA and GPU stuff — that’s a different post and a different headache. The cc_toolchain won’t save you there. System-installed dependencies, override mechanisms, and a healthy tolerance for impurity will.