Gazelle: Python Manifest and modules_mapping

2026/02/18

BUILDbazeldependencies
Series: Gazelle (6 posts)
  1. Gazelle: How import yaml Becomes @pypi//pyyaml
  2. Gazelle: Python Manifest and modules_mapping
  3. Gazelle: ASPECT_GAZELLE_CACHE Internals
  4. Gazelle: Python in a Monorepo, the Invisible Machinery
  5. Gazelle: When It Can’t Help, Hand-Writing BUILD Files
  6. Gazelle: Reading the Source Code

A coworker added _pytest: pytest to our gazelle_python.yaml by hand. The file header literally says GENERATED FILE - DO NOT EDIT. So — was this a hack? Does the auto-generator just not work?

Turns out, even if it did work, it still wouldn’t have caught _pytest. And the reason why is the same across both rules_python and aspect_rules_py.

What is modules_mapping?

When Gazelle sees import cv2 in your Python code, it needs to figure out which pip package provides that. The answer is opencv-python-headless — but there’s no way to know that from the import name alone.

That’s what modules_mapping solves. It’s a lookup table:

modules_mapping:
  cv2: opencv_python_headless
  PIL: pillow
  _pytest: pytest
  yaml: pyyaml

Gazelle consults this mapping to generate the correct deps in your BUILD.bazel files. Without it, import cv2 would produce a broken build.

How the auto-generator works

Both rules_python and aspect_rules_py have their own version of the manifest generator. The approach is basically the same:

  1. Collect all .whl files from your pip dependencies
  2. Open each wheel (it’s just a zip)
  3. Read .dist-info/METADATA to get the package name
  4. Scan all .py files inside to discover importable module names
  5. Build a {module_name: package_name} mapping
  6. Write the YAML

The rules_python version (gazelle/modules_mapping/def.bzl) does it in two phases — per-wheel generation then merge. The aspect_rules_py version (uv/private/manifest/generate.py) does it in one pass. Same idea.

So this isn’t a rules-specific thing. The concept — scan wheels, extract module names, build a mapping — is shared. If you switch from rules_python to aspect_rules_py (or vice versa), you’ll still have a modules_mapping and it’ll still work the same way.

The underscore filter

Here’s where it gets interesting. Both implementations filter out modules that start with underscores.

rules_python uses a regex: ^_|(\\._)+ — skip anything starting with _ or containing ._ segments.

aspect_rules_py does the same thing in Python: it filters path segments starting with _ (except __init__), and if any segment was filtered, the whole path is excluded.

So when the generator scans the pytest wheel and finds _pytest/monkeypatch.py, it thinks: “underscore prefix, that’s a private module, skip it.”

Which means _pytest: pytest will never appear in the auto-generated mapping. You have to add it manually.

Why filter underscores?

Convention. In Python, _-prefixed modules are “private” — internal implementation details that you shouldn’t import directly. The generator assumes you don’t want import _pytest.monkeypatch in your code, so it doesn’t bother mapping it.

Except sometimes you do. Test fixtures, internal tools, monkey-patching utilities — there are legitimate reasons to reach into private modules. And when you do, Gazelle has no idea where to find them.

The practical workflow

In our repo, the auto-generator is actually broken — blocked by an upstream bug in aspect_rules_py. So the whole file is manually maintained right now.

But even when that bug gets fixed, _pytest would still need a manual entry. The auto-generator will handle the 99% case — all the cv2: opencv_python_headless and yaml: pyyaml mappings. The underscore-prefixed imports are the 1% you’ll always have to add yourself.

Steps when you hit this:

  1. Gazelle fails to resolve an import (or generates wrong deps)
  2. Figure out which pip package provides it — usually obvious, sometimes not
  3. Add the mapping to gazelle_python.yaml under modules_mapping: in alphabetical order
  4. Run bazel run //:gazelle to regenerate BUILD files

That’s it. One line in a YAML file.

The bigger picture

This is a good example of Bazel’s general trade-off. It gives you hermeticity and correctness, but the cost is that implicit things become explicit. Python’s import system is famously loose — you can import anything from anywhere at runtime. Bazel needs a strict dependency graph. The modules_mapping is the bridge between those two worlds.

And when the bridge has a gap — like underscore-prefixed modules — you fill it by hand. Not a hack. Just the cost of doing business.