Bazel Tests Passing, but None Running

Here’s a fun one. We have a Python monorepo with about a dozen packages migrated to Bazel. Every py_test target — across every package — reports PASSED. Green across the board.

Zero tests were actually executing.

How We Found It

I was migrating mai_config to Bazel and wanted to verify that the 5 Bazel test targets covered the same 130 tests we run with uv run pytest. So I tried --test_output=streamed to see the individual test results.

Nothing. Empty output. Just “PASSED” with no pytest summary, no test counts, no nothing.

Ran the test file directly to check: python test_serialization.py. Exit code 0, no output. That’s when it clicked.

The Mechanism

aspect_rules_py’s py_test rule runs your test file like this:

python test_serialization.py

That’s it. Not pytest test_serialization.py. Just… Python executing the file.

And what does a typical test file look like?

import pytest
from mai_config import registry

def test_load_default():
    config = registry.load("default")
    assert config is not None

def test_invalid_key():
    with pytest.raises(KeyError):
        registry.load("nonexistent")

No if __name__ == "__main__" block. No pytest.main(). Python imports the file, defines some functions, and exits. Exit code 0. Bazel sees 0 and reports PASSED.

Every. Single. Target.

It Gets Worse

This wasn’t a regression. We checked the commit history — tests were never running in Bazel. Not under aspect_rules_py, and not under rules_python before the migration six days earlier.

To prove it, I created a test file at the old commit with assert False in every function. Bazel: PASSED.

The previous defs.bzl wrapper even auto-added @pypi//pytest as a dependency — someone knew pytest needed to be there. They just missed the part where you actually invoke it.

The Fix

Turns out aspect_rules_py already has the solution built in. There’s a pytest_main parameter on py_test that nobody was using:

py_test(
    name = "test_serialization",
    srcs = ["test_serialization.py"],
    pytest_main = True,  # <-- this is all you need
    deps = [...],
)

When pytest_main = True, the rule generates a pytest entry point script automatically. Proper test discovery, proper reporting, even test sharding support.

We wrapped it in a thin macro so every package gets it by default:

load("@aspect_rules_py//py:defs.bzl", _py_test = "py_test")

def py_pytest_test(name, deps = [], args = [], **kwargs):
    if "@pypi//pytest" not in deps:
        deps = deps + ["@pypi//pytest"]
    args = ["--ignore-glob=*pytest_main*"] + args
    _py_test(name = name, pytest_main = True, deps = deps, args = args, **kwargs)

Find-and-replace py_test → py_pytest_test across all BUILD files. Done. 130 tests now actually run.

The Takeaway

If you’re using py_test with pytest-style test files in Bazel — go check. Right now. Run a target with --test_output=streamed and see if you get actual pytest output. If you just see empty space between Bazel’s markers, your tests aren’t running.