How a Monorepo Does Integration Tests in Bazel

Here’s a question that sounds simple until you actually try to answer it: how do you run an integration test that needs a real Redis instance, inside a hermetic build system that hates side effects?

Because that’s the tension. Bazel wants determinism. Integration tests want real databases. These two goals are fundamentally at odds — and the way our monorepo solves it is one of the cleanest architectural patterns I’ve seen in a build system.

The Core Idea

Every integration test is actually two Bazel targets.

That’s it. That’s the trick. But the implications are surprisingly deep.

Target one is your normal test — go_test, cc_test, py_test, whatever. It’s tagged "manual" so bazel test //... will never accidentally run it. This test knows nothing about how services get started. It just expects environment variables like REDIS_PORT to exist.

Target two is a service_test wrapper from @rules_itest. It boots real dependencies, waits for health checks to pass, injects ports via env vars, then runs the inner test.

def go_integration_test(name, services, tags = [], **kwargs):
    go_test(
        name = name + "_without_services",
        tags = ["manual"] + tags,
        **kwargs
    )
    service_test(
        name = name,
        test = name + "_without_services",
        services = services,
        tags = tags,
    )

So if you define go_integration_test(name = "engines_test", ...), you get:

• engines_test_without_services — the naked test, hidden from normal runs
• engines_test — the real deal, with services booted

The test code itself? Zero knowledge of service lifecycle. It just says “give me Redis at this port” and trusts the build system to make it happen.

Why Two Targets?

You might think this is over-engineered. Why not just start Redis inside the test binary?

A few reasons.

First, separation of concerns. Your test code tests your logic. The BUILD file describes your infrastructure. These are different problems with different owners. The person writing the cache invalidation test shouldn’t need to know how to compile Redis from source.

Second, the "manual" tag is a safety net. Without it, someone running bazel test //... would execute a test that expects a Redis port in an env var that doesn’t exist. Instant flaky failure. The tag makes naked tests invisible to broad queries.

Third — and this is the sneaky one — it scales across languages with almost no modification. The Go macro, the Python macro, the C++ macro — they’re structurally identical. Different inner test rule, same wrapper pattern.

Services Are Just Bazel Targets

This is where it gets interesting. The services your tests depend on aren’t Docker containers. They’re Bazel targets.

Redis? Compiled from C source via Bazel. FoundationDB? Same deal. Full hermeticity — no docker pull, no container runtime, no “works on my machine” because the CI box has a different Redis version.

Each service is defined as an itest_service with a few key properties:

itest_service(
    name = "redis",
    exe = ":redis_server_bin",
    args = ["--port", "$${PORT}"],
    autoassign_port = True,
    health_check = "redis_health_check",
    health_check_timeout = "30s",
)

The autoassign_port = True is doing heavy lifting here. When you run tests in parallel — and in a monorepo, you run a lot of tests in parallel — you can’t have two Redis instances fighting over port 6379. The framework picks a free port, interpolates it into the args via $${PORT}, and passes it to the test as an env var.

No port conflicts. No flaky test because “something else was using that port.” Just works.

Service Dependency DAGs

Some services need other services. FoundationDB is the classic example — you can’t init the database until the server is actually running. So the service definitions form a dependency graph:

1. FDB server starts, health check passes
2. FDB init task runs (creates database, sets config)
3. Only then does the test execute

This is expressed through itest_service_group and deps. The framework walks the DAG, boots things in order, waits for health at each level.

For more complex setups, there are macros that compose multiple services:

• fdb_service() — bundles FDB server + init task + service group into one target
• redis_ensemble() — spins up a multi-shard Redis cluster with port aliases for each shard

From the test’s perspective, it’s still just “Redis is at these ports.” All the orchestration complexity lives in BUILD files.

Hygiene Tests

Here’s a pattern I hadn’t seen before. Some tests don’t test your code at all — they test whether the service itself boots correctly.

service_test(
    name = "turtle_service_for_test_hygiene_test",
    flaky = True,
    services = [":turtle_service_for_test"],
    test = "@rules_itest//:exit0",
)

See that test = "@rules_itest//:exit0"? The test binary immediately exits with code 0. It doesn’t check anything. The entire point is to verify that the service starts up and passes its health check.

If the hygiene test fails, you know the service definition is broken — before any real test ever touches it.

The flaky = True is pragmatic. Service startup is inherently timing-sensitive. Sometimes the health check takes a beat longer. Marking it flaky lets Bazel retry without someone filing a bug that says “this test is nondeterministic.”

The Gotchas

No pattern is perfect. Here’s where this one gets messy.

Tag inconsistency. Some teams use "manual" for the inner test, others use "internal". They mean the same thing functionally — “don’t run this in broad queries” — but the inconsistency makes it harder to reason about the codebase. Classic monorepo drift.

Python is backwards. The Go and C++ macros give the “good name” to the service_test wrapper and suffix the inner test with _without_services. Python does the opposite — the bare name goes to the inner test, and the wrapper gets _with_services. Why? I genuinely don’t know. Convention is a hell of a drug.

No default timeouts. The macros don’t set timeout overrides. This means every team picks their own, and some teams pick nothing, which means Bazel’s default timeout applies. For a test that boots FoundationDB, the default is… optimistic.

Remote execution needs a tag. If your CI uses remote executors, you need no-remote-exec on service tests. Makes sense — you can’t boot a local Redis on a remote machine — but it’s easy to forget. And when you do forget, the error message is not helpful.

Shared TMPDIR. By default, rules_itest gives each service its own temp directory. But some services set hygienic = False, which means they share TMPDIR with the test. This is intentional — some services need to write files the test reads — but it’s a landmine if you don’t expect it.

Why This Pattern Works

I keep coming back to the separation of concerns.

Test code is blissfully ignorant. It reads an env var, connects to a port, runs assertions. It could be talking to a real Redis or a Redis running on the moon — doesn’t matter. The test doesn’t know and doesn’t care.

BUILD files own the orchestration. They say “boot these services, in this order, wait for these health checks, then run this test.” All the complexity is in one place, expressed declaratively.

And because Bazel is Bazel, everything is hermetic. No Docker daemon required. No “did you remember to docker-compose up?” No version drift between local and CI. The Redis your test uses is the Redis that Bazel built from source, right there in the action graph.

The two-target pattern is a small idea. "manual" tag on an inner test, service_test wrapper on the outside. But it composes beautifully — same pattern across Go, C++, Python. Same pattern whether you need one Redis or a five-node FoundationDB cluster. Same pattern whether it’s a 100ms unit test or a 30-second integration behemoth.

The build system handles the messy parts so the test code doesn’t have to.

That’s good infra.