Bazel’s Analysis Phase and In-Memory Cache

Bazel builds happen in three phases — loading, analysis, execution — and most people only think about execution. That’s where the compiler runs. That’s where you see the action count ticking up. But if you’ve ever watched Bazel chew through 30GB of RAM before a single byte gets compiled, the problem isn’t execution. It’s analysis.

I went down this rabbit hole after watching two BazelCon 2023 talks: Dude, Where’s My RAM? on Starlark memory problems, and Maggie Lou’s talk on sharing the in-memory cache. Together they paint a picture of what’s actually happening when Bazel “thinks” — and why that thinking can cost you all your memory.

The three phases, quickly

Every bazel build or bazel test command goes through:

1. Loading — Read and evaluate all the BUILD files, WORKSPACE/MODULE.bazel, and .bzl files. Starlark gets interpreted. Macros expand. The result is a package graph: every target, every rule, every glob(). This is pure Starlark evaluation — no analysis of what depends on what across packages yet.

2. Analysis — Take the loaded targets and the command-line configuration (platform, compilation mode, feature flags), and produce a graph of configured targets. This is where Bazel figures out what actions need to run, what providers each target exposes, and how information flows through the dependency graph. No actual building happens. It’s all planning.

3. Execution — Run the actions. Compile, link, package, test. This is the part that actually touches the filesystem. Actions are cached, parallelized, and optionally farmed out to remote workers.

Loading is usually fast. Execution is where wall-clock time goes. But analysis? Analysis is where memory goes.

What happens during analysis

Analysis is doing more work than people realize. For each target in your build, Bazel creates a configured target — the combination of a rule and a specific configuration. Same rule under different configs produces different configured targets. That’s how Bazel handles cross-compilation, feature flags, and platform transitions without rebuilding the world.

The work breaks down into:

Configured target creation. Every target gets resolved against the current configuration. A cc_library with --compilation_mode=opt is a different configured target than the same cc_library with --compilation_mode=dbg. They produce different actions, different outputs.

Provider propagation. Rules communicate through providers — structured data that flows up the dependency graph. When you write DefaultInfo(files = ...) or CcInfo(compilation_context = ...) in a rule implementation, you’re emitting providers. Downstream targets read those providers to figure out their own include paths, linker flags, runfiles, whatever.

This propagation is transitive. A cc_library deep in your graph emits CcInfo. Every target that depends on it — directly or transitively — merges that info into its own CcInfo. By the time you reach the top-level binary, you’ve accumulated the compilation context of the entire transitive closure. The data is deduplicated via Bazel’s depset structure (a DAG of sets that avoids copying), but the metadata still accumulates.

Aspect propagation. Aspects are rules that attach to existing targets and propagate along the dependency graph. Think of them as cross-cutting concerns — an aspect might collect all .proto files in your transitive deps, or gather IDE metadata from every cc_library. They create their own configured targets, their own providers, their own memory footprint. And they multiply: one aspect applied to a graph of 10,000 targets produces 10,000 additional configured targets.

Skyframe: the in-memory cache

All of this — every configured target, every provider, every action — lives in Skyframe, Bazel’s evaluation framework. Skyframe models the entire build as a DAG of SkyValue nodes. Each node has a key, a value, and edges to its dependencies.

I covered Skyframe’s caching role in Bazel: Cache Mechanisms Demystified. The short version: Skyframe caches node results in the Bazel server’s heap memory. This is why the Bazel server is a long-running daemon — the cache persists across commands.

What matters for analysis specifically:

When you run bazel build //foo twice, the second run checks: have any of the inputs to any Skyframe node changed? If not, the cached value is still good. Skip analysis entirely for that node. This is why incremental builds are fast — most of the graph doesn’t need re-analysis.

But here’s the catch: all of these nodes live in heap memory, simultaneously. Skyframe doesn’t evict nodes between commands by default. The graph grows monotonically within a server session. Build a few different targets, and the Skyframe graph accumulates every configured target from every build you’ve done since the server started.

Why analysis eats all your RAM

The “Dude, Where’s My RAM?” talk nails this. The problem isn’t any single node being too large — it’s the combinatorial explosion of nodes times the data each one carries.

Provider proliferation. Every rule implementation creates providers. Providers contain depsets. Depsets reference other depsets transitively. In a deep graph, the metadata about how to build can dwarf the build outputs themselves. A single top-level target might transitively merge providers from thousands of dependencies, and while depsets avoid copying, they still consume memory for the DAG structure, the flattened iteration caches, and the Starlark objects wrapping them.

Starlark object overhead. Starlark values are Java objects under the hood. Every string, list, dict, struct, and provider instance is a heap object with JVM overhead — typically 16-32 bytes of header per object, before you count the actual data. A rule that creates a struct(foo = "bar") allocates a Java object for the struct, a Java object for the string key, a Java object for the string value, and the wrapper objects tying them together. Multiply by thousands of targets and the overhead is real.

Configuration multiplication. Features like --fat_apk_cpu (build for multiple architectures) or configuration transitions (a target that changes the config for its dependencies) multiply the configured target count. One target with a 1-to-2 transition doubles its subtree. Chain a few transitions and the graph explodes.

Aspects multiply everything again. If you’re running IDE support (IntelliJ plugin), code coverage, or proto generation via aspects, each aspect creates a parallel shadow graph. The memory cost is roughly targets * configurations * aspects.

Put it all together: a monorepo with 50,000 targets, 2 configurations, and 2 active aspects could have 200,000 configured targets in Skyframe. Each carrying providers with transitive closure data. That’s your 30GB heap.

Profiling and diagnosis

So you’ve got a Bazel build eating memory. How do you figure out what’s responsible?

JSON profile

bazel build //your:target --profile=/tmp/bazel-profile.json.gz

Open the result in Chrome’s chrome://tracing or ui.perfetto.dev. You’ll see a timeline of every phase. The analysis phase shows up as a block — how wide it is tells you how long analysis took. But this doesn’t directly show memory.

Starlark CPU profile

bazel build //your:target --starlark_cpu_profile=/tmp/starlark.pprof

This produces a pprof-format CPU profile of Starlark evaluation. Open it with go tool pprof or upload to a pprof viewer. It shows which .bzl functions are burning CPU during loading and analysis — if a macro or rule implementation is doing something expensive (giant list comprehensions, repeated string concatenation), this is where you’ll catch it.

Build Event Protocol (BEP)

bazel build //your:target --build_event_json_file=/tmp/bep.json

BEP logs structured events for every target, action, and test. Tools like BuildBuddy ingest BEP and give you dashboards: analysis time per target, action counts, cache hit rates. If you’re diagnosing a slow analysis across a CI fleet, BEP + BuildBuddy is the way.

Heap dumps

The nuclear option. You can get a JVM heap dump of the Bazel server process:

# Find the Bazel server PID
jps | grep BazelServer

# Dump the heap
jmap -dump:format=b,file=/tmp/bazel-heap.hprof <PID>

Open in Eclipse MAT or VisualVM. Sort by retained size. You’ll see InMemoryMemoizingEvaluator (that’s Skyframe) dominating the heap. Drill into it to find which SkyValue types are hogging memory. This is how the “Dude, Where’s My RAM?” team traced their leak — they found that a custom Starlark rule was creating enormous string lists in a provider that got propagated to every transitive dependent.

Fixing it

You’ve found the problem. Now what?

Trim your providers. The single highest-impact fix. If your rule implementation packs data into providers that downstream targets don’t actually need, you’re paying memory for nothing. Audit what each provider carries and strip it to the minimum. Use depset for transitive data — never flatten a depset into a list in a provider, because that forces materialization of the entire transitive closure into memory.

Reduce configuration transitions. Every transition that changes the configuration for a target’s dependencies creates new configured targets. If you have a transition that only changes one flag, ask whether that flag actually affects the target’s subtree. Trimming unnecessary transitions with --experimental_trim_test_configuration and similar flags can significantly reduce the configured target count.

Watch your aspects. If you’re running aspects you don’t need (IDE aspects in CI, for instance), disable them. Each aspect multiplies the analysis work. In CI where no one needs IDE metadata, that’s pure waste.

Starlark hygiene. In rule implementations:

• Avoid creating large intermediate lists or dicts — they become heap objects
• Use depset instead of list for anything transitive
• Don’t stringify things unnecessarily — Starlark strings are immutable Java objects, and concatenation creates new allocations
• Minimize the data attached to OutputGroupInfo and custom providers

--experimental_analysis_cache_discard_analysis — this flag tells Bazel to discard analysis-phase data from Skyframe once execution starts. The tradeoff: you save memory during execution, but the next incremental build has to redo analysis from scratch instead of reusing cached configured targets. Useful for one-shot CI builds where you don’t care about incremental analysis. Terrible for interactive development where you’re iterating.

Tune the JVM. Bazel’s server runs on the JVM, and its garbage collector settings matter:

startup --host_jvm_args=-Xmx16g
startup --host_jvm_args=-XX:+UseG1GC

The default heap size is often too small for large monorepos, causing GC thrashing that makes analysis slower than it needs to be. But increasing the heap is treating the symptom — you should still figure out why analysis needs that much memory.

Sharing the in-memory cache

Maggie Lou’s BazelCon talk addresses a different angle: what if you could share the Skyframe graph between Bazel server instances?

The problem: every developer’s Bazel server builds its own Skyframe graph from scratch. In a monorepo with thousands of developers, that’s thousands of redundant analysis passes consuming RAM on thousands of machines. Remote execution shares the execution cache, but every client still does its own analysis.

The idea is to serialize Skyframe’s analysis results and share them — either through a central server or a shared filesystem. A developer starting a build could download the pre-computed Skyframe graph for their configuration instead of rebuilding it locally.

This is hard for a few reasons:

• Skyframe nodes reference each other via in-memory pointers. Serializing the graph means converting those pointers to some addressable format and back
• The graph is configuration-dependent. A shared cache needs to handle configuration mismatches (your flags vs. the cached graph’s flags)
• Invalidation is tricky — any change to a .bzl file, BUILD file, or input file could invalidate large portions of the graph

As of late 2024, this remains experimental. But the potential is significant: if you could skip analysis entirely for unchanged portions of the graph, first builds on a clean machine could feel like incremental builds.

The takeaway

Bazel’s analysis phase is where the build gets planned, and that planning has a real cost — measured in RAM, not wall-clock seconds. The Skyframe graph is the beating heart of Bazel’s incrementality, but it’s also the reason your Bazel server can end up holding 20-40GB of heap for a large monorepo.

The mental model that helps: analysis is building a second graph. Not the file graph (that’s loading). Not the action graph (that’s execution). The configured target graph — a combinatorial product of your targets, your configuration, and your aspects. Every node in that graph lives in memory, carries transitive provider data, and never gets evicted until you kill the server.

When that graph gets too big, profile it, trim it, and consider whether you’re paying for analysis data you don’t actually need. Your RAM will thank you.