Introduction
We have briefly covered different types of caches in Bazel, but haven’t really dived deep into the workflow.
Confusing Concepts
Example
[Action Execution]
|-- Outputs: Files produced (foo.o) -> Stored in CAS
+-- Action Result: Metadata about action -> Stored in AC
|-- Inputs: [foo.cpp, foo.h]
|-- Environment: {CXX=g++, FLAGS=-c}
+-- Outputs: [foo.o's hash in CAS]
External Action Cache and Execution
Remote Cache Flow
1. Check if the action result exists in the local output base action cache.
+- If not, check the remote AC (ActionCache.GetActionResult).
2. If the action result exists:
|-- Check if the outputs are in the local output base.
| +- If outputs are present, the action is done.
+-- Otherwise, fetch outputs from the remote CAS (ByteStream.Read).
+- If outputs are retrieved, the action is done.
3. If the action result doesn't exist:
|-- Execute the action locally.
+-- On successful completion:
|-- Upload outputs to the CAS (ByteStream.Write).
+-- Update the action result in the AC (ActionCache.UpdateActionResult).
Remote Execution Flow
1. Check if the action result exists in the local output base action cache.
+- If not, check the remote AC (ActionCache.GetActionResult).
2. If the action result exists:
|-- Check if the outputs are in the local output base.
| +- If outputs are present, the action is done.
+-- Otherwise, fetch outputs from the remote CAS (ByteStream.Read).
+- If outputs are retrieved, the action is done.
3. If the action result doesn't exist:
|-- Check if inputs for the action need to be uploaded (CAS.FindMissingBlobs).
| +- Upload missing inputs to the remote CAS (ByteStream.Write).
|-- Execute the action remotely (Execution.Execute).
+-- On successful completion:
+- Download outputs from the remote CAS (ByteStream.Read).
Intertwined
- Usually on same server
- If you have remote execution set up, you already benefit from remote caching. Remote Execution implicitly includes Remote Caching by managing inputs, outputs, and metadata in the CAS and AC.
- However, remote caching can exist without remote execution (e.g., for local builds).
Network and Disk: What Gets Downloaded and Uploaded
No remote execution, no –remote_cache, with –disk_cache
No remote execution, with –remote_cache, with –disk_cache
With remote execution, with –remote_cache, with –disk_cache
Comparison
End-to-End Flow with Network Interactions
[Action Execution Process]
1. Query AC:
|-- ActionCache.GetActionResult -> Remote AC
+-- If result found:
|-- Check local output base for outputs.
+-- If missing, ByteStream.Read -> Remote CAS (fetch outputs).
2. If action result not found:
|-- Query CAS for missing inputs:
| CAS.FindMissingBlobs -> Remote CAS
|-- Upload missing inputs:
| ByteStream.Write -> Remote CAS
+-- Execute action remotely:
|-- Fetch inputs: ByteStream.Read -> Remote CAS
+-- Run the action.
3. After execution:
|-- Upload outputs:
| ByteStream.Write -> Remote CAS
+-- Update AC with action result:
ActionCache.UpdateActionResult -> Remote AC
Metrics to Monitor
Remote Cache
- Cache hit rate: Percentage of reused actions/artifacts.
- Upload/download times: Average time to interact with the remote cache.
Remote Execution
- Queue time: Time actions spend waiting for remote resources.
- Execution time: Time spent executing actions on remote servers.
- Network usage: Amount of data transferred to/from CAS.
Tools
Bazel Profile
bazel build --profile=build.profile
Bazel BEP (Build Event Protocol)
bazel build --build_event_text_file=events.json
Cross-Platform Cache/Execution
Bazel Implementation Under the Hood
- action get
- spawn
- remote