Memo in GitHub Actions Skipped a Rust Release Build and Caught Runners With Different Compilers

Most CI jobs build the same thing again and again. A docs fix or a typo in a README triggers the whole pipeline, and the release build runs as if the code had changed. This case study puts BXM Memo in front of one such step, a release build of the bxm-canon binary, and keeps Memo’s cache between jobs with actions/cache.

The workflow is in the repository as .github/workflows/case-study-memo.yml. Five jobs run one after another, each on a fresh GitHub-hosted runner. plain builds without Memo for the baseline. cold builds through Memo with an empty cache, warm changes nothing, docs edits the README, and source edits a Rust source file. The inputs are canon/src/**, Cargo.toml and Cargo.lock, and the salt is rustc --version, so a different compiler means a different result. We ran the whole workflow twice.

Two GitHub Actions runs of five jobs each: plain builds took 17 and 23 seconds, cache hits took under two tenths of a second, a source edit missed and named the file, and two runners with a different Rust version missed on the salt

Job	Run 1	Run 2
plain: `cargo build --release`, no Memo	16.9 s	23.0 s
cold: through Memo, empty cache	miss, 13.5 s	miss, 7.9 s
warm: nothing changed	hit, 0.17 s	miss: the salt changed, 10.9 s
docs: README edited	hit, 0.16 s	hit, 0.17 s
source: `main.rs` edited	miss: the salt changed, input canon/src/main.rs changed, 8.6 s	miss: the salt changed, input canon/src/main.rs changed, 18.6 s

A hit took under two tenths of a second for the whole step. Memo put the binary back in about a millisecond (670 µs, 798 µs and 1.0 ms in the three hits), and actions/cache took 0.32 to 0.35 s to restore the cache folder before that. The folder held 448 KB. The hit jobs finished in 5 and 6 seconds from start to end, against 22 and 26 for the plain build. The README edit was still a hit, since the README isn’t an input. The source edit missed and said which file.

Builds on these runners vary a lot. The cold build through Memo ran faster than the plain one in both runs, which is down to the runners, not to Memo. The fair comparison is a hit against any of the build times.

The surprise: two Rust versions

Run 2’s warm job should have been a hit, and Memo said “the salt changed”. The workflow prints the toolchain, so we could check. The cold job’s runner had rustc 1.99.1 on image 20261018. The warm job’s runner had rustc 1.99.0 on image 20261012. GitHub rolls out new runner images gradually, so for a while jobs in one workflow can land on different images. Run 1’s source job missed on the salt too. That run didn’t print the toolchain yet, but the cause is almost certainly the same.

With the compiler in the salt, Memo rebuilt rather than hand back a binary from a different compiler. Leave the salt out and those jobs would have been hits with the other compiler’s output. Whether that matters depends on the step, but for a release binary we’d rather rebuild. It cost one build each time.

How to set it up

Download bxm-memo from the latest release, point BXM_MEMO_CACHE at a folder in the workspace, and wrap the step:

bxm-memo run -n canon-release \
  -i 'canon/src/**' -i canon/Cargo.toml -i canon/Cargo.lock \
  -o canon/target/release/bxm-canon --salt "$(rustc --version)" \
  -- cargo build --release --manifest-path canon/Cargo.toml --bin bxm-canon

Save the folder with actions/cache after the step and restore it before. Memo’s cache is small, because it holds the step’s outputs and not the whole target folder.

The recorded results of both runs are in test/results/case-studies/04-github-actions.txt, with links to the runs. More examples are on the use cases page.