Parallel decompression when substituting

[andrewhamon]

When copying to a remote cache, there is an option to enable parallel-compression. This is great, and can speed up compression significantly!

However, as far as I can tell, there is no way to enable parallel decompression when substituting from cache. A naive perusal of the source code further hints that parallelization is only supported on compression and not decompression.

Decompression speed can easily be a bottleneck, especially with xz which is pretty intensive even on its default settings.

I have also tested empirically that there is no parallelization for decompression:

# snag a modern version of nix
$ nix build nixpkgs#nixVersions.nix_2_25

# snag a large store path
$ result/bin/nix copy --to local /nix/store/flqqrbm3pv9mcxf7xg1j6ssmfwxwrsdx-clang-18.1.8-lib

# compress it into a binary store
$ time result/bin/nix copy --to "file:///$(pwd)/cache-xz?compression=xz&parallel-compression=true" /nix/store/flqqrbm3pv9mcxf7xg1j6ssmfwxwrsdx-clang-18.1.8-lib
result/bin/nix copy --to    663.77s user 4.21s system 1074% cpu 1:02.19 total

# 1074% cpu - definitely parallel compression!

# copy to an uncompressed binary cache
$ time result/bin/nix copy --from "file:///$(pwd)/cache-xz?compression=xz&parallel-compression=true" --to "file:///$(pwd)/cache-none?compression=none&parallel-compression=true" /nix/store/flqqrbm3pv9mcxf7xg1j6ssmfwxwrsdx-clang-18.1.8-lib
result/bin/nix copy --from  --to    13.46s user 1.69s system 100% cpu 15.038 total

# 100% cpu - no parallel decompression

Checklist

• checked latest Nix manual (source)
• checked open feature issues and pull requests for possible duplicates

---

Add 👍 to issues you find important.