llvmPackages.bolt: add experimental bolt hook

[RossComputerGuy]

Things done

Experimental concept, this adds a hook which calls bolt on the installed binaries. The idea is this will optimize the installed binaries. We'll test with just hello however all packages can be tried by enabling useBolt when specifying the cross platform.

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[alyssais]

Do we really need this? I'm not sure every possible thing you can set on a platform needs its own package set, and I especially don't think we need Hydra to build every single-change stdenv.

[RossComputerGuy]

We can drop it before merging but for the purposes of testing this PR, I added it in.

[pwaller]

I think this sort of thing is useful to have. For what it's worth. I don't think its existence needs to promise that it will work, much the same as for pkgsStatic and friends as well. It is quite powerful to have a simple name for this thing to reach for from the shell.

[alyssais]

The problem is that these things also make it way too easy to use them within Nixpkgs, and then we end up with a layering violation where a single package is going through a whole distinct package set out of convenience.

But that's a general problem that needs to be solved, by making these inaccessible from within Nixpkgs while still being a convenience layer on top, so if this is really useful it can stay. I think there's no justification for building this set on Hydra, though.

[RossComputerGuy]

The problem is that these things also make it way too easy to use them within Nixpkgs, and then we end up with a layering violation where a single package is going through a whole distinct package set out of convenience.

We can add in a config option similar to the aliases one which enables the subsets. Then have it disabled in Hydra / CI.

But that's a general problem that needs to be solved, by making these inaccessible from within Nixpkgs while still being a convenience layer on top, so if this is really useful it can stay. I think there's no justification for building this set on Hydra, though.

I kinda agree but its also good to ensure things work.

[alyssais]

That relies on somebody actively monitoring that on Hydra indefinitely, which has not historically happened for this sort of thing. Hydra isn't really a great system for "ensuring things work" due to the lack of notifications.

[alyssais]

The problem is that these things also make it way too easy to use them within Nixpkgs, and then we end up with a layering violation where a single package is going through a whole distinct package set out of convenience.

We can add in a config option similar to the aliases one which enables the subsets. Then have it disabled in Hydra / CI.

Oh that's not a bad idea!

[RossComputerGuy]

hello with gcc - 77kb binary size:

0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata 1584maxresident)k
4520inputs+0outputs (12major+81minor)pagefaults 0swaps

hello with llvm - 49kb binary size:

0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata 2640maxresident)k
0inputs+0outputs (0major+95minor)pagefaults 0swaps

hello with llvm + bolt - 4MB binary size:

0.00user 0.00system 0:00.00elapsed 16%CPU (0avgtext+0avgdata 1568maxresident)k
5192inputs+0outputs (19major+91minor)pagefaults 0swaps

[pwaller]

So far as I understand, for bolt to be able to meaningfully optimize it needs profile data; do you have any plans to have a mechanism to provide that?

[RossComputerGuy]

Yes, I just am not sure how to integrate that yet.

[pwaller]

One problem with profile data is that it has a habit of being non-determinstic. For example, even if you collect profile data through a determistic mechanism, and run it on a workload which is functionally determinstic (such as clang), what controls the bits of the profile is the control flow repeatability inside the workload. See for example the discussion of applying PGO to clang. https://discourse.llvm.org/t/pgo-profile-reproducibility/82861/6.

It is possible to get around this problem to produce a repeatable profile by collecting multiple profiles and figuring out what is common between them (and then making a script that eliminates the non-common parts from the profile), but it is likely quite a bit of effort to make it happen. In particular if things like 'what inode does this include directory have' factor into the computation; since that is something that will vary from one machine to another.