Allow configuration of load limit for nix builds

[jonringer]

Is your feature request related to a problem? Please describe.
We have the ability to configure jobs and cores, however, I would like to propose a third value, load-limit. This will allow for a machine-specific value to be passed to make's -l argument. Previously, we were just using the NIX_BUILD_CORES, but this isn't satisfactory, as machines which are configured to run many jobs will have a low core count set to avoid over-subscribing the available cores, and the low core value being set globally for all builds severely limits the throughput.

Describe the solution you'd like
I would like nix to recieve a --load-limit argument, as well as allow for a load-limit nix.conf value. And in a nix build environment having NIX_LOAD_LIMIT be set to the corresponding value.

In my opinion, the default value for load-limit should be:

max(($load-limit or system_cores), $cores)

or in English, "Take the load-limit value from nix.conf or passed as argument, if not available, then determine the system cores impurely. Take the the cores nix.conf or argument value if it's larger"

Describe alternatives you've considered
None which are satisfactory.

Additional context
Nixpkgs PRs:

• treewide: drop -l$NIX_BUILD_CORES nixpkgs#192447
• stdenv: reintroduce limiting by system load nixpkgs#192799

[jonringer]

looks like #6855 is almost exactly this, with exception of setting the default value.

[Ericson2314]

I am hoping io_uring spawn (see https://lwn.net/Articles/908268/ or https://www.phoronix.com/news/Linux-LPC2022-io_uring_spawn) can open up a paradigm shift where instead of Make etc. pushing a set number of jobs to the OS, they simply queue up as many jobs as they want and the OS pulls from the queue as it sees fit to keep the system running at capacity.

Everything in is the -j family is, frankly, guessing and heuristics, and inefficient "two layer" scheduling where both lowers (the OS and Make are forced to do the best they can with missing information). This in start contrast decides the labor more rationally, with Make just responsible for providing work to be done (so tracking dependencies), and the OS doing all scheduling based on resources.

CC @joshtriplett

[grahamc]

or in English, "Take the load-limit value from nix.conf or passed as argument, if not available, then determine the system cores impurely. Take the the cores nix.conf or argument value if it's larger"

I don't think we should be ignoring the user-specified value even if cores is larger. I don't know why it may be configured that way, but I don't think it is our job to override it.

I also feel that load is a bad metric here. As the kernel says:

This file contains the magic bits required to compute the global loadavg
figure. Its a silly number but people think its important. We go through
great pains to make it work on big machines and tickless kernels.

For people who want to limit by load: what is it about load that is special? Or is it a proxy for CPU usage? Or IO activity? Why lean further into load when we could be limiting network/IO/CPU/memory/etc. via cgroups?

I'm having flashbacks to servers with a load of hundreds or higher, because a single nfs mount was failing IO, not because anything particularly interesting was happening.

[jonringer]

For people who want to limit by load: what is it about load that is special?

I guess for people like myself, where I'm doing a variety of workloads, I have my server "optimised" for doing mostly single threaded builds, but sometimes builds a pretty large package. In which case, I would like to have something which helps mitigate over-subscription of the cores. For example, if I already have the cores at 130% over subscribed, it doesn't help to add more work for it to churn. Even though load isn't the best heuristic, it at least can inform (at some basic level), where it's a potentially good idea to spawn another thread.

[LunNova]

For people who want to limit by load: what is it about load that is special? Or is it a proxy for CPU usage? Or IO activity? Why lean further into load when we could be limiting network/IO/CPU/memory/etc. via cgroups?

I'd be fine not limiting by load at all but can't because cores gets used as the load limit.

[ElvishJerricco]

Why lean further into load when we could be limiting network/IO/CPU/memory/etc. via cgroups?

Won't limiting memory usage with cgroups cause builds to fail, rather than use less memory by spawning fewer processes? Like if I'm building multiple derivations that fluctuate over time in how many cores they can take advantage of, it'd be good to allow any of them to be able to spawn lots of processes. But it's also important that they don't do so if it wouldn't help, because otherwise it will result in OOM failures. To my knowledge, the -l flag is the only way to have make behave well in this regard.

[Atemu]

For anyone following this issue, I want to direct your attention at the two(?) implementations of this issue:

#6855
#8105

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-build-ate-my-ram/35752/8