nix3-build should release memory acquired for evaluation before building the derivation

[poscat0x04]

Describe the bug
I'm not sure if this counts as a bug report or feature request.

Basically currently nix build won't release the pages allocated for evaluation while waiting for nix-daemon to build the final derivation. Since evaluation can take a lot of RAM (easily on the order of a few gigabytes when evaluation a system configuration), this can cause problems if the host is memory starved an the build task is demanding.

The same issue can be mitigated by first calling nix path-info --derivation to build a .drv file (which does the evaluation part) and then calling nix-store --realise on the resulting .drv file (which does the build part). I feel like this counts as major space leak.

One way I can think of to solve this issue is to spawn a subprocess specifically for evaluation, this way the memory can be easily reclaimed by simply killing the subprocess. Not sure how hard it would be to implement though.

Expected behavior

nix build should release unused RAM before calling nix-daemon

nix-env --version output
nix-env (Nix) 2.18.2

Priorities

Add 👍 to issues you find important.

[roberth]

Somewhat of a duplicate, but for the new CLI.
See #5200 for previous discussion including possible solutions and trade offs.

[poscat0x04]

Ugh sorry I couldn't find that issue :/ Should I close mine as a duplicate?

[poscat0x04]

Also my hypothesis is that while the heap allocation used during evaluation got free()d afterwards, no calls to sbrk are made to unmap the allocated memory pages, causing them to still appear as being used to the OS. I don't have any experience with c++ or Boehm GC though so this is just a guess.

[roberth]

You might be onto something. It doesn't seem eager to unmap when reading the code docs, although the default setting is unclear.
Could you try with GC_FORCE_UNMAP_ON_GCOLLECT=1 in the environment?
Source: https://github.com/ivmai/bdwgc/blob/8915b63d987f6034fa53d42c3a2d8a2b978f0614/include/gc/gc.h#L2150

Also if we want to run a GC before building, we should probably call GC_gcollect_and_unmap instead of GC_gcollect, to basically force the same effect, temporarily.

[poscat0x04]

Yes it made a significant difference. Setting the flag makes the memory usage drop from 3.5 gigs to ~800MB. Might want to turn this behavior on by default?

[roberth]

@poscat0x04 could you try #10879?

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-06-12-nix-team-meeting-minutes-152/46958/1