Flake copying performance regressed on macOS in 2.19

[cigrainger]

Describe the bug

I'm on nix-darwin and copying to the store can take several minutes. This first appears on 2.19. 2.18 is fast (a few seconds). E.g. nix run nixpkgs#scc takes >5 minutes.

Steps To Reproduce

Set nix.package to any version > 2.18 and try nix run nixpkgs#scc or nix flake update.

Expected behavior

The speed should be consistent after 2.18.

nix-env --version output

nix-env (Nix) 2.19.6

Additional context

I'm on nix-darwin.

Priorities

Add 👍 to issues you find important.

[emilazy]

It isn‘t 5 minutes bad for me, but I definitely also noticed a regression in flake copying performance on my Intel MacBook Pro after switching from 2.18 to 2.23 to pick up the Darwin sandbox fix. That‘s already one of the most painful parts of flakes, especially when working on Nixpkgs, so it‘s very unfortunate.

[Montmorency]

Also noticing this regression in flake performance and /nix/store interactions. [Current config is here] (https://github.com/Montmorency/nix-darwin).

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-07-17-nix-team-meeting-minutes-162/49255/1

[Mic92]

I cannot reproduce this:

$ nix --version   
nix (Nix) 2.23.3
$ time nix eval --store $HOME/test-store nixpkgs#nix.version
nix eval --store $HOME/test-store nixpkgs#nix.version  1.86s user 7.07s system 50% cpu 17.615 total 

[nix-shell:~/git/nixpkgs]$ sudo rm -rf $HOME/test-store/   
[nix-shell:~/git/nixpkgs]$ nix --version
nix (Nix) 2.18.5                       
[nix-shell:~/git/nixpkgs]$ time nix eval --store $HOME/test-store nixpkgs#nix.version        
"2.18.5"
real    0m22.181s
user    0m2.058s
sys     0m17.071s

In this case the old version was even slower, but my system is also more under load running some nix-builds at the same time.

[roberth]

We might be holding libgit2 wrong. I'm looking into packfile generation to reduce the vast amount of small I/O.