Description
Describe the bug
When using system packages from within a python application symbol lookup error
may happen.
This happens when the system package (like git
) is linked against a different version of libc
compared to the python version (from devenv) is linked against.
A specific case where this happens more often occurs when using a remote git repository as a requirement for your devenv-based python project. devenv calls pip install -r requirements.txt
, which will run git
. git
is usually not defined inside devenv, thus the required libc
is likely to be different from what is used in devenv. The python
wrapper within devenv runs python with LD_LIBRARY_PATH=$DEVENV_PROFILE/lib
, which will force ld to use libraries within $DEVENV_PROFILE/lib
. The libc
that resides in $DEVENV_PROFILE/lib
is a different version than what git
assumes to be ld-ed against.
This results in errors like the following:
Collecting git+https://github.com/pypa/sampleproject.git (from -r /nix/store/g41hqm03k2179v5pkyv889gyrgb370g0-requirements.txt (line 1))
Cloning https://github.com/pypa/sampleproject.git to /tmp/nix-shell.07Ag0j/nix-shell.tKP4lT/pip-req-build-jidkerj5
error: subprocess-exited-with-error
× git version did not run successfully.
│ exit code: 127
╰─> [2 lines of output]
git: error while loading shared libraries: __vdso_gettimeofday: invalid mode for dlopen(): Invalid argument
[end of output]
Potential solutions
- Adding
git
topackages
in devenv will resolve the problem with git, but on my system I got the next binary that resulted in the a similar error (gh
). Adding all personal tools is one way to get over this problem. - Avoid using other binaries within the python application by not using git.
- Resetting
LD_LIBRARY_PATH
upon calling other binaries from within the wrapped devenv python. - Make sure Python libraries are always linked against specific (working) versions of native libraries. This can be done using
patchelf
/autopatchelf
. In addition,LD_LIBRARY_PATH
should not be used so thatld
won't be forced to use different libraries. See PR python: add patchelf option #840 as a poc.
To reproduce
Follow the instructions from the readme:
https://gist.github.com/bobvanderlinden/f31299079b67ddcc1b2c6029cf6569f6
Version
devenv 1.0.2 (x86_64-linux)