Installing Python sources outside the site-packages

[igo95862]

I want to install the Python files outside the site-packages directories so that it is not Python version dependent.

My first idea was to modify the python.purelibdir and python.platlibdir options, however, it is impossible to programatically modify the options unless they are in subproject

I tried setting subdir to an absolute path but then it no longer compiles byte code.

Instead I found this trick:

py_installation.install_sources(
    source_files,
    subdir : fs.relative_to('/' / python_packages_dir, py_installation.get_path('purelib')),
)

You get both byte code compilation and it is installed outside the site-packages. However, it is kind of a hack.

Maybe there is a better solution?

[eli-schwartz]

Why do you want it to be independent of the python version?

Note that on e.g. Debian, the default installation location for install_sources is already "python3/dist-packages" which is independent of version

That's actually a bit of a problem since you can install the interpreter for python 3.8 and python 3.10 and then install a system package for a python 3.10 module, which is then importable in python 3.8 but is broken and fails to work (despite which, pip install thinks it's already installed and doesn't try to install a compatible copy).

You cannot actually use a module with multiple versions of python unless it's a pure library. For applications (such as bubbblejail??) you are limited to whichever version of python is in the script shebang -- and also whichever version of python has your python dependencies installed.

[igo95862]

Bytecode compilation occurs at import time, which also means when running the program as root in order to see the --help text.

I can manipulate the sys.dont_write_bytecode before importing any bubblejail's modules.

Anyway this is a minor detail. The important part is that I want to make bubblejail survive Python upgrades.

If meson does not have a good way of installing the Python sources outside the purelib I will continue using my bytecode compiler. I also plan on using a subproject with my Python namespaces library. It looks like I manipulate the subproject settings to modify the platlibdir of the subproject. Not sure how the byte code compilation will work.

An alternative solution I can think of is recording the build time purelib in the launch script and then checking if it is still in the sys.path before importing bubblejail modules. If it is no longer there that means the Python version was changed so insert it back and maybe emit a warning that user should rebuild their bubblejail. Not sure how vendoring the subprojects will work as I don't want to install the vendored subprojects inside the purelib or they might conflict. Maybe record an extra sys.path in the launch script with the vendored modules path.

[igo95862]

I've upgraded to Python 3.12 and bubblejail still works without rebuilding meaning my approach of installing in to a version independent directory is validated.

[eli-schwartz]

• that doesn't solve the bytecode issue
• it definitely doesn't validate the approach for other software, which means I'm hesitant to provide a mechanism other software might be tempted to use also
  • it's not guaranteed to work for bubblejail either when upgrading to 3.13, just because it happened to work for 3.12

[igo95862]

So far it worked 3.9->3.10->3.11->3.12.

Anyway it does not look like I can control subprojects purelib so I will instead use error() if the -Dpython.purelib does not match the version independent directory I want to use. AUR users will be able to do a git pull to update PKGBUILD with a correct -Dpython.purelib.

[igo95862]

I upgraded to Python 3.13 and everything still works.