Fix niche cases when linking static libs

[xclaesse]

Case 1:

• Prog links to static lib A
• A link_whole to static lib B
• B link to static lib C
• Prog dependencies should be A and C but not B which is already included in A.

Case 2:

• Same as case 1, but with A being installed.
• To be useful, A must also include all objects from C that is not installed.
• Prog only need to link on A.

[xclaesse]

Case1 is a pure-C equivalent to this Rust issue: #11723.

[sdroege]

Does this also solve the the Rust issue if putting the dependencies explicitly into link_whole? And if so, would you mind integrating my test from that other PR here so we can be sure it keeps working? :)

[xclaesse]

@sdroege I'm undecided what to do with Rust yet. As usual, the more I dig the more I realize the problem is bigger than I thought.

For example s1 -> s2 -> r1 -> prog where s1 and s2 are C static lib and r1 is Rust staticlib, the final link of prog fails because it miss symbol from s1. What happens is rustc bundles s2 in r1 but not s1, because ninja backend in generate_rust_target() does not pass all deps to rustc.

using link_whole was a bad idea, which should forbid link_whole of rust targets completely because Meson implement that by passing object files which we don't have access to with rustc.

I leaning toward fixing this in ninja backend, generate_rust_target() does not use target.get_dependencies() correctly maybe... More thinking is needed :P

[xclaesse]

Failing CI here is another problematic edge case, the unit test does:

r = static_library('stuff', 'stuff.rs', rust_crate_type : 'staticlib')
l = static_library('clib', 'clib.c', link_with : r, install : true)

That cannot be supported currently, clib is installed so to be usable it needs to bundle stuff. Meson does that by adding all object files from stuff into clib archive but we can't do that with rustc because we don't have access to individual .o files, only the final .a archive.

Before this PR prog was incorrectly linking to stuff which should not be needed because clib is supposed to bundle it. This PR fix that, but since clib does not actually bundle stuff it now fail to link prog. But the thing is, that situation has always been broken because the installed clib does not bundle stuff which is not installed, that means installed clib was not usable which we never tested.

[sdroege]

That failing case should use link_whole : r in theory, and copying everything into the final .a could be done by first unpacking the libstuff.a or not?

[xclaesse]

One trick we might do is consider rust staticlib just like an external static library (which we cannot link_whole atm) and add code in Meson to extract archives like I did a while ago: #9218

[xclaesse]

That failing case should use link_whole : r in theory, and copying everything into the final .a could be done by first unpacking the libstuff.a or not?

In theory it should get promoted to link_whole implicitely, but Meson is not doing that because it knows it's broken: https://github.com/mesonbuild/meson/blob/master/mesonbuild/build.py#L1391. I think that should be turned into an hard error for now, it is not supported use-case.

[eli-schwartz]

That failing case should use link_whole : r in theory,

Installed static library: i
Uninstalled static rust library: rstuff
Uninstalled static C library: cstuff

If i does link_with: cstuff this is supposed to auto-promote to link_whole, which we do by recording ninja rules that build each .o file and another rule that runs ar on the .o.files and emits a libcstuff.a file.

The auto-promotion is not the problem here, we can and should auto-promote rust libraries too. The problem is the unpacking.

EDIT:

In theory it should get promoted to link_whole implicitely, but Meson is not doing that because it knows it's broken

Right, I remember that.

[xclaesse]

I added a PR that does the promotion, but abort later when it tries to extract object files. That has those advantages:

• Catch if user used link_whole in the first place.
• Avoid installing unusable static libs, better error out early.
• Group together with the CustomTarget case which has the same issue and potential fix.

[xclaesse]

Ok, CI fails because this PR is also fixing the same issue as described in #10699, but that PR had to be reverted because of #10745 for which we luckily added a unit test. That means we need to find a proper fix now. On the plus side, it means this PR should fix #11165 too.

[eli-schwartz]

It would be nice to reapply the original commits if we can get it to work...

Is there part of this PR that doesn't depend on that and which we can merge independently?

[xclaesse]

I think I have a proper fix, extract_all_objects() should include the object file for MSVC PCH. Unfortunately I lost access to my Windows machine (VPN seems down) so let's see if CI likes it.

[xclaesse]

Got CI to pass now.

[xclaesse]

@eli-schwartz any objection?

[jpakkane]

Anyone still have issues?