Additional packages in buildroot when building SRPM

[FrostyX]

From Matrix

pgnd:
with a simple test.spec, https://is.gd/vRBpwO, and jq added to pkg build chroot,
i still get the ""stderr: sh: line 1: jq: command not found"".
stumped. or missing the obvious ...
if anyone can spare a look --> https://download.copr.fedorainfracloud.org/results/pgfed/vulkan-wsi-layer-pgnd/srpm-builds/08653401/builder-live.log.gz

FrostyX:
thats because the failure is when building a SRPM and at that point we are
probably not installing the additional packages into the buildroot. IIRC we do
so only for custom method. Thank you for this minimal reproducer, it was
helpful. At the moment I am not sure what should we do about it though

The specfile is here: https://pagure.io/pgnd/vulkan-wsi-layer-pgnd/blob/main/f/vulkan-wsi-layer.spec
The package does something like this at the very top:

%global _test      %( curl -s "https://filesamples.com/samples/code/json/sample1.json" | jq .fruit )

which looks insane to me but it is obviously meant as a reproducer. I am not sure what the real use-case is.

However, this seems like an easyfix? We can just do this for SRPM builds as well

copr/frontend/coprs_frontend/coprs/views/backend_ns/backend_general.py

Line 190 in 8695206

build_record["buildroot_pkgs"] = build_config.get("additional_packages")

[pgnd]

i put the reproducer here:

https://pastebin.com/raw/G8ZhHT3W

, so it'll persist awhile ...

[xsuchy]

While the implementation may be easy (Frostyx claims that it can be oneliner), there is an easy workaround (use a custom method). Therefore, I will give it low priority.

[pgnd]

on the COPR site @

https://copr.fedorainfracloud.org/coprs/.../edit_chroot/fedora-41-x86_64/

it currently explicitly states:

"Edit chroot 'fedora-41-x86_64'
Packages: [ ]
You can add additional packages to the minimal buildroot of this chroot.
These packages will be always present before the build starts.
"

NOTE, "always".

it seems to me that implementing the fix is simpler, but --
-- if the "one-liner" fix can't be prioritized, can the misleading on-site documentation at least be timely ammended?

in order to save others' some time/effort figuring out that there IS a problem, that a workaround exists, and then determining how to do a 'Custom' build, swithcing to it and testing it?

[praiskup]

When I try reproduce locally mock --buildsrpm your.spec, I get:

...
Building for target x86_64
sh: line 1: jq: command not found
setting SOURCE_DATE_EPOCH=1739491200
Wrote: /builddir/build/SRPMS/vulkan-wsi-layer-1.0.0-0.20250218_072456.pgnd.fc41.src.rpm
Finish: rpmbuild -bs
Finish: buildsrpm
INFO: Done(vulkan-wsi-layer.spec) Config(fedora-41-x86_64) 0 minutes 1 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-41-x86_64/result
Finish: run

IOW, it builds the source RPM just fine. Yes, I see the shell's complaint that jq is not installed, but that's just a "warning". The original Copr build failure has this $(jq) thingy probably in a much worse place, so it makes it fatal.

This pretty much resonates with this topic: release-engineering/dist-git#72 . From the infra perspective, we don't actually need to produce source RPM (and parse spec files) to successfully import the spec into our dist-git instance, and then start RPM builds (!= SRPM build).

Practically, though, this is a spec file bug. Spec files should stay as self-standing as possible, generating SRPM shouldn't be a big (or even insecure, like touching the interned by macro) deal.

@xsuchy

While the implementation may be easy (Frostyx claims that it can be oneliner)

Hm, I'm not convinced, because #534 - I don't think we run rpkg-util in mock?

@pgnd

if the "one-liner" fix can't be prioritized, can the misleading on-site documentation at least be timely ammended?

Would you mind helping us with the documentation? I can think of two topics:

• While spec file macros are Turing Complete, and users may do very weird things there, they shouldn't. This is most of the time insecure; in Copr, you are fine, because we give you disposable workers. But any other user (human) that is parsing your spec file is doing so all the time (even when, e.g., generating srpm) and they are now silently forced to download stuff from the internet via Curl. And it can easily twist into much worse accidents...

• there are differences in source methods; most of them are run on host, some in a Mock chroot. We don't want to let our users to "break the machines" easily by just allowing them to install packages there. Custom method is run in chroot, and we allow installation there. Make srpm has root in chroot. Yes, if user breaks the machine, it's OK - we detect it, and delete - but we try to recyclate the machines as much as possible because they are not cheap (e.g., we re-use builder machines for multiple builds in the same project).

[pgnd]

"silently forced to download stuff" ?
noone's forced to use a COPR spec. specific notice to NOT use the spec, and that it's for personal use only, should also provide a ... rather strong hint.

COPR is used here to fill local needs unmet by Fedora project's official packaging, &/or to provide basis for test/devel.

what 'should' be in a COPR spec is an opinion -- until it becomes a 'must'.
if COPR's now imposing functional 'musts', for one or more project-justified reasons, that's useful to know; and, here, COPR becomes less useful.

Would you mind helping us with the documentation?

the page says listed packages are added, and always available to the chroot.

if that's NOT the case, don't say it; rather, say what IS the case.

e.g., "packages listed here are ONLY added to chroot if a Custom build is specified; otherwise they are ignored & remain uninstalled.

it's completely unclear to me what the intent actually is.

[praiskup]

I didn't want to make the discussion heated, sorry. As many times before (mostly on Matrix) I just wanted to help you, and give some well-intentioned suggestions. The most of the maintainers out there consider rpmbuild -bs *.spec to be a completely safe operation, while it never is. I need to be especially careful myself, because I'm helping with/fixing with so many in-the-wild spec files.

Then, I wanted to bring #534 into the consideration, this is definitely not a one-liner eventually.

the page says listed packages are added, and always available to the chroot.

Not sure which page you mean. If you could submit a pull request against the docs where you see the problem, 🚀 please do.

[FrostyX]

this is definitely not a one-liner eventually.

Sorry, this was a bad judgment on my side. I didn't realize this:

there are differences in source methods; most of them are run on host,

[pgnd]

no worries, no heat & no help req'd.

this has become far too complicated. i'm simply filing what's seen here as a bug.

here's a picture:

despite the 'stmt' there, 'jq' is not added to the chroot; and is unavailable for use in the spec.

it's irrelevant whether it's jq, or another pkg, one or many.

i'm not trying to change any functionality; i'm clear my usage/interests don't typically align with COPR/Fedora projects.

i'm simply suggesting that the page "says what it does, and does what it says".

as mentioned, it's unclear to me what 'COPR project' intends here.

[praiskup]

But these specified packages are installed into the RPM build root, or? But note the difference, RPM buildroots are different from the "source build" buildroot (sources are built just once for all RPM buildroots). Then, not every source build method is executed in buildroot - some are executed directly on host. That includes the rhpkg.

[praiskup]

This screenshot may help: