Use in-tree Cabal library for cabal-install tests with custom setup.

[mpickering]

Allow using different Cabal library versions for cabal-install tests with custom setup.

The idea here is to pass a `--package-db` flag to `cabal-install` which
contains just `Cabal` and `Cabal-syntax` of the specific version. This
allows `cabal-install` tests to use the in-tree `Cabal` version,
something which you can easily run into and get very confused about when
writing tests.

There are a few options which can be passed to `cabal-tests` executable
to control which Cabal library you will test against.

1. --boot-cabal-lib specifies to use the Cabal library bundled with the
   test compiler, this is the default and existing behaviour of the
   testsuite.
2. --intree-cabal-lib=<root_dir> specifies to use Cabal and Cabal-syntax
   from a specific directory, and `--test-tmp` indicates where to put
   the package database they are built with.
3. --specific-cabal-lib=<VERSION> specifies to use a specific Cabal
   version from hackage (ie 3.10.2.0) and installs the package database
   into --test-tmp=<DIR>

The end result is that changes in the Cabal library can be tested with
cabal-install tests in the testsuite.

There have been a number of confusing issues with people writing tests
for changes in the Cabal library which never ran because of
cabal-install tests always used the boot Cabal library (see #9425
for one).

Fixes #9681

[Mikolaj]

This implements #9681.

[Mikolaj]

BTW is this ready for review (I don't see the label)?

[mpickering]

I have been concentrating so far on getting CI working for the normal CI configurations, which it now does.

There are some further things I want to work on:

• Adding some more configurations of testing different Cabal versions to CI (ie 9.8.1 with Cabal 3.10.1.0 rather than 3.11.0.0)
• Going through the testsuite and checking that tests which query GHC version actually mean to query the GHC version or instead mean to query the Cabal library version.

But I think it would be good to merge this firstly.

[Mikolaj]

I see. Great. That's normally signalled by setting the review_needed labels, which is why I ask. Let me set it instead.

[Mikolaj]

That looks great. However, is there a possibility this breaks somebody's workflow? If so, can we mitigate that, give some porting hints, deprecate things in advance? If not, why?

[mpickering]

I think by default, no there isn't. Because if you run the tests locally then the same behaviour will happen as before (using the boot Cabal version rather than the in-tree one). Perhaps that is confusing in itself.

You have to explicitly pass a flag to cabal-tests in order to get it to use a different Cabal version than the one you already have. You are right that I still need to document this feature in the documentation as well. I will try to do that this afternoon.

[mpickering]

I added some documentation now so I think this first step is ready to merge after #9671 is merged.

[ulysses4ever]

synopsis: Clarify the semantics of the -package-db flag

Is it possible to update the PR title and description in accordance with the actual changes?

[ulysses4ever]

I added some documentation now so I think this first step is ready to merge after #9671 is merged.

This is #9671.

[mpickering]

I have rebased and this patch is ready to merge.

[alt-romes]

This is a great improvement to the testsuite. I've been very confused myself regarding changes to Cabal not working as intended in the testsuite because of this (and going mad...). Thanks!

[ulysses4ever]

Terrific, thanks!

[Mikolaj]

I'm afraid there are conflicts to solve before this can auto-merge.

[Kleidukos]

@Mergifyio backport 3.12

[mergify]

backport 3.12

✅ Backports have been created

• #9999 Use in-tree Cabal library for cabal-install tests with custom setup. (backport #9671) has been created for branch 3.12