Add test cases that reproduce sdist --project-file.

[philderbeast]

This is a WIP. Struggling a bit to navigate the test set up, I opted instead to create a small set of tests for reproducing the problem of cabal sdist --project-file not respecting the project file option, #7241.

I'm testing this with:

> cabal --version
cabal-install version 3.8.0.20220526
compiled using version 3.8.0.20220526 of the Cabal library

[philderbeast]

If in the sdist/projects-default-no folder then any of the calls in repro.txt result is sdist going up the tree like this:

> cabal sdist all
Wrote tarball sdist to
/.../cabal/dist-newstyle/sdist/Cabal-3.9.0.0.tar.gz
Wrote tarball sdist to
/.../cabal/dist-newstyle/sdist/cabal-testsuite-3.tar.gz
Wrote tarball sdist to
/.../cabal/dist-newstyle/sdist/Cabal-syntax-3.9.0.0.tar.gz
Wrote tarball sdist to
/.../cabal/dist-newstyle/sdist/cabal-install-3.9.0.0.tar.gz
Wrote tarball sdist to
/.../cabal/dist-newstyle/sdist/cabal-install-solver-3.9.0.0.tar.gz
Wrote tarball sdist to
/.../cabal/dist-newstyle/sdist/solver-benchmarks-3.tar.gz
Wrote tarball sdist to
/.../cabal/dist-newstyle/sdist/Cabal-QuickCheck-3.9.0.0.tar.gz
Wrote tarball sdist to
/.../cabal/dist-newstyle/sdist/Cabal-tree-diff-3.9.0.0.tar.gz
Wrote tarball sdist to
/.../cabal/dist-newstyle/sdist/Cabal-described-3.9.0.0.tar.gz
Wrote tarball sdist to
/.../cabal/dist-newstyle/sdist/Cabal-tests-3.tar.gz
Wrote tarball sdist to
/./cabal/dist-newstyle/sdist/cabal-benchmarks-3.tar.gz

For the sdist/projects-default-yes folder that has a cabal.project file, a project with the default name, the behaviour is different but each time it is not respecting the --project-file option:

> cabal sdist all
Error: cabal: sdist of p-0.1: Could not find module: P with any suffix:
["gc","chs","hsc","x","y","ly","cpphs","hs","lhs","hsig","lhsig"]. If the
module is autogenerated it should be added to 'autogen-modules'.

> cabal sdist all --project-file=cabal.project
Error: cabal: sdist of p-0.1: Could not find module: P with any suffix:
["gc","chs","hsc","x","y","ly","cpphs","hs","lhs","hsig","lhsig"]. If the
module is autogenerated it should be added to 'autogen-modules'.

> cabal sdist all --project-file=cabal.sub-rs.project
Error: cabal: sdist of p-0.1: Could not find module: P with any suffix:
["gc","chs","hsc","x","y","ly","cpphs","hs","lhs","hsig","lhsig"]. If the
module is autogenerated it should be added to 'autogen-modules'.

> cabal sdist all --project-file=cabal.dot-uv.project
Error: cabal: sdist of p-0.1: Could not find module: P with any suffix:
["gc","chs","hsc","x","y","ly","cpphs","hs","lhs","hsig","lhsig"]. If the
module is autogenerated it should be added to 'autogen-modules'.

[Mikolaj]

result is sdist going up the tree like this

If I understand what is says, perhaps it's because the place the project file is in is the root from which sdist operates, not the directory sdist was called in?

the behaviour is different but each time it is not respecting the --project-file option

Thank you for the repro. So which of the four calls should work fine? BTW, did you try moving the options before all? Before sdist?

[Kleidukos]

Marking this PR as draft 🙂

[philderbeast]

Tests of sdist with --project-file

$ tree -P '*.project|*.test.hs' --prune
.
├── cabal.ignore-project.test.hs
├── cabal.no-project.test.hs
├── cabal.project
├── Projects-Default-No
│   ├── cabal.dot-uv.project
│   ├── cabal.dot-uv.test.hs
│   ├── cabal.ignore-project.test.hs
│   ├── cabal.no-project.test.hs
│   ├── cabal.sub-pq.project
│   ├── cabal.sub-pq.test.hs
│   ├── cabal.sub-rs.project
│   └── cabal.sub-rs.test.hs
└── Projects-Default-Yes
    ├── cabal.dot-uv.project
    ├── cabal.dot-uv.test.hs
    ├── cabal.ignore-project.test.hs
    ├── cabal.no-project.test.hs
    ├── cabal.project
    ├── cabal.project.test.hs
    ├── cabal.sub-pq.project
    ├── cabal.sub-pq.test.hs
    ├── cabal.sub-rs.project
    └── cabal.sub-rs.test.hs

3 directories, 21 files

There are of the two subdirectories, one has a cabal.project and the other doesn't. This is the default project. There are two important things to notice with these tests.

1. All the tests with a supplied --project-file option pick up a default cabal.project instead; either the one one in the current directory or the one from the parent directory, one level up. I think this behaviour is wrong and the supplied --project-file option should be respected.

   Before I'd put a project there, one level up, the project probing had gone all the way up to Cabal's own cabal.project as can be seen by this diff after that change:

   $ git diff
   ...
   --- a/cabal-testsuite/PackageTests/SDist/Respect-Project-File/Projects-Default-No/cabal.sub-rs.out
   +++ b/cabal-testsuite/PackageTests/SDist/Respect-Project-File/Projects-Default-No/cabal.sub-rs.out
   @@ -1,12 +1,2 @@
   # cabal sdist
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/Cabal-3.11.0.0.tar.gz
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/cabal-testsuite-3.tar.gz
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/Cabal-syntax-3.11.0.0.tar.gz
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/cabal-install-3.11.0.0.tar.gz
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/cabal-install-solver-3.11.0.0.tar.gz
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/solver-benchmarks-3.tar.gz
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/Cabal-QuickCheck-3.11.0.0.tar.gz
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/Cabal-tree-diff-3.11.0.0.tar.gz
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/Cabal-described-3.11.0.0.tar.gz
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/Cabal-tests-3.tar.gz
   -Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/cabal-benchmarks-3.tar.gz
   +Wrote tarball sdist to <ROOT>/cabal.sub-rs.dist/work/./dist/sdist/p-0.1.tar.gz

2. The --ignore-project option works, as witnessed by each cabal.ignore-project.test.hs when the package in the same directory as the test is used. Note the parse failure when sublibraries are used. I think we could provide a better error message there:

   $ cabal sdist
   ...
   Errors encountered when parsing cabal file ./uv.cabal:
   
   uv.cabal:8:22: error:
   unexpected "u"
   
       7 |   visibility: public
       8 |   exposed-modules: U.u
       |                      ^
   

[philderbeast]

This was CI green but out of date with the master branch. I've just rebased.

[andreasabel]

Thanks, great to have these tests!

Would it be possible to add to each test (I guess the .hs files) a comment stating what this test is testing and what the expected outcome is (like "should fail with...", "should succeed...")?
I think this would make it easier for posterity to comprehend the tests, and to fix tests should they be failiing because of a change in cabal...

[philderbeast]

@andreasabel I've left notes in the *.test.hs files saying that either the test is correct or saying that the test is actually producing wrong output because cabal sdist --project-file=... is not respecting the project. In the later case I've included patches that alter the *.out file to what I would expect if cabal sdist did respect the project.

The cabal-testsuite/PackageTests/SDist/Respect-Project-File/README.md gives an overview of the tests.

[philderbeast]

Currently comparing cabal sdist output with cabal v2-sdist output and they're not the same. I thought the v2- commands (forwards-compatible aliases) are now the same as their non-prefixed sibling commands so cabal build and cabal v2-build do the same thing. That is not the case for cabal sdist and cabal v2-sdist.

[andreasabel]

Thanks, @philderbeast , for equipping the test with descriptions!

[andreasabel]

@philderbeast : Thanks!
When ready, please apply the "squash+merge me" label so this PR gets squashed and merged by Mergify.

[philderbeast]

@andreasabel I squashed manually.

Label merge+no rebase is necessary when the pull request is from an organisation.

[Mikolaj]

@philderbeast: thank you very much again for the exotic label justification ceremony.

BTW, I think, after you squashed, you forgot to rebase just before merging, which resulted in a tangled git history (not that tangled and that's not a big deal, but helps a bit when trawling the commit list, e.g., when bisecting problems). I'm sorry about the trouble --- normally that's one of the things mergify does for us, rebasing and testing as many times as needed, if the branch moves while it's working.

[andreasabel]

@philderbeast wrote:

Label merge+no rebase is necessary when the pull request is from an organisation.

Ah, I didn't know about that. Can I read up on this somewhere?
E.g. for the Agda repo I disabled the "merge commit" option for PRs.