Implementation and purpose of Show-Build-Info

[fendor]

EDIT:

Final implementation

Finally the chosen implementation has been the third ("Dump build info as side effect of Setup.hs build") with the pr #7498

---

Nomenclature (uses recursive name bindings)

Cabal = `lib:Cabal`
cabal = `exe:cabal`
v2-build = nix-style build command for project building (happens in exe:cabal)
Setup.hs build = Cabal build system, e.g. what happens under the hood (happens in lib:Cabal)
v2-show-build-info = nix-style show-build-info command, cli interface for users (happens in exe:cabal)
Setup.hs show-build-info = Cabal show-build-info system, performs actual work (happens in lib:Cabal)

---

The command show-build-info is intended to help tooling based on Cabal to find detailed information on how to build each component individually. It is not intended to expose Cabal details, such as what preprocessors are executed, etc... It also does not subsume Cabal's build process, i.e. having the output of show-build-info does not suffice to build a cabal component in certain situations. For example it is still Cabal's task to generate Paths_*.hs. Therefore, show-build-info only works to compile a project, if Cabal has basically built the project, already.

But then tooling such as hie-bios and hsinspect can use the output of show-build-info to feed them into a GHC session and provide IDE functionality.

Implementation

Extracting compiler information is theoretically not a problem for cabal. Since we parse the .cabal file description and already solve for which dependencies to use, basically all information should be present, right? However, Cabal has been designed with extensibility in mind and achieved that by allowing developers to overwrite certain cycles of a build via custom Setup.hs hooks. These hooks are basically a way for a developer of a Cabal package to overwrite or redefine various build-steps in Cabal.
This means that while exe:cabal knows everything about the package to build, lib:Cabal can change compilation flags when building the project.
Therefore, exe:cabal alone can not provide us with the low-level details on how to compile a component, as only lib:Cabal has the full build information (when using build-type: Custom). Thus, show-build-info must be a part of lib:Cabal, as long as we have build hooks in order to generate the correct build-info reliably.

For the implementation itself, we present three ideas:

Dump build info as side effect in v2-build

Idea: Add an option to exe:cabal v2-build to dump the build information during the build. We would explicitly invoke Setup.hs show-build-info from the v2-build command.

Pros:

• No extra command
• Keeps build-info up-to-date automatically
• Can be easily used by editors without extra invocation

Con:

• Performs more work than necessary
  • build will compile a component, although we might not need to (e.g. build a component of which we only need the ghc options)
  • Could be mitigated by additional flag, such as --only-prebuild
• Maintaining a Cabal command is a lot of work and error-prone.
• Handling build hooks correctly is non-trivial.

Issues:

• What components are available?
  • Solvable via plan.json
    • Doesn't include disabled components right now.
• Which ghc is going to be used?
• How to tell whether build-info is outdated?
  • --dry-run to the rescue! Tells you when a component needs to be rebuilt.

New command, v2-show-build-info, which calls Setup.hs show-build-info

Idea: manually trigger the generation of build information.

Pros:

• Manual control
  • Can be explicitly instructed to give possibly outdated information
• Minimal amount of work possible
  • Don't build a component if you don't have to.
• Extensible
  • New flags might be added, such as --buildinfo-project that prints project global information (quickly).

Con:

• Preprocessing of relevant files might be missing
• Maintaining a cabal command is a lot of work and error-prone
• Handling build hooks correctly is non-trivial.

Issues:

• What components are available?
  • Solvable via plan.json
    • Doesn't include disabled components right now.

Dump build info as side effect of Setup.hs build

Idea: Generate build info whenever Cabal builds a component and put it into the "old-style" builddir of Setup.hs build, i.e. next to setup-config.

Pros:

• Probably the quickest solution as we already parsed LocalBuildInfo/setup-config. Standalone s-b-i command has to redo that work.
• Keeps build-info up-to-date automatically
• Can be easily used by editors without extra invocation
• Best way to avoid missing bugs and corner-cases
  • Handles custom hooks correctly

Con:

• Discovering build-info files is not as trivial as possible.

Issues:

• What components are available?
  • Solvable via plan.json
    • Doesn't include disabled components right now.
• How to tell whether build-info is outdated?
  • --dry-run to the rescue! Tells you when a component needs to be rebuilt.

Implementation details

We can generate the build-info and write it to the build directory of each component (e.g. dist-newstyle/build/.../<package-name>).

After performing a v2-build, exe:cabal writes out build-info.list file to dist-newstyle/cache, containing the list of all build-info files which might exist, i.e. all enabled components. This file can be parsed by external tools to discover all build-infos of the project.

Optionally, we can add a command v2-ide that does:

• Give us the location of each individual build info file
• Give us the build-information for a given filepath/target.
• List all components in the project (enabled and disabled)
• Give us project wide information (such as compiler version) without running the solver (really fast!)

Summary

I discussed show-build-info quite thoroughly with @DanielG, and we concluded that the third approach is probably the cleanest solution.

Current PR: #7478 which basically implements approach 2.

[fendor]

This issue serves as a discussion point that was missing so far.

cc @jneira, @emilypi, @mpickering, @fgaz, @Mikolaj and everyone else who wants to give input!

[jneira]

Each technical doc you write is better than the previous one, congrats!

The discover of the config file is the worse caveat of 1 and 3 i can think of. What happened if the user changes the cabal build dir? We would need just s-b-i to know where s-b-i is placed! When cabal will change the default build dir all clients will have to change as well.

What about doing 1 or 3 and add a flag to the build command to output the content of the file (or at least its location)
Sorry if I am missing something obvious.

[jneira]

Another option could be let the caller decide the location of the generated file: --dump-buildinfo=path and fallback inside the build dir if not set (or create two flags)

[fendor]

I think the latter is a good, lightweight option that we can add to cabal v2-build

[jneira]

@emilypi @Mikolaj @phadej it would be great unblock the feature, choosing one of both open alternatives (dedicated command or v2-build flag), thoughts?

[Mikolaj]

If there is no strong preference either way from the feature's users nor implementors, then an extra flag sounds less intrusive for users not interested in the feature, doesn't it? Such users just see one more line deep down in the help output and the manual (as opposed to a more prominent mention for a command).

Regarding codebase intrusiveness, I guess it's similar, given that the flag needs a counterpart in cabal.project, etc. (unless it's reflected fully automatically; I don't know).

How would the flag interact with cabal install and other commands?

[fendor]

I think the flag --dump-buildinfo=<path> should have no effect on other commands and is therefore only part of BuildFlags.

However, a flag such as --enable-dumpbuildinfo (name subject to bikeshed) should be part of ConfigFlags (Thus part of NixStyleFlags), and interact "as expected" in the sense it generates the build artefacts.

[DanielG]

--enable-dumpbuildinfo

So is the idea for this new flag that this is what controls the generation of buldinfo.json by Setup.hs build and --dump-buildinfo= writes the list of paths for each component's buildinfo.json into the given file?

[fendor]

@DanielG Yes exactly.

Maybe even one step further and combine the results into one build-info.json (e.g. <> all components), since we can (for performance reason) parse the plan.json ourselves and get the location of each build-info.json quite easily.