Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Setup hooks #9551

Merged
merged 3 commits into from
Apr 18, 2024
Merged

Setup hooks #9551

merged 3 commits into from
Apr 18, 2024

Conversation

sheaf
Copy link
Collaborator

@sheaf sheaf commented Dec 21, 2023 •
edited by alt-romes
Loading

This PR contains the implementation of the new Hooks build-type, as described in the Haskell Foundation Tech Proposal.

It builds on several previous PRs. For review, you should only need to look at the Introduce SetupHooks and SetupHooks: add tests commits.

New modules:

  • Distribution.Simple.SetupHooks.Internal contains the internal implementation details of the API for the Hooks build type
  • Distribution.Simple.SetupHooks.Rule defines fine grained rules as specified in the proposal, for use in pre-build hooks
  • Distribution.Simple.SetupHooks.Errors defines error constructors pertaining to the Hooks build type

There are barely any changes to Cabal outside the introduction of these modules. Essentially, the changes consist of defining defaultMainWithSetupHooksArgs, and the introduction of variants build_setupHooks, configure_setupHooks... of existing functions (with e.g. build = build_setupHooks noBuildHooks) which insert the hooked actions at the appropriate times.

New libraries:

  • Cabal-hooks is a new package that exports the exposed interface for package authors to write their own SetupHooks.hs modules. For the time being, it mostly re-exports datatypes from Cabal, together with some slightly higher-level functions for use in the API, all through the single module Distribution.Simple.SetupHooks.
    As explained in the HF tech proposal, a possible future goal would be to make this package no longer depend on Cabal, perhaps making Cabal depend on it instead. This would limit the amount of internal details from Cabal we are exposing through the Hooks API, which would have the potential to improve maintainability of user packages relying on the Hooks API.

TODO:

  • Rebase this PR once preparatory MRs land.
  • Update this PR once the discussion on the tech proposal settles down, with a final design.
  • Ensure all documentation is up-to-date relative to the exposed API.
  • Add changelog entry.
  • Address the remaining SetupHooks TODOs in the code.

--

@sheaf sheaf marked this pull request as draft December 21, 2023 13:40
@sheaf sheaf changed the title Draft: Setup hooksip/setup hooks mr Draft: Setup hooks Dec 21, 2023
@sheaf sheaf force-pushed the wip/setup-hooks-mr branch 25 times, most recently from 9446424 to 0f9ad07 Compare December 22, 2023 15:15
@sheaf sheaf force-pushed the wip/setup-hooks-mr branch 3 times, most recently from 90dc2a8 to 7b94e0b Compare January 11, 2024 15:22
@sheaf sheaf force-pushed the wip/setup-hooks-mr branch 3 times, most recently from 2c8cae9 to f70ef62 Compare April 15, 2024 18:34
@sheaf
Copy link
Collaborator Author

sheaf commented Apr 15, 2024

I've been applying a few fixes, in particular to the Binary instances of datatypes involved in pre-build rule. The main point is that we want to be able to treat arguments to rules as opaque blobs of data on the build system side, which means that any such data must always be prefaced by its length (otherwise the build system doesn't know how much data to read). I am planning to get this merged as soon as CI passes now.

@sheaf sheaf force-pushed the wip/setup-hooks-mr branch 3 times, most recently from 3089a6f to 4b173c6 Compare April 16, 2024 09:11
@sheaf sheaf added the merge me Tell Mergify Bot to merge label Apr 16, 2024
@mergify mergify bot added the merge delay passed Applied (usually by Mergify) when PR approved and received no updates for 2 days label Apr 18, 2024
@sheaf @Mikolaj
Split-off and re-export installFileGlob
58af854 
This commit exposes installFileGlob as a generally useful
part of the API which users might want to call, e.g. in their
custom Setup scripts.
@sheaf @Mikolaj
Split off file monitoring types into Cabal library
b2a62b9 
This commit splits off the file monitoring types from cabal-install into
the Cabal library, so that they can be referred to in pre-build rules
for SetupHooks. This will allow package authors with Hooks build-type to
monitor files and directories specified by globbing.
@sheaf @Mikolaj
Introduce SetupHooks
c5f9933 
This commit introduces a new build-type, Hooks. A package using this
build type should provide a SetupHooks.hs module which exports
a value `setupHooks :: SetupHooks`. This is intended to replace the
Custom setup type. This allows Cabal to have finer-grained information
about the build, instead of having an opaque Setup executable to invoke.

Tests include:

  - error when returning an invalid component diff in a
    per-component pre-configure hook
  - error when declaring pre-build rules whose dependency
    graph contains cycles
  - error when we cannot find a dependency of a pre-build rule
  - warning when there are pre-build rules that are declared but
    never demanded
  - correctness tests for SetupHooks, e.g. that
    pre-build rules are run in dependency order (see the
    `SetupHooksRuleOrdering` test)
@mergify mergify bot merged commit b9084c9 into haskell:master Apr 18, 2024
44 checks passed
@geekosaur geekosaur mentioned this pull request Apr 29, 2024
Merged
2 tasks
geekosaur added a commit that referenced this pull request Apr 29, 2024
@geekosaur
copy an import list from #9551
7579201 
because `System.Process.Internals` just (like, within the past
hour or so) started exporting a name we are using.
geekosaur added a commit that referenced this pull request Apr 29, 2024
@geekosaur
copy an import list from #9551
fef808f 
because `System.Process.Internals` just (like, within the past
hour or so) started exporting a name we are using.
mergify bot added a commit that referenced this pull request Apr 29, 2024
@mergify @ulysses4ever @geekosaur
CI: force MacOS jobs to use Intel runners (macos-13) (backport #9949) (…
d871c1e 
…#9956)

* CI: force MacOS jobs to use Intel runners (`macos-13`)

GitHub just switched macos-latest to the ARM chips (now alisasing
`macos-14`), and it brings a bunch of problems.

- First of all, GHC's 8.8 and 8.6 don't exist there.
- ghcup and llvm are unavailable

For the time being, lets stay on the previous version of the runner.

(cherry picked from commit d36e0d0)

# Conflicts:
#	.github/workflows/validate.yml

* CI: GitHub MacOS runners lost ghcup since 2024-04-27, so use haskell-action/setup instead

(cherry picked from commit 082d952)

* fixup! more compat with new macos runners

(cherry picked from commit 326a1f6)

* !fixup resolve conflicts

* copy an import list from #9551

because `System.Process.Internals` just (like, within the past
hour or so) started exporting a name we are using.

---------

Co-authored-by: Artem Pelenitsyn <a.pelenitsyn@gmail.com>
Co-authored-by: brandon s allbery kf8nh <allbery.b@gmail.com>
geekosaur added a commit that referenced this pull request Apr 29, 2024
@geekosaur
copy an import list from #9551
b050bdf 
because `System.Process.Internals` just (like, within the past
hour or so) started exporting a name we are using.
geekosaur added a commit that referenced this pull request Apr 29, 2024
@mergify @ulysses4ever @geekosaur
CI: force MacOS jobs to use Intel runners (macos-13) (backport #9949) (…
a58d8a2 
…#9956)

* CI: force MacOS jobs to use Intel runners (`macos-13`)

GitHub just switched macos-latest to the ARM chips (now alisasing
`macos-14`), and it brings a bunch of problems.

- First of all, GHC's 8.8 and 8.6 don't exist there.
- ghcup and llvm are unavailable

For the time being, lets stay on the previous version of the runner.

(cherry picked from commit d36e0d0)

* CI: GitHub MacOS runners lost ghcup since 2024-04-27, so use haskell-action/setup instead

(cherry picked from commit 082d952)

* fixup! more compat with new macos runners

(cherry picked from commit 326a1f6)

* !fixup resolve conflicts

* copy an import list from #9551

because `System.Process.Internals` just (like, within the past
hour or so) started exporting a name we are using.

---------

Co-authored-by: Artem Pelenitsyn <a.pelenitsyn@gmail.com>
Co-authored-by: brandon s allbery kf8nh <allbery.b@gmail.com>
Mikolaj added a commit that referenced this pull request May 1, 2024
@Mikolaj
CI: stop validating GHC 7 (backport #9886) (#9957)
280aaa8 
3.12 changelog fixup (#9922)

* Incorporate Brandon’s suggestions

See #9920.

* Incorporate Artem’s suggestions

See #9920.

* Do not repeat yourself

* Fix release notes grammar (#9924)

* Fix release notes grammar

See #9920.

* Fix whitespace

* Support GHC 9.12

(cherry picked from commit da6bdef)

* Fix changelog/readme (backport #9935) (#9936)

* Fix changelog/readme

(cherry picked from commit ea0f464)

* Remove previous release date

---------

Co-authored-by: Francesco Ariis <fa-ml@ariis.it>

* Tell zlib not to use pkg-config in GitLab CI.

(cherry picked from commit 62c74fe)

* Revert "Mark ForeignLibs test as broken with ghc-8.4.4"

This reverts commit a90d44f.

(cherry picked from commit d0a690b)

* CI: drop validation of GHC 7

Changes:
- bump GHC_FOR_RELEASE to 9.4.8
- bump action versions
- uniform quoting style
- satisfy actionlint
- fix order: setup Haskell before cache restore (uses setup.haskell-outputs)
- use `--ignore-project` in `cabal install hackage-repo-tool`
- use GHC_FOR_RELEASE also in validate-old-ghcs
- closes #8858: deleted comment
- closes #9858 by dropping container and using ghcup to setup ghcs

GHCs that do not install on ubuntu-22.04 with GHCup are dropped, meaning we only keep GHC 8.0.2 and up.

(cherry picked from commit 29dc53c)

# Conflicts:
#	.github/workflows/validate.yml

* fix validate.yml conflicts

How is this backport conflicting with _itself_?

* copy an import list from #9551

because `System.Process.Internals` just (like, within the past
hour or so) started exporting a name we are using.

* CI: force MacOS jobs to use Intel runners (macos-13) (backport #9949) (#9956)

* CI: force MacOS jobs to use Intel runners (`macos-13`)

GitHub just switched macos-latest to the ARM chips (now alisasing
`macos-14`), and it brings a bunch of problems.

- First of all, GHC's 8.8 and 8.6 don't exist there.
- ghcup and llvm are unavailable

For the time being, lets stay on the previous version of the runner.

(cherry picked from commit d36e0d0)

* CI: GitHub MacOS runners lost ghcup since 2024-04-27, so use haskell-action/setup instead

(cherry picked from commit 082d952)

* fixup! more compat with new macos runners

(cherry picked from commit 326a1f6)

* !fixup resolve conflicts

* copy an import list from #9551

because `System.Process.Internals` just (like, within the past
hour or so) started exporting a name we are using.

---------

Co-authored-by: Artem Pelenitsyn <a.pelenitsyn@gmail.com>
Co-authored-by: brandon s allbery kf8nh <allbery.b@gmail.com>

* Merge branch '3.12' into mergify/bp/3.12/pr-9886

* Update validate.yml

github nicely decided to ~revert~ the OS X validate fix when I rebased on top of it.

* make validate.yml consistent with master
mergify bot added a commit that referenced this pull request May 1, 2024
@mergify @Mikolaj
CI: stop validating GHC 7 (backport #9886) (#9957) (#9957)
815b141 
3.12 changelog fixup (#9922)

* Incorporate Brandon’s suggestions

See #9920.

* Incorporate Artem’s suggestions

See #9920.

* Do not repeat yourself

* Fix release notes grammar (#9924)

* Fix release notes grammar

See #9920.

* Fix whitespace

* Support GHC 9.12

(cherry picked from commit da6bdef)

* Fix changelog/readme (backport #9935) (#9936)

* Fix changelog/readme

(cherry picked from commit ea0f464)

* Remove previous release date

---------

Co-authored-by: Francesco Ariis <fa-ml@ariis.it>

* Tell zlib not to use pkg-config in GitLab CI.

(cherry picked from commit 62c74fe)

* Revert "Mark ForeignLibs test as broken with ghc-8.4.4"

This reverts commit a90d44f.

(cherry picked from commit d0a690b)

* CI: drop validation of GHC 7

Changes:
- bump GHC_FOR_RELEASE to 9.4.8
- bump action versions
- uniform quoting style
- satisfy actionlint
- fix order: setup Haskell before cache restore (uses setup.haskell-outputs)
- use `--ignore-project` in `cabal install hackage-repo-tool`
- use GHC_FOR_RELEASE also in validate-old-ghcs
- closes #8858: deleted comment
- closes #9858 by dropping container and using ghcup to setup ghcs

GHCs that do not install on ubuntu-22.04 with GHCup are dropped, meaning we only keep GHC 8.0.2 and up.

(cherry picked from commit 29dc53c)

# Conflicts:
#	.github/workflows/validate.yml

* fix validate.yml conflicts

How is this backport conflicting with _itself_?

* copy an import list from #9551

because `System.Process.Internals` just (like, within the past
hour or so) started exporting a name we are using.

* CI: force MacOS jobs to use Intel runners (macos-13) (backport #9949) (#9956)

* CI: force MacOS jobs to use Intel runners (`macos-13`)

GitHub just switched macos-latest to the ARM chips (now alisasing
`macos-14`), and it brings a bunch of problems.

- First of all, GHC's 8.8 and 8.6 don't exist there.
- ghcup and llvm are unavailable

For the time being, lets stay on the previous version of the runner.

(cherry picked from commit d36e0d0)

* CI: GitHub MacOS runners lost ghcup since 2024-04-27, so use haskell-action/setup instead

(cherry picked from commit 082d952)

* fixup! more compat with new macos runners

(cherry picked from commit 326a1f6)

* !fixup resolve conflicts

* copy an import list from #9551

because `System.Process.Internals` just (like, within the past
hour or so) started exporting a name we are using.

---------

Co-authored-by: Artem Pelenitsyn <a.pelenitsyn@gmail.com>
Co-authored-by: brandon s allbery kf8nh <allbery.b@gmail.com>

* Merge branch '3.12' into mergify/bp/3.12/pr-9886

* Update validate.yml

github nicely decided to ~revert~ the OS X validate fix when I rebased on top of it.

* make validate.yml consistent with master

Co-authored-by: Mikolaj <281893+Mikolaj@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@ulysses4ever
Copy link
Collaborator

This is a big feature. Would it be okay for it to wait until cabal-install-3.14? Or does it call for an exception in our conservative backporting strategy? @Mikolaj? @sheaf? @alt-romes? @andreabedini?

@geekosaur
Copy link
Collaborator

I would wait.

@ulysses4ever
Copy link
Collaborator

Yeah, on today's Cabal call we decided to wait.

@Chaman-veteran Chaman-veteran mentioned this pull request Jun 16, 2024
Closed
@andreabedini andreabedini mentioned this pull request Jun 27, 2024
Closed
@cdepillabout cdepillabout mentioned this pull request Jun 27, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alt-romes alt-romes alt-romes left review comments

@andreabedini andreabedini andreabedini approved these changes

@gbaz gbaz gbaz approved these changes

@geekosaur geekosaur geekosaur approved these changes

Assignees
No one assigned
Labels
merge delay passed Applied (usually by Mergify) when PR approved and received no updates for 2 days merge me Tell Mergify Bot to merge
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@sheaf @andreabedini @alt-romes @gbaz @ulysses4ever @geekosaur @adamgundry