Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define switch invariants rather than "base packages" #3894

Merged
merged 20 commits into from
Mar 12, 2020

Conversation

AltGr
Copy link
Member

@AltGr AltGr commented Jun 27, 2019

NOTE: this is WIP and should be rebased once #3478 is merged to avoid
patch duplication

Currently, we define switches with a set of "base packages", that
normally correspond to the compiler, and that are strongly locked
(unless --unlock-base is specified). This is somehow inherited from
the older "compiler definitions" that weren't packages.

This PR proposes to switch to a much more flexible notion of "switch
invariant", which can be an arbitrary dependency formula (same format
as depends:, minus the extended formulas).

  • The invariant is defined in SWITCH/.opam-switch/switch-config
  • It can be automatically inferred from the base packages (so the
    upgrade should be smooth. No global upgrade needed)
  • This allows easy (and controllable) automatic upgrades of compilers
    (typically 'ocaml-system')
  • The solver will always enforce it (using a virtual package
    internally, without consequences on the optim criteria)
  • Base packages are still there, but informative only (and they help
    for migration). They will be automatically updated on changes

This works quite well, but some small parts are still missing on top
of it:

  • Switch creation needs updating to generate the invariants, either
    inferred from the arguments on the CLI, or the otherwise known base
    packages
  • --unlock-base won't work anymore, it needs to affect the
    invariant instead of the base packages
  • provide CLI options for switch invariant control (show and change)
  • check stuff that relies on base packages, like opam fixup
  • we may consider extending the switch-export format to include
    invariants
  • there is some regression on conflict messages

@AltGr
Copy link
Member Author

AltGr commented Sep 5, 2019

Auto-merge with the merged #3960 was completely broken (duplicated definitions), hence the failures.
Rebasing...

@AltGr
Copy link
Member Author

AltGr commented Mar 9, 2020

Rebased. I am still working on improving the conflict messages, but it ended up being much more difficult than expected.

So I say we don't stall this any longer, knowing that there will be a regression on conflict error messages when the compiler versions are in cause; that'll be improved in another PR asap

@AltGr AltGr added the PR: LGTM label Mar 9, 2020
@rjbou rjbou force-pushed the switch-invariants branch 2 times, most recently from ccac087 to 3e9440b Compare March 10, 2020 17:07
in particular, conflict messages.

Besides checking for regressions, this is also a useful way to track the
evolution of these messages in a clearer way.
they sometimes got split into version subsets for A cflt B and B.1 cflt A
For consistency with the computation of invalidated and available packages.
This is done from base packages when no invariant is defined, and the
switch-config version is older than 2.1
AltGr and others added 12 commits March 11, 2020 16:55
- fix handling of the invariant in installability queries
- remove the remaining use of "base" packages for cleanup, it was still
  enforcing that base packages don't change version.
  A more involved cleanup could be implemented if the solvers can't cope
The hack we used, based on the base packages, no longer works
The approach is to
- remove the invariant while resolving
- update the configured invariant, keeping the package names that were
  part of it, but replacing the constraints that no longer match with
  adjusted ones.

Note that this should work OK for allowing compiler up/downgrades, but
can lead to an empty invariant if you just switched compiler
implementation. Doing the simpler things still seems better in this case
than some magic, though, and the user is told how to update.
To make things much simpler than what we had (with lots of code to
choose the compilers when creating switches on existing projects), the
switch creation is now done in a single pass (compiler + local
packages), which means the single call to the solver determines the
compiler: this should be much more reliable.

Then the "switch invariant" is set according to the packages with the
"compiler" flag that got installed.

Note that this should also be much more friendly when creating a switch
in a directory that contains compiler definitions (this is detected, and
the "default compiler" selection isn't used).
Now called 'invalidated' packages
Based on, and closes ocaml#3816 and ocaml#3817
aspcud will still eat your memory in some cases, but it won't crash on
our submitted file.
it just doesn't scale anymore
@AltGr
Copy link
Member Author

AltGr commented Mar 12, 2020

The last test on appveyor apparently failed due to OCaml 4.10, but otherwise all tests seem OK.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants