New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forward compat scheme #4899

Closed
hvr opened this Issue Nov 20, 2017 · 5 comments

Comments

Projects
None yet
5 participants
@hvr
Member

hvr commented Nov 20, 2017

Generally, a given cabal-install release is only expected to understand (cabal-)spec(ification)-versions that were known at the time of release.

The Cabal-1.12 release, and more recently the Cabal-2.0 release have made it apparent that we don't have a good forward-compatibility story in place, thereby limiting our ability to evolve the .cabal format. Concretely, cabal-install-1.10 is not expected to understand package meta-data declared using the cabal-version:1.12 spec(ification)-version; also, upon encountering such a future .cabal file in the package index, cabal-install-1.10 ought to gracefully recover (instead of failing fatally to parse the package index). A similar situation occurred with the Cabal-2.0 release, which also introduced new syntax which previous parsers would fail to recognise, again resulting in fatal parsing failures.

To prevent this from happening again, the following forward compatibility has been devised, which is intended to make it possible to evolve the lexical and syntactical structure of the .cabal format in future, while not breaking legacy clients.


New-style Cabal-Specification Version Declaration

A new-style spec-version declaration at the beginning of a the .cabal file (without any preceding whitespace) and follow the following case-insensitive grammar (expressed in RFC5234 ABNF):

newstyle-spec-version-decl = "cabal-version" *WS ":" *WS newstyle-spec-version *WS

newstyle-spec-version      = NUM "." NUM [ "." NUM ]

NUM    = DIGIT0 / DIGITP 1*DIGIT0
DIGIT0 = %x30-39
DIGITP = %x31-39
WS = %20

The use of a new-style spec-version is

  • invalid before spec-version 1.12,
  • optional starting with spec-version 1.12,
  • recommended starting with spec-version 2.0, and
  • mandatory starting with spec-version 2.1.

It's also assumed that the following invariant holds:

  • If a new-style spec-version declaration is present, its value must match the spec-version as determined by a full .cabal parser.

Scanning the Cabal-Specification Version

  1. If a .cabal file contains a valid new-style spec-version declaration, it is authoritative;
  2. otherwise, the spec-version must be below 2.1 (and either a full .cabal parser or a heuristic cabal-version-scanner (tbd) must be used to determine the exact spec-version).

The new-style spec-version declaration is designed to be simple to parse by means of common string operations. A simple implementation is shown below

scanSpecVersion :: ByteString -> Maybe Version
scanSpecVersion bs = do
    fstline':_ <- pure (BS.Char8.lines bs)

    -- parse <newstyle-spec-version-decl>
    -- normalise: remove all whitespace, convert to lower-case
    let fstline = BSW8.map toLowerW8 $ BSW8.filter (/= 0x20) $ BS.toStrict fstline'
    ["cabal-version",vers] <- pure (BSS.split ':' fstline)

    -- parse <spec-version> tolerantly
    ver <- simpleParse (BSS.unpack vers)
    guard $ case versionNumbers ver of
              [_,_]   -> True
              [_,_,_] -> True
              _       -> False

    pure ver
  where
    -- | Translate ['A'..'Z'] to ['a'..'z']
    toLowerW8 :: Word8 -> Word8
    toLowerW8 w | 0x40 < w && w < 0x5b = w+0x20
                | otherwise            = w

Appendix: Compatiblity with clients prior to cabal 2.0

Since this new scheme is only understood properly starting with cabal 2.0, older clients need to avoid being exposed to spec-versions 2.0 and newer.

To this end we exploit that cabal 2.0 started using the incremental secure 01-index.tar package index by default, while cabal versions prior to cabal 2.0 use the (non-incremental/non-secure) 00-index.tar package index by default (NB: cabal 1.24 was the first release that added experimental non-default/opt-in support for the secure 01-index.tar), by establishing the following hackage-side invariant:

  • the legacy index 00-index.tar.gz contains only .cabal files with a spec version below 2

This way, the huge install-base of legacy cabal clients prior to cabal 2.0 keep working without requiring modifications, as they won't be exposed to incompatible .cabal files; with the unfortunate exception that the (hopefully uncommon) cabal clients prior to cabal 1.12 may still be exposed to incompatible cabal versions using the >=-less spec-version declarations.

hvr added a commit to hvr/cabal that referenced this issue Nov 20, 2017

Implement preliminary support for new forward-compat scheme
This provides a provisional (i.e. hacky) retrofitted implementation of
the forward-compat scheme described in haskell#4899 for the cabal-2.0 branch.

This hack works by constructing a dummy package description in case
the package description fails to be parsed via the standard parser,
and we detect a new-style cabal-spec declaration.

hvr added a commit to hvr/cabal that referenced this issue Nov 24, 2017

Have the solver reject packages with a too-new/unsupported spec-version
This compares the request spec-version to the lib:Cabal's version in order to
determine whether cabal is able to properly understand the package. If it's
newer than the currently linked lib:Cabal version it's turned into a global
`FailResult` which the solver treats as desired in terms of backtracking and
error reporting.

This is related to the new spec-version forward-compat scheme (see haskell#4899).

This is complements haskell#4900 for the 2.0 branch.

hvr added a commit to hvr/cabal that referenced this issue Nov 24, 2017

Have the solver reject packages with a too-new/unsupported spec-version
This compares the request spec-version to the lib:Cabal's version in order to
determine whether cabal is able to properly understand the package. If it's
newer than the currently linked lib:Cabal version it's turned into a global
`FailResult` which the solver treats as desired in terms of backtracking and
error reporting.

This is related to the new spec-version forward-compat scheme (see haskell#4899).

This is complements haskell#4900 for the 2.0 branch.

hvr added a commit to hvr/cabal that referenced this issue Nov 25, 2017

Have the solver reject packages with a too-new/unsupported spec-version
This compares the request spec-version to the lib:Cabal's version in order to
determine whether cabal is able to properly understand the package. If it's
newer than the currently linked lib:Cabal version it's turned into a global
`FailResult` which the solver treats as desired in terms of backtracking and
error reporting.

This is related to the new spec-version forward-compat scheme (see haskell#4899).

This is a forward-port of haskell#4907 to the `master` branch
@23Skidoo

This comment has been minimized.

Member

23Skidoo commented Dec 8, 2017

Related: #4448.

@gbaz

This comment has been minimized.

Collaborator

gbaz commented Dec 11, 2017

This seems like a good way forward.

@phadej phadej added this to the 2.2 milestone Jan 14, 2018

phadej added a commit to phadej/cabal that referenced this issue Jan 14, 2018

Implement preliminary support for new forward-compat scheme
This provides a provisional (i.e. hacky) retrofitted implementation of
the forward-compat scheme described in haskell#4899 for the cabal-2.2 branch

This hack works by constructing a dummy package description in case
the package description fails to be parsed via the standard parser,
and we detect a new-style cabal-spec declaration.
@phadej

This comment has been minimized.

Collaborator

phadej commented Jan 14, 2018

@hvr, should the ABNF contain end-of-line character? '\n' ?

phadej added a commit to phadej/cabal that referenced this issue Jan 14, 2018

Implement preliminary support for new forward-compat scheme
This provides a provisional (i.e. hacky) retrofitted implementation of
the forward-compat scheme described in haskell#4899 for the cabal-2.2 branch

This hack works by constructing a dummy package description in case
the package description fails to be parsed via the standard parser,
and we detect a new-style cabal-spec declaration.

phadej added a commit to phadej/cabal that referenced this issue Jan 14, 2018

Implement preliminary support for new forward-compat scheme
This provides a provisional (i.e. hacky) retrofitted implementation of
the forward-compat scheme described in haskell#4899 for the cabal-2.2 branch

This hack works by constructing a dummy package description in case
the package description fails to be parsed via the standard parser,
and we detect a new-style cabal-spec declaration.

@phadej phadej referenced this issue Jan 14, 2018

Closed

Issue 4899 forward compat scheme #5028

0 of 4 tasks complete

phadej added a commit to phadej/cabal that referenced this issue Jan 15, 2018

Implement preliminary support for new forward-compat scheme
This provides a provisional (i.e. hacky) retrofitted implementation of
the forward-compat scheme described in haskell#4899 for the cabal-2.2 branch

This hack works by constructing a dummy package description in case
the package description fails to be parsed via the standard parser,
and we detect a new-style cabal-spec declaration.

phadej added a commit to phadej/cabal that referenced this issue Jan 16, 2018

Implement preliminary support for new forward-compat scheme
This provides a provisional (i.e. hacky) retrofitted implementation of
the forward-compat scheme described in haskell#4899 for the cabal-2.2 branch

This hack works by constructing a dummy package description in case
the package description fails to be parsed via the standard parser,
and we detect a new-style cabal-spec declaration.
@phadej

This comment has been minimized.

Collaborator

phadej commented Jan 17, 2018

merged in #5046

@theindigamer

This comment has been minimized.

theindigamer commented May 21, 2018

Can the docs be updated to reflect this change? I just encountered this issue and am having a hard time understanding what I should do if I want to express >= 2.2 (I'm using common stanzas). Nevermind, the docs have been updated in #5131.

vu3rdd added a commit to LeastAuthority/wormhole-client that referenced this issue Aug 16, 2018

fix CI build because apparently cabal versions are no longer specifie…
…d as a range

CI reports a failure:
  unexpected cabal-version higher than 2.2 cannot be specified as a range. See
  haskell/cabal#4899
  expecting ".", "-", white space, "&&" or "||" >=2.2
  setup-Simple-Cabal-2.2.0.1-x86_64-linux-ghc-8.0.2: Failed parsing "./hwormhole.cabal".

This change also relaxes the lower bound to 2.0 instead of the very new 2.2 and see
how that goes.

vu3rdd added a commit to LeastAuthority/wormhole-client that referenced this issue Aug 16, 2018

fix CI build because apparently cabal versions are no longer specifie…
…d as a range

CI reports a failure:
  unexpected cabal-version higher than 2.2 cannot be specified as a range. See
  haskell/cabal#4899
  expecting ".", "-", white space, "&&" or "||" >=2.2
  setup-Simple-Cabal-2.2.0.1-x86_64-linux-ghc-8.0.2: Failed parsing "./hwormhole.cabal".

This change also relaxes the lower bound to 2.0 instead of the very new 2.2 and see
how that goes.

vu3rdd added a commit to LeastAuthority/wormhole-client that referenced this issue Aug 20, 2018

fix CI build because apparently cabal versions are no longer specifie…
…d as a range

This change also relaxes the lower bound to 1.24 instead of the very new 2.2 and see
how that goes.

CI reports a failure:
  unexpected cabal-version higher than 2.2 cannot be specified as a range. See
  haskell/cabal#4899
  expecting ".", "-", white space, "&&" or "||" >=2.2
  setup-Simple-Cabal-2.2.0.1-x86_64-linux-ghc-8.0.2: Failed parsing "./hwormhole.cabal".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment