lower pvp-bounds: extraneous version range for a dependency on an internal library

[mfine]

General summary/comments

Publishing with pvp-bounds: lower has stopped working with no change in tooling. Hackage is now returning an "extraneous version range" on an internal library error and failing the upload with 400 "Invalid package" - previously this hadn't been the case.

I see this check in cabal - potentially this is from updated software on hackage?

Steps to reproduce

Publish a package to hackage containing an executable with an unbounded dependency on the package's library with a stack.yaml containing pvp-bounds: lower.

Expected

The publish has in the past passed without issues. In hackage I see that the unbounded dependency on the package's library has a lower bound in wolf-0.3.23.cabal, like:

executable wolf-actor
    build-depends:
        wolf >=0.3.23,

Actual

Now the publish fails with:

Getting file list for /Users/mark/repos/wolf/
Building sdist tarball for /Users/mark/repos/wolf/
Checking package 'wolf' for common mistakes
Uploading wolf-0.3.24.tar.gz... unhandled status code: 400
Error: Invalid package

The package has an extraneous version range for a dependency on an internal library: wolf >=0.3.24, wolf >=0.3.24, wolf >=0.3.24. This version range includes the current package but isn't needed as the current package's library will always be used.
Upload failed on wolf-0.3.24.tar.gz

I am able to get the publish to pass by manually removing the bounds on the internal library in wolf-0.3.24.cabal, like:

executable wolf-actor
    main-is: actor.hs
    build-depends:
        wolf,

Stack version

$ stack --version
Version 1.4.0, Git revision e714f1dd3fade19496d91bd6a017e435a96a6bcd (4640 commits) x86_64 hpack-0.17.0

Also tried with 1.5.0:

$ ~/Downloads/stack-1.5.0-osx-x86_64/stack --version
Version 1.5.0, Git revision 63267f94d7c819cbecc2d59aa259d17240838e43 (4845 commits) x86_64 hpack-0.17.1

Method of installation

• Official binary, downloaded from fpcomplete's package repository

[borsboom]

I just hit the same error uploading stack-1.5.0 with --pvp-bounds=both.

[borsboom]

I'm not sure if this is related, but --pvp-bounds also seems to drop the custom-setup section.

[mfine]

There's probably much better solutions, but something like filtering out the package from adding bounds master...mfine:mfine/filter-internal-dependencies

[Blaisorblade]

Lookin at #3413's "user story": it seems the bug was hard to debug because stack upload didn't warn/inform it was modifying the .cabal fine.
@harendra-kumar if that makes sense to you, do you want to add an info message to that effect? (Not sure on the right priority, but a warning seems overkill).

That's in addition to fixing this specific pvp-bounds bug—the warning will stay useful. People will never need this piece of info if pvp-bounds has either bugs or bad interactions with other components.

[harendra-kumar]

When the --pvp-bound options is in effect it can say something like - "applying upper and lower bounds to dependencies...". In a more verbose mode it can tell exactly what bounds it applied to which packages.

[ChShersh]

I also can't upload to Hackage with any option for --pvp-bounds. My .cabal file doesn't contain dependencies at all but I see error:

Uploading universum-0.7.1.tar.gz... unhandled status code: 400
Error: Invalid package

The package has an extraneous version range for a dependency on an internal library: universum <0.8. This version range includes the current package but isn't needed as the current package's library will always be used.
Upload failed on universum-0.7.1.tar.gz

I have benchmarks in package which depend on library itself.

[mgsloan]

@ChShersh Ah, thanks for the more minimal repro. I think I've resolved this, but can't test the hackage upload. I've pushed a fix to master, can you please test after running stack upgrade --git? (or building from the stack repo)

[ChShersh]

@mgsloan Thanks for your work! Sorry for long answer...

I tested after stack upgrade --git. This didn't fix issue :(

I'm testing on this universum repository in master branch while trying to upload version 0.7.1.1.

Stack version

$ stack --version
Version 1.5.1, Git revision 600c1f01435a10d127938709556c1682ecfd694e (4861 commits) x86_64 hpack-0.17.1

Observed error

$ stack sdist --pvp-bounds upper
$ stack upload --pvp-bounds upper .
Getting file list for /home/fenx/programming/haskell/serokell/universum/
Building sdist tarball for /home/fenx/programming/haskell/serokell/universum/
Checking package 'universum' for common mistakes
Uploading universum-0.7.1.1.tar.gz... unhandled status code: 400
Error: Invalid package

The package has an extraneous version range for a dependency on an internal library: universum <0.8. This version range includes the current package but isn't needed as the current package's library will always be used.
Upload failed on universum-0.7.1.1.tar.gz

[mgsloan]

@ChShersh Hmm, did stack upgrade --git give a warning about it not being on the path? It looks like you are still using 1.5.1 but the HEAD version is 1.6

May be helpful to do which stack, and compare to where stack upgrade --git is installing it.

[ChShersh]

@mgsloan Sorry, false alarm! Yes, I did stack upgrade --git but seems like I needed to restart something to update used version of stack. Everything works now! 🎉

[mgsloan]

Awesome!

[mfine]

@mgsloan using stack sdist --pvp-bounds lower stripped my executable field to just main-is and removed my build-depends and default-language and ghc-options (and gives me cabal check warnings). Opened a new issue #3549.

$ Version 1.6.0, Git revision 64794699803ff997dd9a1ff999c342c73d99bb20 (5322 commits) x86_64
Compiled with:
- Cabal-2.0.0.2
- Glob-0.9.0
- HUnit-1.6.0.0
- MonadRandom-0.5.1
- QuickCheck-2.10.1
- StateVar-1.1.0.4
- aeson-1.2.2.0
- aeson-compat-0.3.7.1
- annotated-wl-pprint-0.7.0
- ansi-terminal-0.6.3.1
- ansi-wl-pprint-0.6.8.1
- array-0.5.2.0
- asn1-encoding-0.9.5
- asn1-parse-0.9.4
- asn1-types-0.3.2
- async-2.1.1.1
- attoparsec-0.13.2.0
- attoparsec-iso8601-1.0.0.0
- auto-update-0.1.4
- base-4.10.0.0
- base-compat-0.9.3
- base-orphans-0.6
- base-prelude-1.2.0.1
- base16-bytestring-0.1.1.6
- base64-bytestring-1.0.0.1
- basement-0.0.3
- bifunctors-5.4.2
- binary-0.8.5.1
- bindings-uname-0.1
- bitarray-0.0.1.1
- blaze-builder-0.4.0.2
- blaze-html-0.9.0.1
- blaze-markup-0.8.0.0
- byteable-0.1.1
- bytestring-0.10.8.2
- call-stack-0.1.0
- case-insensitive-1.2.0.10
- cereal-0.5.4.0
- clock-0.7.2
- comonad-5.0.2
- conduit-1.2.12.1
- conduit-extra-1.1.17
- connection-0.2.8
- containers-0.5.10.2
- contravariant-1.4
- cookie-0.4.2.1
- cryptohash-0.11.9
- cryptohash-sha256-0.11.100.1
- cryptonite-0.24
- cryptonite-conduit-0.2.0
- data-default-class-0.1.2.0
- deepseq-1.4.3.0
- digest-0.0.1.2
- directory-1.3.0.2
- distributive-0.5.3
- dlist-0.8.0.3
- easy-file-0.2.1
- echo-0.1.3
- ed25519-0.0.5.0
- either-4.4.1.1
- exceptions-0.8.3
- extra-1.6
- fail-4.9.0.0
- fast-logger-2.4.10
- file-embed-0.0.10.1
- filelock-0.1.1.2
- filepath-1.4.1.2
- foundation-0.0.16
- free-4.12.4
- fsnotify-0.2.1.1
- generic-deriving-1.11.2
- ghc-boot-th-8.2.1
- ghc-prim-0.5.1.0
- gitrev-1.3.1
- hackage-security-0.5.2.2
- hashable-1.2.6.1
- hastache-0.6.1
- hfsevents-0.1.6
- hourglass-0.2.10
- hpack-0.19.3
- hpc-0.6.0.3
- hspec-2.4.4
- hspec-core-2.4.4
- hspec-discover-2.4.4
- hspec-expectations-0.8.2
- hspec-smallcheck-0.4.2
- http-api-data-0.3.7.1
- http-client-0.5.7.0
- http-client-tls-0.3.5.1
- http-conduit-2.2.3.2
- http-types-0.9.1
- ieee754-0.8.0
- integer-gmp-1.0.1.0
- integer-logarithms-1.0.2
- lifted-base-0.2.3.11
- logict-0.6.0.2
- memory-0.14.8
- microlens-0.4.8.1
- microlens-mtl-0.1.11.0
- microlens-th-0.4.1.1
- mime-types-0.1.0.7
- mintty-0.1.1
- mmorph-1.1.0
- monad-control-1.0.2.2
- monad-logger-0.3.25.1
- monad-loops-0.4.3
- mono-traversable-1.0.2.1
- mtl-2.2.1
- neat-interpolation-0.3.2.1
- network-2.6.3.2
- network-uri-2.6.1.0
- old-locale-1.0.0.7
- old-time-1.1.0.3
- open-browser-0.2.1.0
- optparse-applicative-0.14.0.0
- optparse-simple-0.0.3
- parsec-3.1.11
- path-0.6.1
- path-io-1.3.3
- path-pieces-0.2.1
- pem-0.2.2
- persistent-2.7.0
- persistent-sqlite-2.6.2
- persistent-template-2.5.3
- pid1-0.1.2.0
- prelude-extras-0.4.0.3
- pretty-1.1.3.3
- primitive-0.6.2.0
- process-1.6.1.0
- profunctors-5.2.1
- project-template-0.2.0
- quickcheck-io-0.2.0
- random-1.1
- regex-applicative-0.3.3
- regex-applicative-text-0.1.0.1
- resource-pool-0.2.3.2
- resourcet-1.1.9
- retry-0.7.4.2
- rts-1.0
- safe-0.3.15
- scientific-0.3.5.2
- semigroupoids-5.2.1
- semigroups-0.18.3
- setenv-0.1.1.3
- silently-1.2.5
- smallcheck-1.1.2
- socks-0.5.5
- split-0.2.3.2
- stm-2.4.4.1
- stm-chans-3.0.0.4
- store-0.4.3.2
- store-core-0.4.1
- streaming-commons-0.1.18
- syb-0.7
- tagged-0.8.5
- tar-0.5.0.3
- template-haskell-2.12.0.0
- temporary-1.2.1.1
- text-1.2.2.2
- text-metrics-0.3.0
- tf-random-0.5
- th-abstraction-0.2.6.0
- th-expand-syns-0.4.3.0
- th-lift-0.7.7
- th-lift-instances-0.1.11
- th-orphans-0.13.4
- th-reify-many-0.1.8
- th-utilities-0.2.0.1
- time-1.8.0.2
- time-locale-compat-0.1.1.3
- tls-1.4.0
- transformers-0.5.2.0
- transformers-base-0.4.4
- transformers-compat-0.5.1.4
- unicode-transforms-0.3.3
- unix-2.7.2.2
- unix-compat-0.4.3.1
- unix-time-0.3.7
- unliftio-0.1.1.0
- unliftio-core-0.1.0.0
- unordered-containers-0.2.8.0
- uri-bytestring-0.3.0.1
- uuid-types-1.0.3
- vector-0.12.0.1
- vector-algorithms-0.7.0.1
- void-0.7.2
- x509-1.7.2
- x509-store-1.6.5
- x509-system-1.6.6
- x509-validation-1.6.9
- yaml-0.8.23.3
- zip-archive-0.3.1.1
- zlib-0.6.1.2

Warning: this is an unsupported build that may have been built with different
versions of dependencies and GHC than the officially release binaries, and
therefore may not behave identically.  If you encounter problems, please try
the latest official build by running 'stack upgrade --force-download'.

2017-11-03 17:48:41.655051: [info] Getting project config file from STACK_YAML environment
@(Stack/Config.hs:875:13)
2017-11-03 17:48:41.655718: [debug] Loading project config file stack-lts-6.yaml
@(Stack/Config.hs:910:13)
2017-11-03 17:48:41.657208: [debug] Decoding build plan from: /Users/mark/.stack/build-plan/lts-6.35.yaml
@(Stack/Snapshot.hs:151:5)
2017-11-03 17:48:42.401487: [debug] Using standard GHC build
@(Stack/Setup.hs:613:9)
2017-11-03 17:48:42.401780: [debug] Getting global package database location
@(Stack/GhcPkg.hs:46:5)
2017-11-03 17:48:42.402087: [debug] Asking GHC for its version
@(Stack/Setup/Installed.hs:98:13)
2017-11-03 17:48:42.402183: [debug] Getting Cabal package version
@(Stack/GhcPkg.hs:185:5)
2017-11-03 17:48:42.402232: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Log.hs:37:3)
2017-11-03 17:48:42.402290: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Log.hs:37:3)
2017-11-03 17:48:42.403322: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Log.hs:37:3)
2017-11-03 17:48:42.438317: [debug] Process finished in �[92m34ms�[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Log.hs:44:3)
2017-11-03 17:48:42.440323: [debug] Process finished in �[92m37ms�[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Log.hs:44:3)
2017-11-03 17:48:42.457822: [debug] Process finished in �[92m55ms�[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc --numeric-version
@(System/Process/Log.hs:44:3)
2017-11-03 17:48:42.457956: [debug] GHC version is: ghc-7.10.3
@(Stack/Setup/Installed.hs:102:13)
2017-11-03 17:48:42.458033: [debug] Resolving package entries
@(Stack/Setup.hs:252:5)
2017-11-03 17:48:42.458134: [debug] Trying to decode /Users/mark/.stack/loaded-snapshot-cache/x86_64-osx/ghc-7.10.3/lts-6.35.cache
@(Data/Store/VersionTagged.hs:66:5)
2017-11-03 17:48:42.474185: [debug] Success decoding /Users/mark/.stack/loaded-snapshot-cache/x86_64-osx/ghc-7.10.3/lts-6.35.cache
@(Data/Store/VersionTagged.hs:70:13)
2017-11-03 17:48:42.474573: [debug] Starting to execute command inside EnvConfig
@(Stack/Runners.hs:170:18)
2017-11-03 17:48:42.476566: [info] Getting file list for /Users/mark/repos/shakers/
@(Stack/SDist.hs:117:5)
2017-11-03 17:48:42.476889: [debug] Parsing the targets
@(Stack/Build/Target.hs:473:3)
2017-11-03 17:48:42.499228: [debug] Start: getPackageFiles /Users/mark/repos/shakers/shakers.cabal
@(Stack/PrettyPrint.hs:134:16)
2017-11-03 17:48:42.501159: [debug] Finished in 1ms: getPackageFiles /Users/mark/repos/shakers/shakers.cabal
@(Stack/PrettyPrint.hs:134:16)
2017-11-03 17:48:42.502752: [debug] Getting global package database location
@(Stack/GhcPkg.hs:46:5)
2017-11-03 17:48:42.502959: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Log.hs:37:3)
2017-11-03 17:48:42.532056: [debug] Process finished in �[92m29ms�[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Log.hs:44:3)
2017-11-03 17:48:42.532813: [debug] Run process: /Users/mark/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_1.22.5.0_ghc-7.10.3 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.5.0 sdist --list-sources /private/var/folders/wc/40h798sj7ln5gn8r6nmvqq880000gn/T/stack-sdist92904/source-files-list
@(System/Process/Log.hs:37:3)
2017-11-03 17:48:42.552440: [debug] Process finished in �[92m19ms�[0m: /Users/mark/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_1.22.5.0_ghc-7.10.3 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.5.0 sdist --list-sources /private/var/folders/wc/40h798sj7ln5gn8r6nmvqq880000gn/T/stack-sdist92904/source-files-list
@(System/Process/Log.hs:44:3)
2017-11-03 17:48:42.553129: [info] Building sdist tarball for /Users/mark/repos/shakers/
@(Stack/SDist.hs:119:5)
2017-11-03 17:48:42.553904: [debug] Parsing the targets
@(Stack/Build/Target.hs:473:3)
2017-11-03 17:48:42.576938: [debug] Start: getPackageFiles /Users/mark/repos/shakers/shakers.cabal
@(Stack/PrettyPrint.hs:134:16)
2017-11-03 17:48:42.577871: [debug] Finished in 0ms: getPackageFiles /Users/mark/repos/shakers/shakers.cabal
@(Stack/PrettyPrint.hs:134:16)
2017-11-03 17:48:42.578256: [debug] Finding out which packages are already installed
@(Stack/Build/Installed.hs:60:5)
2017-11-03 17:48:42.578378: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --global --no-user-package-db dump --expand-pkgroot
@(System/Process/Log.hs:37:3)
2017-11-03 17:48:42.609340: [debug] Process finished in �[92m30ms�[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --global --no-user-package-db dump --expand-pkgroot
@(System/Process/Log.hs:44:3)
2017-11-03 17:48:42.609671: [debug] Ignoring package haskeline due to wanting version 0.7.4.0 instead of 0.7.2.1
@(Stack/Build/Installed.hs:190:5)
2017-11-03 17:48:42.609738: [debug] Ignoring package terminfo due to wanting version 0.4.1.0 instead of 0.4.0.1
@(Stack/Build/Installed.hs:190:5)
2017-11-03 17:48:42.609812: [debug] Ignoring package Cabal due to wanting version 1.22.8.0 instead of 1.22.5.0
@(Stack/Build/Installed.hs:190:5)
2017-11-03 17:48:42.609959: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /Users/mark/.stack/snapshots/x86_64-osx/lts-6.35/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Log.hs:37:3)
2017-11-03 17:48:42.660918: [debug] Process finished in �[92m50ms�[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /Users/mark/.stack/snapshots/x86_64-osx/lts-6.35/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Log.hs:44:3)
2017-11-03 17:48:42.661517: [debug] Run process: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /Users/mark/repos/shakers/.stack-work/install/x86_64-osx/lts-6.35/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Log.hs:37:3)
2017-11-03 17:48:42.687239: [debug] Process finished in �[92m25ms�[0m: /Users/mark/.stack/programs/x86_64-osx/ghc-7.10.3/bin/ghc-pkg --user --no-user-package-db --package-db /Users/mark/repos/shakers/.stack-work/install/x86_64-osx/lts-6.35/7.10.3/pkgdb dump --expand-pkgroot
@(System/Process/Log.hs:44:3)
2017-11-03 17:48:42.694087: [info] Checking package 'shakers' for common mistakes
@(Stack/SDist.hs:359:5)
2017-11-03 17:48:42.694404: [warn] Package check reported the following warnings:
Packages using 'cabal-version: >= 1.10' must specify the 'default-language' field for each component (e.g. Haskell98 or Haskell2010). If a component uses different languages in different modules then list the other ones in the 'other-languages' field.
@(Stack/SDist.hs:370:9)
2017-11-03 17:48:42.695247: [info] Wrote sdist tarball to /Users/mark/repos/shakers/.stack-work/dist/x86_64-osx/Cabal-1.22.5.0/shakers-0.0.35.tar.gz
@(Stack/PrettyPrint.hs:63:22)