Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nixos/haskell: Added quickjump option to the haskell mkDerivation #75942

Closed
wants to merge 7 commits into from

Conversation

@mgttlinger
Copy link
Contributor

@mgttlinger mgttlinger commented Dec 19, 2019

Undestroyed version of #75939

Motivation for this change

When visiting local documentation via hoogle, currently for most packages the quickjump index is missing so you only get a sad error when pressing "s" to search in the current documentation.

Things done

Tested building my system including pandoc, xmonad, xmonad-contrib (and all their deps) from the new tree and checked that now quickjump works.

Marking as WIP until the whole tree has been built as requested in the comments.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@mgttlinger
Copy link
Contributor Author

@mgttlinger mgttlinger commented Dec 20, 2019

@cdepillabout Any hints on where to look regarding the issue of the missing lsp-test that ofborg complains? This prevents me from starting my own CI build of all packages.

@cdepillabout
Copy link
Member

@cdepillabout cdepillabout commented Dec 20, 2019

@mgttlinger Ah, I'm pretty sure that the lsp-test thing is unrelated to this PR, and instead caused by one of the normal haskell package set updates. You should be able to ignore it for now.


Also, I thought about this a little more, and it might be asking too much to have you run this against all unbroken packages.

Why don't you take this off "draft" and we'll see what @peti has to say about merging it.

@mgttlinger
Copy link
Contributor Author

@mgttlinger mgttlinger commented Dec 20, 2019

I set a decent machine as CI runner who will compile all packages over the weekend (hopefully) so that should be fine.

How should I ignore that lsp-test thing? It prevents your script from evaluating?

@mgttlinger mgttlinger marked this pull request as ready for review Dec 20, 2019
@cdepillabout
Copy link
Member

@cdepillabout cdepillabout commented Dec 20, 2019

How should I ignore that lsp-test thing? It prevents your script from evaluating?

Oh, I see.

If you wanted to figure out what the problem is, you could grep for where lsp-test_0_8_2_0 is used somewhere in pkgs/development/haskell/ and fix it so it is no longer broken.


The haskell-updates branch gets automatic updates to Haskell packages from Hackage. For packages that are in Stackage LTS, but also have a later version Hackage, two versions of the package get created. So the normal lsp-test package is probably the normal version from Stackage LTS, while a later derivation matching the latest version on Hackage is also created. It appears to used to be named `lsp-test_0_8_2_0.

What probably happened is that the automatic updates to haskell-updates created a more recent version of lsp-test, probably called something like lsp-test_0_9_0_0 or something.

What you can do is to find where lsp-test_0_8_2_0 is incorrectly being used and update it to use lsp-test_0_9_0_0. This will hopefully work.

If you find that this fixes everything, we'd really appreciate if you send a PR to the haskell-updates branch with your fix.

These types of problems happen every once in a while, and we are really grateful with any help we can get fixing things.

@mgttlinger
Copy link
Contributor Author

@mgttlinger mgttlinger commented Dec 20, 2019

Ok I fixed that in #76087 but now I am unable to build all packages given that something which is not marked broken apparently depends on something broken.

@cdepillabout
Copy link
Member

@cdepillabout cdepillabout commented Dec 20, 2019

I am unable to build all packages given that something which is not marked broken apparently depends on something broken.

I think I understand what is going on, but unfortunately I don't have any good ideas on ways to fix this for you.


For any given package, you might be able to discard it from your list of packages to test if one of it's transitive dependencies is marked broken.

This might require a substantial change to the script I wrote though.

@peti peti force-pushed the NixOS:haskell-updates branch from 6484351 to 1f79abf Dec 20, 2019
@cdepillabout
Copy link
Member

@cdepillabout cdepillabout commented Dec 21, 2019

@peti Do you want to make a call on whether or not to merge this in?

I'm slightly in favor of merging it in as-is, and fixing up any breakage later.

@peti peti force-pushed the NixOS:haskell-updates branch 2 times, most recently from 1f7024e to 7d2cc64 Dec 26, 2019
peti added a commit that referenced this pull request Dec 27, 2019
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

Closes #75942.
@peti
Copy link
Member

@peti peti commented Dec 27, 2019

Merged in 14f29c4.

@peti peti closed this Dec 27, 2019
@peti
Copy link
Member

@peti peti commented Dec 27, 2019

The Hydra evaluation https://hydra.nixos.org/eval/1562201 will show what effect this change has on the package set.

@peti
Copy link
Member

@peti peti commented Dec 27, 2019

The change breaks builds with ghc-8.4.4, presumably because the older compiler ships a version of haddock that does not support this flag. See the Hydra build logs I mention above for more details. We cannot merge the change as-is, I'm afraid. There needs to be some version guard added to the patch.

@peti peti reopened this Dec 27, 2019
@mgttlinger
Copy link
Contributor Author

@mgttlinger mgttlinger commented Dec 27, 2019

I see. Any pointers howto accomplish such a version check? I'll try to fix this tomorrow if I manage to figure out how to do that.

@peti peti force-pushed the NixOS:haskell-updates branch 2 times, most recently from aa46cdd to fdd64fb Dec 27, 2019
This update was generated by hackage2nix v2.15.0-11-g2fb6f7e from Hackage revision
commercialhaskell/all-cabal-hashes@5ccecb7.
@cdepillabout
Copy link
Member

@cdepillabout cdepillabout commented Dec 28, 2019

@peti Thanks for creating an evaluation on Hydra for this. It was easy to see what ended up breaking.


@mgttlinger If you look in the generic-builder.nix file, you can see a couple different version checks for the ghc version.

Here's one you could use as an example:

, enableHsc2hsViaAsm ? stdenv.hostPlatform.isWindows && stdenv.lib.versionAtLeast ghc.version "8.4"

(You'll of course be checking for 8.6 instead of 8.4 though)

peti added 3 commits Dec 29, 2019
This update was generated by hackage2nix v2.15.0-11-g2fb6f7e from Hackage revision
commercialhaskell/all-cabal-hashes@bb9a7bf.
This update was generated by hackage2nix v2.15.0-9-g650d530 from Hackage revision
commercialhaskell/all-cabal-hashes@dc5b0e7.
This update was generated by hackage2nix v2.15.0-9-g650d530 from Hackage revision
commercialhaskell/all-cabal-hashes@b2a082b.
@mgttlinger
Copy link
Contributor Author

@mgttlinger mgttlinger commented Dec 30, 2019

@peti @cdepillabout Anything else I should do here or is this good now?

@cdepillabout
Copy link
Member

@cdepillabout cdepillabout commented Dec 31, 2019

@mgttlinger Could you instead add the condition on the doHaddockQuickJump function argument?

That will make it easier for people to override (in case they are doing something tricky like using a recent version of haddock with an older compiler). It is also similar to how a lot of the other arguments work.

Once you make this change, I will test it out.

@mgttlinger
Copy link
Contributor Author

@mgttlinger mgttlinger commented Dec 31, 2019

@cdepillabout
Copy link
Member

@cdepillabout cdepillabout commented Dec 31, 2019

@mgttlinger Thanks!

Using this PR, I built the docs for conduit, and the doc-index.json file (which I believe is created with the --quickjump option) was created as expected:

$ nix-build -A haskellPackages.conduit.doc
/nix/store/p6jlyqnxshc85z1d6a8vgkil3aw3my2a-conduit-1.3.1.2-doc
$ find /nix/store/p6jlyqnxshc85z1d6a8vgkil3aw3my2a-conduit-1.3.1.2-doc/ -name doc-index.json
/nix/store/p6jlyqnxshc85z1d6a8vgkil3aw3my2a-conduit-1.3.1.2-doc/share/doc/conduit-1.3.1.2/html/doc-index.json

I then tried building dlist with ghc-8.4.4 and it appears to not have the doc-index.json file:

$ nix-build -A haskell.packages.ghc844.dlist.doc
/nix/store/djihj1a0aimi79ahnlwny02rww6vf3d6-dlist-0.8.0.7-doc
$ find /nix/store/djihj1a0aimi79ahnlwny02rww6vf3d6-dlist-0.8.0.7-doc -iname doc-index.json
$ # no results

@peti I think this should be good to merge in, but you might want to just cherry-pick the three relevant commits:

@peti peti force-pushed the NixOS:haskell-updates branch from 319711e to 18e33d3 Jan 5, 2020
peti added a commit that referenced this pull request Jan 5, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
peti added a commit that referenced this pull request Jan 5, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
@peti peti force-pushed the NixOS:haskell-updates branch from 700b7e0 to c8b3888 Jan 5, 2020
peti added a commit that referenced this pull request Jan 5, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
@peti peti force-pushed the NixOS:haskell-updates branch from c8b3888 to 4799433 Jan 5, 2020
peti added a commit that referenced this pull request Jan 5, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
@peti peti force-pushed the NixOS:haskell-updates branch from 4799433 to d5b2d9e Jan 5, 2020
peti added a commit that referenced this pull request Jan 5, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
@peti peti force-pushed the NixOS:haskell-updates branch from d5b2d9e to 5de6f10 Jan 5, 2020
peti added a commit that referenced this pull request Jan 5, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
@peti peti force-pushed the NixOS:haskell-updates branch from 5de6f10 to 23eb5a9 Jan 5, 2020
peti added a commit that referenced this pull request Jan 5, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
@peti peti force-pushed the NixOS:haskell-updates branch from 23eb5a9 to 24ebeb9 Jan 5, 2020
peti added a commit that referenced this pull request Jan 5, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
@peti peti force-pushed the NixOS:haskell-updates branch from 24ebeb9 to 3ab6627 Jan 5, 2020
@mgttlinger
Copy link
Contributor Author

@mgttlinger mgttlinger commented Jan 8, 2020

Should I fix the conflicts with the main branch or is that unneeded given that it seems the relevant changes have already been included in the above force pushed commits?

@peti
Copy link
Member

@peti peti commented Jan 8, 2020

No need to fix anything. Thanks!

@peti peti closed this Jan 8, 2020
peti added a commit that referenced this pull request Jan 10, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
peti added a commit that referenced this pull request Jan 10, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
peti added a commit that referenced this pull request Jan 10, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
peti added a commit that referenced this pull request Jan 10, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
peti added a commit that referenced this pull request Jan 10, 2020
When visiting local documentation via hoogle, currently for most packages the
quickjump index is missing so you only get a sad error when pressing "s" to
search in the current documentation.

The quickjump option is only supported by the haddock utility that's shipped
with ghc 8.6.x or later.

Closes #75942.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.