New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

`stack hoogle` command #55

Closed
chrisdone opened this Issue May 24, 2015 · 28 comments

Comments

Projects
None yet
6 participants
@chrisdone
Member

chrisdone commented May 24, 2015

I want this! It should:

  1. Run hoogle with the right database according to the stack config in the directory in which it's run (so it's easily ran from a terminal or by IDEs).
  2. Automatically download the hoogle database from stackage.org e.g. https://www.stackage.org/snapshot/lts-2.1/db.hoo for lts-2.1.
  3. Put it in some reasonable place like ~/.stack/hoogle-databases/lts-2.1.hoo.
  4. Or when running inside docker, use the hoogle database provided by the docker.

The last docker point means we probably need to sync up some things.

With this command I could provide some nice Emacs support with a live hoogle search buffer, and I'd basically never have to leave Emacs to open my browser to read docs again. This closes off two big pain points for me: (1) always having to open the browser and leave my editor environment, (2) needing an internet connection to view docs. This will also basically make my whole haskell-docs package redundant which I'll be able to deprecate in favor of stack.

Pinging @manny-fp and @snoyberg for feedback/opinions.

@chrisdone chrisdone self-assigned this May 24, 2015

@chrisdone

This comment has been minimized.

Member

chrisdone commented May 24, 2015

Second thoughts, seems hoogle's -d expects a directory. So it'd probably be better as:

~/.stack/hoogle-databases/lts-2.1/lts-2.1.hoo

This has some potential for putting other databases in there, such as ones generated from your local projects, which could be neat for megarepos.

@snoyberg snoyberg modified the milestone: Second release May 25, 2015

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented May 29, 2015

We should probably be careful about doing too much work with Hoogle 4, when I want to push forward on Hoogle 5, which changes most of this stuff quite dramatically. It's a bit of an awkward middle situation though.

@snoyberg snoyberg modified the milestones: Third release, Second release, Later improvements, First stable release (0.1.0.0?) Jun 3, 2015

@borsboom borsboom modified the milestones: P2: Should, P3: Optional Oct 7, 2015

@snoyberg snoyberg assigned borsboom and unassigned chrisdone Oct 14, 2015

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Nov 1, 2015

I've just pushed something to Hoogle about 10 patches before the minimum-viable product, but it's enough to see where the holes are. Given Hoogle HEAD, you can do hoogle generate --local and it will use ghc-pkg to figure out which packages are installed and where there docs are, then hoogle map and it will find map or hoogle server and it will launch a server at localhost which you can search from. Clicking links will still go to the remote docs, but it is based on the local Hoogle info.

When doing that under Stack there are 122 packages, but only 23 have docs (those shipped with GHC), because Stack doesn't include docs by default. To make stack hoogle work reasonably then stack hoogle would have to generate docs for the immediate dependencies of a module (at least), either always as it was going, or when you typed stack hoogle.

@borsboom

This comment has been minimized.

Contributor

borsboom commented Nov 1, 2015

Cool! stack build --haddock (aka stack haddock) already generates docs incrementally (i.e. it will build them for dependencies that don't already have them), so doing that implicitly before stack hoogle (with an option to skip for the impatient) would probably work.

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Nov 1, 2015

That works as a solution - it's just a shame that you have to reconfigure all the packages and that it's so slow. Also it fails on Windows, but I'm sure that's fixable: #1266.

@borsboom

This comment has been minimized.

Contributor

borsboom commented Nov 3, 2015

One thing that will help with this is that we're planning to support settings defaults for build options on the configuration file. That means you could configure Stack to build haddocks for dependencies by default.

It may also be possible to implement just adding haddocks to a dependency without having to rebuild the library. I looked into this a while back and it seemed feasible, but tricky to integrate into the current build plan construction/execution pipeline.

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Nov 9, 2015

Given things in their current state, if stack hoogle did:

  • Build Hoogle from https://github.com/ndmitchell/hoogle (easy with #1278)
  • stack haddock (and ideally, if that worked better, see #1317)
  • stack exec -- hoogle generate --local --database=.stack-work/hoogle
  • stack exec -- hoogle $ARGS --database=.stack-work/hoogle

Then we'd have enough that someone could do stack hoogle filter or stack hoogle server and it would work.

chrisdone added a commit that referenced this issue Mar 22, 2016

@chrisdone

This comment has been minimized.

Member

chrisdone commented Jun 27, 2016

Implemented! Made a document here for anyone who wants to try it out: https://gist.github.com/chrisdone/78c1c1c052a853697cf3b902de086172

@chrisdone chrisdone closed this Jun 27, 2016

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Jun 27, 2016

I tried it last night, works nicely. Any chance of a stack hoogle --server which runs stack hoogle -- --server --local? I find that a very nice way to use Hoogle via Stack.

@chrisdone

This comment has been minimized.

Member

chrisdone commented Jun 27, 2016

@ndmitchell That makes sense to me!

@chrisdone

This comment has been minimized.

Member

chrisdone commented Jun 27, 2016

@ndmitchell Made an issue here: #2310

@krisajenkins

This comment has been minimized.

krisajenkins commented Jun 27, 2016

Works perfectly for me. 👍

@creswick

This comment has been minimized.

creswick commented Apr 10, 2018

I think I may be missing something... this doesn't seem to index the local packages. e.g.:

$ stack hoogle -- -i myLocalFunctionNameThatIsIndeedExported
No results found.
$ stack hoogle -- map
Prelude map :: (a -> b) -> [a] -> [b]
...

All the local packages haddock successfully. This should index the local packages, right?

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Apr 10, 2018

Try stack hoogle -- is:package which should give you a good idea of which packages Hoogle has indexed. Is it your dependencies? Or all of Stackage?

@creswick

This comment has been minimized.

creswick commented Apr 10, 2018

@ndmitchell it looks like it's indexed our local packages, and many (likely all) of our dependencies:

$ stack hoogle -- is:package --count=900000 | wc
272 544 5208

Grepping that out put for our packages also shows matches, (our packages are in that list)

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Apr 10, 2018

What about grepping for the package that exports myLocalFunctionNameThatIsIndeedExported?

@creswick

This comment has been minimized.

creswick commented Apr 10, 2018

I did do that, and it is in the list.

That function is exported from local package Foo, which is in the list, and used by package Bar, which is also in the list.

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Apr 10, 2018

If it's in the list suggest searching for package:Foo a and see what comes up. Is it a case of not indexing anything in Foo? Or just that this particular function is not being indexed for some reason.

@creswick

This comment has been minimized.

creswick commented Apr 10, 2018

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Apr 10, 2018

Don't know... If you look at the haddock output you might find a file Foo.txt in that directory or one up. Is there output in the Haddock docs?

@creswick

This comment has been minimized.

creswick commented Apr 10, 2018

@creswick

This comment has been minimized.

creswick commented Apr 11, 2018

Ok, I think anyone can reproduce this in the following way:

$ git clone https://github.com/creswick/chatter.git
$ cd chatter
$ stack setup
$ stack hoogle -- -i trainStr
<snipped lots of output>
No results found
$ stack hoogle -- -i map
map :: (a -> b) -> [a] -> [b]
...

Chatter haddock's successfully, and defines trainStr :: Tag t => POSTagger t -> String -> IO (POSTagger t) in NLP.POS, and exports that module in chatter.cabal

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Apr 11, 2018

I get:

C:\Neil\chatter>stack hoogle -- -i trainStr
trainStr :: Tag t => POSTagger t -> String -> IO (POSTagger t)
chatter NLP.POS
Train a tagger on string input in the standard form for POS tagged
corpora:


trainStr tagger "the/at dog/nn jumped/vbd ./."

I am on 64 bit Windows with:

C:\Neil\chatter>stack --version
Version 1.6.5, Git revision 24ab0d6ff07f28276e082c3ce74dfdeb1a2ca9e9 (5514 commits) x86_64 hpack-0.20.0

C:\Neil\chatter>hoogle --version
Hoogle 5.0.17.2, http://hoogle.haskell.org/

It's possible my Hoogle is actually Hoogle HEAD (I don't really remember what/when I compiled it), so if the latest versions don't work, try Hoogle HEAD.

@creswick

This comment has been minimized.

creswick commented Apr 11, 2018

@creswick

This comment has been minimized.

creswick commented Apr 12, 2018

Ahha! This does work with:

  • Stack 1.6.5
  • Ghc 8.4.1
  • Hoogle head (a49b83cde500d250c81afc5edb90cbe416517c8a or newer)
@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Apr 13, 2018

Not sure if the latest Hoogle would have worked or not, but I'll upload a new one just to be sure (once Hackage recovers). With that I suggest we close this issue?

@creswick

This comment has been minimized.

creswick commented Apr 13, 2018

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Apr 17, 2018

I just released Hoogle 5.0.17.3 which has all the changes from HEAD, so should work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment