A metadata system for global objects + pretty markdown parsing and display (see the readme). For example:
... flowing rivers of technical poetry ...
function foo() ...
Documenting multiple methods of a function will concatenate the doc strings together, in the order the methods were originally defined, and this is overloadable.
By default @doc treats unadorned strings as equivalent to md"" (i.e. markdown syntax) but you can actually put anything before the arrow to associate it with the function.
Docs can be queried with e.g.
@doc @doc # For macros
(and you can try these out on this branch – the repl help key ? will work too)
@doc doc"..." ->
It seems to me that not all the TODOs need to be done before we merge this. It would be great if we can have something that exists along with the existing help for a bit, while we transition and fix everything and deprecate the existing system.
I agree completely – the TODOs are more general things that would be nice to have eventually, but sure, this is basically usable as is.
I found that it was quite easy to just parse the old helpdb.jl file, put its contents into the new help storage and have the new help-system displaying 99% of what there was before:
I suggest merging this and then taking it from there.
Merging something (or the threat of merging something) does seem like the best way to make progress.
I'm a little concerned about packages starting to use this in a way that doesn't degrade gracefully, but I can get over that fear for the sake of making forward progress. Go for it.
This is exactly what we want to sort out in the middle of the dev cycle. @one-more-minute Please merge when you are ready.
We may want to start using in a few packages and then improve from there.
Perhaps squash some of the commits too?
I would say squashing isn't necessary for merging this. (If there were intermediate breakages I might think differently, but that's a lot of green checkmarks above.)
make "flavour" slightly less readable
implement Docs module
support for macros
update to track Markdown.jl/julia-breaking branch
support for methods
implement Text and HTML types
improve error messages
give each module its own META
support one-line syntax
support for types
fixes for deprecations in Base
Get the repl to use Docs primarily
That's OK, I was due to learn some git rebasing anyway. The history should be a lot cleaner now.
Ok, try ?@doc at the REPL and then get going writing some great docs ;)
update lint of at-doc syntax
Without a doubt, this is one of the best things about julia 0.4. 🍰
@timholy How can that be? It isn't even mentioned in NEWS.md.
[I hope the irony is obvious]
I love the description of the foo function in @doc @doc. I always wondered what the main purpose of that function was. 😃
added NEWS for #8791
@one-more-minute do you happen to know how much of http://spec.commonmark.org/0.11/ you've covered so far? (If the intention is to follow that spec?)
Great, this was one of the mostly missed features for me. One question: does the first item in the todo list mean we will get rid of the -> at some point?
Edit: I see now this is the point of #8965, never mind.
@MichaelHatherly Definitely, although most likely we'll have some documentation-specific extensions such as LaTeX and tables. It'd be great to get a test suite for CommonMark running, though.
Basically, at the moment basic stuff (headers, emphasis, links) is well supported, but more unusual things like nested lists / the lazy rule for block quotes aren't. It should be fine for basic uses, though.
Markdown.jl is meant to be easy to hack on and extend – if you're interested, PRs would be very welcome, and I'll make sure Base stays in sync with the main repo.
@one-more-minute great, I'll probably get a chance to in a few weeks time.
Awesome, can't wait to see what you come up with
This is very exciting. Thank you all for pushing this through.
I guess now is a good time to bring up the matter of standardizing headers. For comparison, R has the following ...
@milktrader, see #8966
@StefanKarpinski Glad to help!
I added some really basic documentation examples in basedocs.jl – hope that's OK. It's not meant to be final in any way so much as to get something going, and give people something to try out in the repl (e.g. you can do ?fft, @time, ?r"" etc. now) to get a feel for it.
I'm using it over in https://github.com/mauro3/Traits.jl, works a treat!
That's great to see 👍
One of the really sweet things @MichaelHatherly added to Lexicon was the ability to dispatch on help. I confess that due to other commitments I haven't followed this effort at all; is that functionality supported?
I actually just added this (3c2e2c4):
julia> @doc "foo1" foo() = 1
foo (generic function with 1 method)
julia> @doc "foo2" foo(x) = 2
foo (generic function with 2 methods)
Ok, I see there's also the idea of having a searching version of @which, which is pretty neat. How well does it work? I'd definitely be interested in integrating something like that.
I haven't tested it extensively (I'm only using Docile/Lexicon in a subset of my private code tree, basically the stuff that was developed after these packages became available). But everything I've tried so far suggests it's the cat's meow.
Specifically, I was wondering about this for ? or @doc, not for @which (although implementing for @which would be awesome too).
I only mention @which because it seems like a good idea to implement a general @whiches (or whatever) which returns the array of methods which fuzzy-match – then adding support to ? in the repl should be pretty easy.
That's only an implementation detail, though – I'm definitely on board with fuzzy searching for docs in general. @MichaelHatherly So you have this to working in Docile/Lexicon already?
The query(::Function, (DataType...)) method in Lexicon does left-to-right partial matching, since I found doing an unordered ⊆ match on the type signatures typically gave more results than was helpful. The @query macro does exact method matching in the same way as @which.
Having a more intelligent fuzzy-matcher could be quite nice. Not sure what it would look like though.
Pushing forward on completing the transition - it seems that the next step is to transition help to the new documentation system.
Does this mean that our standard library documentation should be translated to Markdown first to leverage the new system best? This may not work best with readthedocs - does it have to use rst?
I think readthedocs requires RST, yes. That said, we already have our own docs template and website, so I'm not sure readthedocs really buys us that much – @MichaelHatherly has done a lot of work on generating great documentation pages and might be interested in helping to move away from it entirely.
OTOH, it wouldn't be hard to generate RST manual pages from the inline docs using Markdown.jl, so that's another option.
The important thing is to have some way to share content between Base docs and the manual – otherwise there'll be a lot of repeated work.
There's also http://www.mkdocs.org/, which @tshort has been using recently (https://tshort.github.io/Sims.jl/) in conjunction with Docile.
I was thinking the same about readthedocs. The manual and Base docs should just be equally accessible, but they don't share much of anything. They can be done in different ways without impacting anything, I feel.
mkdocs looks really cool!
Embedded Docile/Base 0.4 docs can be used with Mkdocs and published on either readthedocs or GitHub Pages. It's really easy.
Github Pages works great with Mkdocs. I haven't tried it, but one advantage of readthedocs is it provides online searching.