Documentation generator for Jsonnet

[davidzchen]

Since we are storing comments in the AST, we can make use of this to build a documentation generator for Jsonnet, such as for documentation for functions.

[davidzchen]

One thought about this:

Currently, we support both Python-style comments (#) and C/C++/JavaScript-style comments (// and /* */). While Jsonnet is a superset of JSON, which derives from JavaScript, we also borrow from Python quite a lot.

However, inline documentation in Python documentation generators is very different than those for languages with C-family syntaxes. For example, Python docstrings use a special string literal """ (see epydoc documentation). On the other hand, inline documentation for C-family languages and JavaScript use special syntax within comments (see Doxygen and various JavaScript documentation generators).

Jsonnet has its own block string syntax (|||) similar to Python's multi-line string literal. However, from what I understand, that might not be very suitable for inline documentation. Comments seem to be the better approach, and since we are already storing comments in the AST, generating documentation from them should be relatively straightforward.

If we are generating documentation from comments, I propose that we adopt an existing syntax, such as the following which is one of the most common used by JavaScript, Java, and many C++ projects:

{
  /**
   * Returns whether the string a is prefixed by the string b.
   *
   * @param a The input string.
   * @param b The prefix.
   * @return true if string a is prefixed by the string b or false otherwise.
   */
  startsWith(a, b):
    if std.length(a) < std.length(b) then
      false
    else
      std.substr(a, 0, std.length(b)) == b,
}

On a similar note, I am also thinking that it would be better to only support either Python-style or C/C++-style comments rather than both in order to keep the language as simple and code as consistently as possible, but that can be a separate discussion.

[sparkprime]

Yeah the javadoc style I think is what we want. I guess we'll document files, whatever the top-level object is, and if the top-level object is an object then all of the fields of that object (recursively if they are objects and so on).

I have a feeling documentation is mainly important for the public API of a file so documenting locals would not be necessary. I.e., you can put comments on them, but they wouldn't make it into generated HTML.

The benefit of having all 3 comment types is easier copy-pastability from various languages whose syntax significantly overlaps with Jsonnet.

[davidzchen]

Agreed. Generating documentation only makes sense for public APIs, such as the functions provided by the Jsonnet stdlib.

Can you elaborate on the copy-pastability use case? Is this mainly for transitioning from other configuration languages to Jsonnet?

[mikedanese]

Right now the stdlib documentation has a subset of the functions that are actually in the stdlib. The html also does not support linking to specific headers as far as I can tell. Solving this would be great for discoverability of stdlib apis.

[sparkprime]

I'd divide this into 2 parts -- a first part that scans a given Jsonnet file, and emits JSON describing all the documented features of the file, by doing a simple (trivial really) static analysis to identify the features, finding the appropriate comment for that feature, and parsing the content of the comment according to javadoc syntax (and perhaps some analysis of the files).

And then a second part that generates HTML from that JSON.

Bonus points if there is a standard format (doesn't have to be JSON) that already exists for this intermediate form, especially if there is a variety of existing renderers for that format.

[roidelapluie]

Can we get Markdown instead of HTML?

note: I am maintaining a jsonnet lib (http://github.com/grafana/grafonnet-lib)

[sbarzowski]

@roidelapluie Do you mean markdown as an output format?

[roidelapluie]

Yes.

[sparkprime]

Probably a good first step either way

…

On Nov 8, 2017 3:55 PM, "Julien Pivotto" ***@***.***> wrote: Yes. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#122 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKQ3jetfdcJATQv-3wY6w6flNTDway-ks5s0hVCgaJpZM4HsNNX> .

[pranavgupta1234]

Is there any API currently present using which we can access comments written inside jsonnet ?

[ghostsquad]

I'm very interested in this. Especially as a way to increase adoption, since it can be hard to understand what functions/args are needed for libraries, and all the existing IDE add-ons to do auto complete seem to have stalled.

[sbarzowski]

I think @sparkprime has some progress on this from the code analysis side. I was recently looking into building html for doc pages (with Jsonnet), with the intention of using that for stdlib first, and later for other projects as well if it goes well.

[legovaer]

Just bumped into this issue. I wanted to let you know that I'm currently working on a tool that is parsing jsonnet files in order to extract doc blocks the same way as the PHP Docblock tool works. I assume that I will be able to release an initial version of this tool later this week.

It only parses the file, in a second stage I will add a Markdown generator and make sure that it's easy to change the generator in order to support other formats in the future.

[sbarzowski]

@legovaer Cool! Please let us know when you have something ready. I'm very interested.

It only parses the file, in a second stage I will add a Markdown generator and make sure that it's easy to change the generator in order to support other formats in the future.

You may want to consider generating JSON, which is then consumed to produce markdown/html/pdf/whatever. This way the analysis and the presentation are completely decoupled.

[legovaer]

Hi all, I just released the first alpha version of my tool called JDP (Jsonnet Docblock Parser). Please feel free to review & test it as much as possible. Make sure that you create new issues for feature requests and/or bugs.