New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Online API browser #53

Closed
Hywan opened this Issue Feb 14, 2017 · 34 comments

Comments

@Hywan
Copy link
Member

Hywan commented Feb 14, 2017

Hello fellow @hoaproject/hoackers and users!

I would like to have a nice online API browser for our libraries. Ready?

Intro

So far, we have tried http://api.hoa-project.net/, which is the result of Doxygen output.

Is it a solution we could maintain? Do we have better alternatives? What do you expect from an API browser?

Solutions for PHP

Personally, I don't like any of them.

Solutions in other languages

Build our own?

I really like the approach of rustdoc, the style, the fact that the code must be extremely documented, and the API is not the first-citizen. Look at Tagua VM, the https://tagua-vm.github.io/parser/ page represents the documentation of the tagua-parser library.

For each class, we have a long description. Then we have modules (namepaces in PHP) with a description.

For each function (for instance parse), we have a description, and an example. The API is even not documented. Well, this discussion is part of the RFC #58.

But the browsing experience is great.

Outro

All API documentation generator I tried in PHP didn't catch me. In other languages, like Rust or Elm, we have very nice ideas. We must get inspired from them.

I am not afraid of building a new tool. I personally thing this is the direction to take.

Thoughts?

@Hywan Hywan added this to the Roadmap for 2017 milestone Feb 14, 2017

@Hywan Hywan added the enhancement label Feb 15, 2017

@Hywan Hywan self-assigned this Feb 15, 2017

@vonglasow

This comment has been minimized.

Copy link
Member

vonglasow commented Feb 16, 2017

Have you tried apidoc ?

@1e1

This comment has been minimized.

Copy link
Contributor

1e1 commented Feb 16, 2017

Quick and dirty answer:
How can it be filled by Praspel and typed functions (new in PHP7)?
Can we use Atoum for extracting some examples?
I will enjoy if the generator need no more comments.

@1e1

This comment has been minimized.

Copy link
Contributor

1e1 commented Feb 16, 2017

Remember the code is the best documentation!
Notice: in an online API browser we are mainly looking for public methods.

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Feb 17, 2017

@vonglasow APIDoc is for Rest API, no?
@1e1 We need to expose all methods, whatever the visibility they have, you don't know who is using your libray and for what. For the rest, I didn't understand what you meant, sorry :-/.

@1e1

This comment has been minimized.

Copy link
Contributor

1e1 commented Feb 23, 2017

This API browser would not add extra comments for generating the documentation.
There is enough constraints from other tools. It should use these constraints as inputs.

@Pierozi

This comment has been minimized.

Copy link
Member

Pierozi commented Mar 7, 2017

As RFC #52 will produce high quality and dynamic content
We should be able to extract Markdown from code and generate HTML verison.

Maybe http://www.mkdocs.org could be useful?

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Mar 7, 2017

MKDocs looks nice, but I am afraid this is too much oriented for documentation, and not API documentation, aka code-oriented. What do you think? Do you think it will be difficult to bend MKDocs for an API browser?

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Mar 16, 2017

Current implementation https://github.com/Hywan/Kitab.

@harikt

This comment has been minimized.

Copy link

harikt commented Mar 17, 2017

It looks like you love parsing the docblocks. I have came across lithium which does something similar. Not a standalone one. But it has some plugin when installed your whole framework docs all will be shown.

The plugin is : https://github.com/UnionOfRAD/li3_docs

The documentation written in class is as

https://github.com/UnionOfRAD/lithium/blob/5d9a06b8667e20490ef5ccb3275be8824f77666b/action/Controller.php#L15

once it is rendered you will see something as : http://li3.me/docs/api/lithium/1.1.x/lithium/action/Controller .

I guess this is what you are looking for. I looked at Kitab, but as it has less information how the docs should be written etc I am not very sure also.

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Mar 17, 2017

@harikt The format is defined in RFC #58. The output is not exactly what we want. We are looking for something closer to what rustdoc does.

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Mar 17, 2017

Yesterday progressions

A namespace page

kitab_1

On the left, we have the navigation, which is “all the sibling data”. You must understand it as “namespaces in the same level”.

On the center, we see one namespace in details, with all entities it contains: Classes, interfaces, traits, and functions.

A class page

kitab_2

Today progression

The class page has been enhanced with more data. Markdown is converted into HTML for the doc comment attached to methods. We have syntax highlighting. Markdown to HTML does not auto-convert entity names (like class names) into links yet. This is expected in a close future.

screen shot 2017-03-17 at 10 02 55-fullpage

The class documentation will be placed just before the “Methods” heading.

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Mar 21, 2017

Today progression

And now we have a static search engine
screen shot 2017-03-21 at 11 12 47

It is kind of slow when we have more than 600 files to analyse, but I am working on a cache (the slow part is to build the search index, we can cache it, but I don't know how to orchestrate that).

Anyway, it works great 🙂. I have decided to write it with Elm, because why not.

Here is the Search.elm module.

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Mar 24, 2017

Great news! The search data are now compiled ahead-of-time. What does it mean? 3 files are now computed:

  1. search-database.json, contains all the search information (never used by the client-side, never loaded, only needed for the compilation step, moved to the frontend just for debugging),
  2. search-index.js, contains the serialized and compressed index, it is compiled ahead-of-time by Kitab, the client only unserializes it and here we go,
  3. search-metadata.js, contains metadata useful to print results to the user, like URL, real symbol name, and description.

Having the index and the metadata separated really helps to reduce the size of the index (by a factor of 65% with my recent tests).

Why doing this? To reduce the initialisation of the search on the client-side. Now, initialisation of the search drops from 17s to… 369ms 🎉 with an index representing 606 files. Here are more numbers:

Description Kitab only Kitab and its dependencies
Number of PHP files to analyse 29 606
Time to compile everything 0.25s 6.57s
Size of search-database.json 5.4Kb 149Kb
Size of search-index.js 4.4Kb 199Kb
Size of search-metadata.js 3.9Kb 117Kb
Time to initialize the search on client-side ? 369ms

I am very happy with these results. Not to say that search-index.js and search-metadata.js are easily cachable with a 3xx HTTP code. They are purely static.

The size of the search module search.elm.js is 92Kb. So for Kitab only, the client loads 100.3Kb, and for Kitab and its dependencies, the client loads 408Kb, once.

We have a complete reactive UI with an advanced search for 600+ files for 400Kb. This is still less than Ember or Angular alone —just the framework, not the app— (source).

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Apr 26, 2017

Interface support for the HTML target are better.
Class and interface constants are supported by both the IR and the HTML target (including constant visibility, and default values, for inline or grouped form, resp. const A; const B; or const A, B;), see Hywan/Kitab@ad3a722.

screen shot 2017-04-26 at 15 29 47

@Grummfy

This comment has been minimized.

Copy link
Member

Grummfy commented Apr 27, 2017

Seems really nice. Just a nice feature, for future usage outside Hoa could be a way to +/- compile it to other template style/design

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Apr 27, 2017

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented May 22, 2017

Progression summary:

  1. Attributes are supported (Hywan/Kitab@9f37627),
  2. Introduce namespace description, see below please (Hywan/Kitab@9474b34 and Hywan/Kitab@c6b8f3e),
  3. Better global/unnamespaced entities support (Hywan/Kitab@b61fce8),
  4. Functions are supported (Hywan/Kitab@1c753b1),
  5. Links to source have been added, please see below (Hywan/Kitab@7fa57a1 and some others).

Note about namespace description. Any classes, interfaces, traits, or functions can have a block comment attached to it, in a single place. However, for namespaces, we have many locations where to write the documentation. Because this is not obvious and undeterministic, I have decided to look for a README.md file in the directory representing the namespace —with regards to the PSR-4 standard. If such a file exists, then we use it as the namespace description. Thus, I have introduced a new --with-composer option to read the PSR-4 mappings from a composer.json file. Thoughts? My first idea was to add a new special Description.md, but using README.md makes a lot of sense: It describes the content of the current directory, and it is displayed as HTML on Github, and Gitlab, which is very useful too.

About links to source, it is attached to each entities (classes, interfaces, traits, functions, and even methods).

Here is an online example https://hywan.github.io/Kitab/kitab/index.html of https://github.com/Hywan/Kitab. On this documentation homepage, you can see the README.md of the src/ directory. Below that, you have the Classes and Namespaces Sections. I have found a bug just right now, so links to namespaces are broken, but here is the link to the Kitab\Compiler namespace, https://hywan.github.io/Kitab/kitab/compiler/index.html, and here is the documentation for the Kitab\Compiler\Compiler class, https://hywan.github.io/Kitab/kitab/compiler/Compiler.html. You can see the Attributes, and Methods Section. All the [src] links are working. For example, links to the compile method, https://hywan.github.io/Kitab/kitab/compiler/Compiler.source.html#L.91-106. The static search engine is also working.

Please, fill bugs or discuss here :-). Feedbacks are really appreciated!

@Grummfy

This comment has been minimized.

Copy link
Member

Grummfy commented May 22, 2017

can you add an example where you use php 7 synthaxe with return, scalar type, etc.

Otyherwise nothing special to say it seems really nice

@Hywan

This comment has been minimized.

@Grummfy

This comment has been minimized.

Copy link
Member

Grummfy commented May 22, 2017

Well in this case, I would expect that "Type" or "File" would be clickable because thier are class that exist inside the documentation. For that's the only expected things that I see.

As extra information/plugins, I really enjoy to have stuff like phpxref to cross check the usage. But not directly accessible.

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented May 22, 2017

As extra information/plugins, I really enjoy to have stuff like phpxref to cross check the usage. But not directly accessible.

Can you give an example please?

@Grummfy

This comment has been minimized.

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented May 22, 2017

This a tree explorer, yes, and… ? I don't understand what you are asking :-).

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented May 29, 2017

Bug in namespace links is now fixed, https://hywan.github.io/Kitab/kitab/index.html.

Next steps:

  • Being able to configure the logo,
  • Add a index.html file at the root of the project to redirect to a particular namespace index (becase https://hywan.github.io/Kitab/ does not link to anything, we have to choose the kitab directory),
  • Make more code “clickable”, like class-like types in parameters,
  • Add page titles.

After that, we will be good to try a 1.0 beta, and deploy it on Hoa.

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented May 30, 2017

Live: https://hywan.github.io/Kitab/kitab/compiler/Compiler.html.

screen shot 2017-05-30 at 12 39 04

And here we go, since Hywan/Kitab@1950f62, all qualified names are clickable. I still have an issue with extends that are not linkable, a small conflict with Prism (c-like profile).

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Jun 7, 2017

Progression summary

I find interesting how to document things without saying: “Return bla bla” or “Get bla bla”, because the name of the function or method must be self-explanatory! So taking the example of inNamespace in https://hywan.github.io/Kitab/kitab/compiler/intermediaterepresentation/Entity.html, I used the following description:

An entity is in a namespace if its fully-qualified name contains at least one namespace separator (aka \).

Actually, this is the only explanation we expect at this step. We know the method is called inNamespace so saying something like: “Check if the name is in a namespace” is totally stupid.

This is interesting to rethink how we write documentation.

@Pierozi

This comment has been minimized.

Copy link
Member

Pierozi commented Jun 7, 2017

Really nice, I like the Namespace page related to Readme, a good one 👍

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Jun 7, 2017

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Jul 4, 2017

And now, Kitab can test your examples with ./bin/kitab test. More at https://github.com/Hywan/Kitab/pull/3.

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Jul 4, 2017

What's next?

  • Kitab can be used as a dependency.
  • Documenting everything inside Kitab.
  • Try to use Kitab with a non-PSR-4 project.
  • Move Kitab into the Hoa organisation.
  • Release the 1.0 version.
@brzuchal

This comment has been minimized.

Copy link

brzuchal commented Jul 5, 2017

None of the links to generated documentation doesn't work. Could you provide working one?

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Jul 5, 2017

Yes, we are migrating the repository into the Hoa organisation since this morning. Give me some hours to fix everything :-).

@Hywan Hywan referenced this issue Aug 17, 2017

Open

steps for next version - bc break #74

0 of 5 tasks complete
@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Nov 21, 2017

@Hywan

This comment has been minimized.

Copy link
Member

Hywan commented Nov 21, 2017

The 0.10 version has been tagged https://github.com/hoaproject/Kitab/releases. The version is stable. We use it at work.

It is implemented inside hoa/test. We will start to deploy the documentation on the hoa-project.net site soon.

@Hywan Hywan closed this Dec 29, 2017

@wafflebot wafflebot bot removed the in progress label Dec 29, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment