RFC: Use doxygen for generating API docs

[jonas]

Hi,

To address issue #26, I've started working on converting the header file so doxygen can be used to generate API docs. It's not complete yet, hence the "RFC" prefix. I would however like to know if the general direction is good. A couple of things to consider:

• The comments are quite invasive, especially for the #define statements. If it is preferable, the comments could go before and simply reference the symbols they document.
• I'm using markdown as much as possible to keep the comments readable.
• Comments follow the initial formatting of utf8proc (recent comments didn't follow this):

/** Some title.
 *  Note the two spaces before the main text in the comment.
 */

• @return and @param are used, although it requires that the comments are restructured.

You can see the docs here: http://jonas.nitro.dk/utf8proc/html/index.html
With the important part being: http://jonas.nitro.dk/utf8proc/html/utf8proc_8h.html

I've also briefly looked at whether to use Sphinx for generating docs through the breathe plugin. This might allow us to create better looking docs. I don't know if that would be of interest, else I also found a way to use bootstrap with doxygen to make it look more modern.

[ I'm considering using utf8proc in tig so I am very excited that you have taken over maintaining this great library. ]

[jonas]

Forgot to mention that I didn't integrate anything into the build system for now.

[tkelman]

Cool! This is probably a case of "he who implements it gets to decide what tool to use," but this looks not bad at all. Would breathe allow us to host everything on readthedocs? Judging by http://cppformat.readthedocs.org/en/stable/ as an example, that does look a lot nicer than the doxygen default.

[jiahao]

I'm quite used to doxygen, so I wouldn't mind.

I would make a few small changes:

@@ -242,7 +242,7 @@ TCL_SUBST              =
 # members will be omitted, etc.
 # The default value is: NO.

-OPTIMIZE_OUTPUT_FOR_C  = NO
+OPTIMIZE_OUTPUT_FOR_C  = YES

 # Set the OPTIMIZE_OUTPUT_JAVA tag to YES if your project consists of Java or
 # Python sources only. Doxygen will then generate output that is more tailored
@@ -793,7 +793,7 @@ RECURSIVE              = NO
 # Note that relative paths are relative to the directory from which doxygen is
 # run.

-EXCLUDE                =
+EXCLUDE                = NEWS.md lump.md

 # The EXCLUDE_SYMLINKS tag can be used to select whether or not files or
 # directories that are symbolic links (a Unix file system feature) are excluded

[jonas]

@tkelman Yes, as I understand it, breathe allows doc hosting on readthedocs. I didn't actually try to set it up yet. In any case, Doxygen is required to "lift" the docs from source.

@jiahao Thanks. Was not sure whether the external files made sense.

[stevengj]

Maybe the #define statements should be replaced by enums?

Thanks for doing this; improving the documentation was the last thing I wanted to do before releasing utf8proc-1.2, and I agree that Doxygen is the natural choice here.

There is no particular reason to stick with the original commenting style of utf8proc, since we are now the upstream — it would be better to use whatever is conventional for Doxygen.

[stevengj]

the (native-endian UTF-32) unicode codepoints to re-encode

[jonas]

Thanks for the comments. I've tried to address the suggestions in the additional commits. I kept the changes separate so that they can easily be included or dropped in a final squashed commit.

Regarding the enum's it really cleans up the docs a lot. I had to use a typedef for the categories to not conflict with the utf8proc_category method and decided to also use a typedef for the options to keep things consistent.

[stevengj]

typedef enum { .... } utf8proc_bidi_class_t for consistency?

[stevengj]

Great, can you update your doc page so that we can see what it looks like?

[jonas]

@stevengj It's updated.

[stevengj]

What is the docs directory for? The generated docs? I'm not sure we should distribute those in the tarball as opposed to just posting them.

[nalimilan]

Distributions prefer shipping docs with packages rather than only online. That's required some work in Julia, so if it works here please keep them.

[jonas]

It goes with this line in Doxyfile:

+OUTPUT_DIRECTORY       = docs

It contains docs/html, docs/man etc.

[stevengj]

Looks good, can you squash the commits for merging?

[jonas]

@stevengj Squashed. I also added 'Fix #26' to the commit title.

[stevengj]

Now, the next steps:

• How do we autogenerate the docs for the github-created tarballs?
• How do we autogenerate the docs for readthedocs or similar, so that the online docs are auto-updated in sync with github?

[nalimilan]

No idea about the second point, but as regards GitHub-generated tarballs, I think they're best avoided. Usually, projects have a make dist or make distcheck target which builds everything that is needed an creates the tarball. A good make distcheck tries to build the resulting tarball and run the tests from it, so that you can be sure you didn't forget a file in the release, or some silly thing like this.

I'm not saying this is very high priority, but that's what I see as a good practice.

[jonas]

Once we've added a `make docs` and `make dist`, we can look into configuring Travis to build release assets and upload them to GitHub: http://docs.travis-ci.com/user/deployment/releases/ As for posting docs to readthedocs I can look into it. It looks we need to install a webhook to notify ReadTheDocs and then add some configuration files to run Doxygen and Sphinx: https://docs.readthedocs.org/en/latest/webhooks.html#github https://breathe.readthedocs.org/en/latest/readthedocs.html