Add automatic API-doc generation & deployment #183

zbeekman · 2020-05-14T21:43:50Z

EDIT: Preview the documentation here: https://fortran-lang.github.io/stdlib-docs/index.html
NOTE: Autodeploy failed, not sure if it's due to this being a PR from a fork, or if ${{ secrets.ACTIONS_DEPLOY_KEY }} was never setup, or if it is incompatible with cross-repository deployment.

PR to automatically generate and deploy documentation using FORD published via GH pages.

Notes:

I think we should move the specs to a dedicated directory, and then they can be rendered by FORD.
It would be nice to deploy these to something like stdlib.fortran-lang.org.
- I can make this happen but we need a DNS entry so I can add a CNAME file in the deployed gh-pages repo CC: @milancurcic @certik
May want to move the WORKFLOW.md and STYLEGUIDE.md documents into the FORD pages dir

For now this PR is a WIP.

Begins to address issues:

milancurcic · 2020-05-14T22:01:48Z

Thanks, Zaak! I think this is a great first step forward.

Pending others' reviews, let's move this forward and I can tackle the DNS record. I agree stdlib.fortran-lang.org is a good subdomain for this.

After this is merged, then the GH actions will push to stdlib-docs repo. Then we can simply enable it to serve at the default location (fortran-lang.github.io/stdlib-docs), check that it looks good, then we can set up the CNAME record.

.github/workflows/doc-deployment.yml

zbeekman · 2020-05-14T22:10:49Z

After this is merged, then the GH actions will push to stdlib-docs repo. Then we can simply enable it to serve at the default location (fortran-lang.github.io/stdlib-docs), check that it looks good, then we can set up the CNAME record.

Yeah, that was my thinking too. Currently it will deploy aggressively because I want to check that it's all working. (Assuming the ACTIONS_DEPLOY_KEY secret is properly set and I haven't mucked something up in the workflow file. I think @certik set it for me a while ago, but someone with admin should double check.)

zbeekman · 2020-05-14T22:16:07Z

API-doc-FORD-file.md

+source: true
+proc_internals: true
+md_extensions: markdown.extensions.toc
+graph: true


It currently takes ~20 minutes to generate the graphs. We may decide to disable this. We could set this to false and then manually use the entity metadata to enable a subset of graphs.

MAXRANK could be limited to 3. However, this will only marginally reduced the time. Because more procedures are expected, disabling the automatic generation of the graphs seems to be needed.

When building during CI waiting ~20 minutes isn't the end of the world, IMO. I've added parameters to prune/limit the depth and nodes per graph. I think it's OK to leave this in the workflow, but could be convinced otherwise.

Is this now done for every PR that gets submitted to stdlib?

20 minutes is not the end of the world, but it significantly slows down the development.

Do you know why it takes so long?

Do you know why it takes so long?

I guess that the issue is the generation of the graphs for all generated (hundreds?) functions in the different submodules.

We can also limit MAXRANK to 3 as suggested by @jvdp1. Should speed things up, and folks can extrapolate to higher rank code.

@zbeekman do you know how to restrict the rank? As it is now, all tests finish quickly except this documentation test that takes forever. Anything we can do to speed it up would help, at least in our initial stages. As we turn into production (we move functionality from experimental to main), we can revisit this.

@zbeekman do you know how to restrict the rank? As it is now, all tests finish quickly except this documentation test that takes forever. Anything we can do to speed it up would help, at least in our initial stages. As we turn into production (we move functionality from experimental to main), we can revisit this.

@certik Yes: MAXRANK=<n> here Do you want me to submit a PR restricting API documentation to public and protected entities only, reducing the MAXRANK and turning off search index and graph building unless documentation is going to be deployed? Also, what RANK should we restrict to? 2 or 3?

If you could do that, that would be awesome. Can search index + graph building fail? If so, then we have to test it. Last thing we want is to fail at deploy.

Regarding rank, I don't know, try 3 and if it is fast enough, let's keep it.

@certik

Can search index + graph building fail?

I've never seen search indexing fail and I don't think I've ever seen graph building fail, but this one I'm more nervous about.

zbeekman · 2020-05-14T22:16:58Z

API-doc-FORD-file.md

+project: Fortran-lang/stdlib
+summary: A community driven standard library for (modern) Fortran
+src_dir: src
+exclude_dir: src/tests


I excluded the tests from the documentation. We could include them if we wish but processing takes longer and it might dilute the API documentation with other stuff.

I agree with not including the tests in the API documentation. Furthermore, specs include some examples for each procedure.

zbeekman · 2020-05-14T22:17:35Z

doc/index.md

@@ -0,0 +1,3 @@
+---
+title: Specs, examples & user documentation
+---


This section needs expanding/sign-posting.

zbeekman · 2020-05-14T22:17:55Z

doc/specs/index.md

+title: specs
+---
+
+@todo Explain what these are, how to write them, why they're needed, etc.


This section also probably needs expanding/signposting.

zbeekman · 2020-05-14T22:19:00Z

doc/specs/stdlib_experimental_error.md


+[TOC]


The anchor/backlink syntax seems to be different in python markdown, and we have the automatic TOC extension enabled, so I decided to use that rather than manually maintain a TOC.

- Also, see if redeployment will work now

certik

That looks good to me as a start, thanks @zbeekman!

Milan, can you work with Zaak to get the dns setup and merge this PR when it's time?

zbeekman · 2020-05-14T22:43:10Z

@milancurcic or @certik (or anyone else with full admin) can you please check and confirm that the ACTIONS_DEPLOY_KEY secret is properly set?

The deployment failed, but I'm not 100% sure why. If needed I can switch to a personal token.

zbeekman · 2020-05-14T22:49:31Z

@milancurcic or @certik (or anyone else with full admin) can you please check and confirm that the ACTIONS_DEPLOY_KEY secret is properly set?

The deployment failed, but I'm not 100% sure why. If needed I can switch to a personal token.

I think this is due to this being a PR from a fork because secrets are not available in this context. Would it be OK to push this branch to the main repo and try the PR again to confirm deployment will work?

certik · 2020-05-14T23:08:13Z

I pushed it in #184. Let me know if you have rights to push into that branch.

zbeekman · 2020-05-15T00:17:50Z

Thanks @certik. I found a good work around for testing purposes: I added the secret to my fork which let me deploy successfully. If CI turns green here I'm happy to merge.

jvdp1

Thank you @zbeekman for this. It is a really good start! Really appreciated! Some things need to be completed (and it will come time), but now we can mention this website to potential users of stdlib.

jvdp1 · 2020-05-15T06:22:33Z

API-doc-FORD-file.md

+source: true
+proc_internals: true
+md_extensions: markdown.extensions.toc
+graph: true


MAXRANK could be limited to 3. However, this will only marginally reduced the time. Because more procedures are expected, disabling the automatic generation of the graphs seems to be needed.

jvdp1 · 2020-05-15T06:22:44Z

API-doc-FORD-file.md

+project: Fortran-lang/stdlib
+summary: A community driven standard library for (modern) Fortran
+src_dir: src
+exclude_dir: src/tests


I agree with not including the tests in the API documentation. Furthermore, specs include some examples for each procedure.

jvdp1 · 2020-05-15T06:25:33Z

doc/specs/stdlib_experimental_error.md


+[TOC]


API-doc-FORD-file.md

A was missing Co-authored-by: Jeremie Vandenplas <jeremie.vandenplas@gmail.com>

milancurcic · 2020-05-15T17:43:39Z

@zbeekman The CI is green and I will now merge this. Thank you!

zbeekman added 3 commits May 14, 2020 17:36

Add automatic API-doc generation & deployment

1d4e930

Fix bad action name

09bc276

Move specs into FORD pages specs dir

ded1f63

milancurcic requested review from scivision, certik, cmacmackin and jvdp1 May 14, 2020 22:01

Update specs w/ title meta data and automated TOC

6c9c2bb

zbeekman commented May 14, 2020

View reviewed changes

.github/workflows/doc-deployment.yml Outdated Show resolved Hide resolved

zbeekman commented May 14, 2020

View reviewed changes

doc/index.md

@@ -0,0 +1,3 @@

---

title: Specs, examples & user documentation

---

Copy link

Member Author

zbeekman May 14, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section needs expanding/sign-posting.

zbeekman commented May 14, 2020

View reviewed changes

Add note about navbar to FORD API doc landing page

5e9c5e7

- Also, see if redeployment will work now

certik approved these changes May 14, 2020

View reviewed changes

Add favicon

b22b6e6

@info -> @note & shorten pages titlebar name

703c17e

certik mentioned this pull request May 14, 2020

Zb documentation #184

Closed

zbeekman added 3 commits May 14, 2020 19:49

Try resolving error by removing username

0348e36

Add-back branch/tag deploy guard

ffe5f50

Ensure that forks don't push to docs repo

7fc3e21

jvdp1 reviewed May 15, 2020

View reviewed changes

API-doc-FORD-file.md Outdated Show resolved Hide resolved

Fix typo found by @jvdp1

bc26bf1

A was missing Co-authored-by: Jeremie Vandenplas <jeremie.vandenplas@gmail.com>

milancurcic merged commit ce987d2 into fortran-lang:master May 15, 2020

This was referenced May 18, 2020

Consolidate specification docs into a master specification doc #175

Open

Documentation #4

Closed

zbeekman deleted the zb-documentation branch May 19, 2020 13:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add automatic API-doc generation & deployment #183

Add automatic API-doc generation & deployment #183

zbeekman commented May 14, 2020 •

edited

Loading

milancurcic commented May 14, 2020

zbeekman commented May 14, 2020

zbeekman May 14, 2020

jvdp1 May 15, 2020

zbeekman May 15, 2020

certik May 15, 2020

jvdp1 May 15, 2020

zbeekman May 18, 2020

certik May 18, 2020

zbeekman May 18, 2020

certik May 18, 2020

zbeekman May 19, 2020

zbeekman May 14, 2020

jvdp1 May 15, 2020

zbeekman May 14, 2020

zbeekman May 14, 2020

zbeekman May 14, 2020

jvdp1 May 15, 2020

certik left a comment

zbeekman commented May 14, 2020

zbeekman commented May 14, 2020 •

edited

Loading

certik commented May 14, 2020

zbeekman commented May 15, 2020

jvdp1 left a comment

jvdp1 May 15, 2020

jvdp1 May 15, 2020

jvdp1 May 15, 2020

milancurcic commented May 15, 2020


		[TOC]


		[TOC]

Add automatic API-doc generation & deployment #183

Add automatic API-doc generation & deployment #183

Conversation

zbeekman commented May 14, 2020 • edited Loading

milancurcic commented May 14, 2020

zbeekman commented May 14, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

certik left a comment

Choose a reason for hiding this comment

zbeekman commented May 14, 2020

zbeekman commented May 14, 2020 • edited Loading

certik commented May 14, 2020

zbeekman commented May 15, 2020

jvdp1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

milancurcic commented May 15, 2020

zbeekman commented May 14, 2020 •

edited

Loading

zbeekman commented May 14, 2020 •

edited

Loading