Proposal: start generating authoritative API reference docs

[ahmetb]

Currently, we don't have API reference documentation.

For serving API we have spec.md which I've been able to find multiple errors in it, and it's a manual maintenance.

For build API, we also have a hand-written informal syntax at builds.md.

---

• Kubernetes API reference docs generated with gen-apidocs which uses /swagger.json or /apis paths exposed on Kubernetes API server

• Istio API reference docs are generated with protoc-gen-doc codegen plugin because Istio provides their api as protos

Knative doesn't fit either (we can use k8s /apis endpoint yet because it doesn't expose CRD OpenAPI specs).

So I propose we start generating these things from the source code (i.e pkg/apis at each repo) and import them to the knative/docs repo after a release happens.

I'll work on something right away if there are no objections.

/area API
/kind docs

[RichieEscarez]

There are also these godocs:
https://godoc.org/github.com/knative/build
https://godoc.org/github.com/knative/eventing
https://godoc.org/github.com/knative/eventing-sources
https://godoc.org/github.com/knative/serving

[ahmetb]

Oh yeah, those exist for Kubernetes or Istio, too. But they aren't necessarily one of the more legible documentation of the API. They're often cluttered with lots of method declarations. I mostly want something similar to Kubernetes API ref docs.

[ahmetb]

Please take a look at these prototypes:

• build: https://gist.github.com/ahmetb/158d0b58a132330d926a7a9e65cd0d85
• serving: https://gist.github.com/ahmetb/bf321e7a1aff24fa36c2f60d4e080d18

I generated these with some custom code I hacked (until we can use k8s api docs generator when k8s api begins providing openapi specs for crds).

If there's agreement, I'd like to send these .md files as pull requests and maintain this for a few releases, then hand-off as parts of release process.

[evankanderson]

This looks great! Any notes you have about how to perform this process would be great, because I was just starting to take a look at this as well. Three items: 1. Can we strip off the <br/> tags that the tool is generating for longer comments? 2. Can we remove or format differently the "+optional" comments at the end of the docstring? 3. I *really* like the inlining of the spec in the object definition. Can we do the same with status? If the modified tooling is available somewhere, I'd be happy to take a swing at making these changes.

…

On Wed, Jan 2, 2019 at 9:06 AM Ahmet Alp Balkan ***@***.***> wrote: Please take a look at these prototypes: - build: https://gist.github.com/ahmetb/158d0b58a132330d926a7a9e65cd0d85 - serving: https://gist.github.com/ahmetb/bf321e7a1aff24fa36c2f60d4e080d18 I generated these with some custom code I hacked (until we can use k8s api docs generator when k8s api begins <kubernetes/kubernetes#71192> providing openapi specs for crds). If there's agreement, I'd like to send these .md files as pull requests and maintain this for a few releases, then hand-off as parts of release process. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#636 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHlyNyV1GnGBo0P4lD-7TnyGxX11-_oiks5u_OcPgaJpZM4ZHKtM> .

-- Evan Anderson <argent@google.com>

[ahmetb]

@evankanderson I documented the process at https://github.com/ahmetb/gen-knative-api-reference-docs so you should be try on your machine.

1. Can we strip off the
   tags that the tool is generating for longer comments?

I tried replacing only \n\n with a <br/><br/> which separated paragraphs. It looked better except when we used new lines to separate list items in code. But it may be a good trade-off.

Can we remove or format differently the "+optional" comments at the end of the docstring?

Weird enough most fields marked as +optional actually aren't. I assume that's an api machinery thing. But yeah we should be able to strip off those with simple find & replace.

3. I really like the inlining of the spec in the object definition. Can we do the same with status?

Yeah. I was already doing inlining for spec and status, but then reduced only to spec to make the output doc shorter (the XxxStatus types are also listed in the doc separately).

It's done here:
https://github.com/ahmetb/gen-knative-api-reference-docs/blob/8f51b5cf7cf66bf282e9bc2ae19f9c9012a4b301/template/members.tpl#L33

I thought of inlining the relevant fields like container, too. But they are kinda huge and make the entire doc hard to navigate.

[evankanderson]

Yes, I thought that the single level of inlining was probably the right balance.

…

On Wed, Jan 2, 2019 at 9:42 AM Ahmet Alp Balkan ***@***.***> wrote: @evankanderson <https://github.com/evankanderson> I documented the process at https://github.com/ahmetb/gen-knative-api-reference-docs so you should be try on your machine. 1. Can we strip off the tags that the tool is generating for longer comments? I tried replacing only \n\n with a <br/><br/> which separated paragraphs. It looked better except when we used new lines to separate list items in code. But it may be a good trade-off. Can we remove or format differently the "+optional" comments at the end of the docstring? Weird enough most fields marked as +optional actually aren't. I assume that's an api machinery thing. But yeah we should be able to strip off those with simple find & replace. 1. I *really* like the inlining of the spec in the object definition. Can we do the same with status? Yeah. I was already doing inlining for spec and status, but then reduced only to spec to make the output doc shorter (the XxxStatus types are also listed in the doc separately). It's done here: https://github.com/ahmetb/gen-knative-api-reference-docs/blob/8f51b5cf7cf66bf282e9bc2ae19f9c9012a4b301/template/members.tpl#L33 I thought of inlining the relevant fields like container, too. But they are kinda huge and make the entire doc hard to navigate. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#636 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHlyN_dsCo2EqrPK_GAUMEpHU3PN-VFEks5u_O-VgaJpZM4ZHKtM> .

-- Evan Anderson <argent@google.com>

[ahmetb]

@samodell @RichieEscarez any ideas where should we put content like these? I propose something like the following which lets us keep multiple versions of the API spec revisioned simultaneously:

docs/
  serving/
    api/
      v0.2/
        api-reference.md
      v0.3/
        api-reference.md
  build/
      ...
  eventing/
      ...

[ahmetb]

Here's eventing docs for the sake of completeness:

• eventing: https://gist.github.com/ahmetb/00163327d073880c51d03e5f3b063145#file-eventing-md
• event-sources: https://gist.github.com/ahmetb/00163327d073880c51d03e5f3b063145#file-event-sources-md

[RichieEscarez]

We intend to version the docs by branch (TBD - some time after we get a draft of the docs site up).

Therefore the structure would be more along the lines of:

knative/docs:

master branch (active development of v.next):
    docs/
        serving/
            api/
                api-reference.md

v0.2.x branch:
    docs/
        serving/
            api/
                api-reference.md

v0.3.x branch:
    docs/
        serving/
            api/
                api-reference.md

FYI: per the Docsy/Hugo template for the in-development docs site, I will need to next everything under two additional directories:

knative/docs

GitHub Branch:
    content/
        en/
            docs/.......

example in my repo (All still in-progress/testing so can and might change)

[ahmetb]

@RichieEscarez the reason I suggested keeping API ref versions checked in and revisioned altogether is you can link to each API version within the same site (i.e. not a different subdomain).

Kubernetes currently keeps it all in the same branch https://github.com/kubernetes/website/tree/578b006360c85812dbe909dc56cedad5985a69b0/static/docs/reference/generated/kubernetes-api and this helps go back and fix issues like broken parts (by simply regenerating with the "fixed" generator etc).

[ahmetb]

@RichieEscarez another benefit is that you can link your friends to a particular version much more easily like

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.12/
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/

and have the URL clearly speak for itself as for which version is being shown.