Make it easier to navigate from docs to API/schemas and follow $ref

[garethsb]

Navigating from the docs part of an NMOS spec to the relevant API/schemas, and between $ref-ed schemas isn't easy for readers on amwa-tv.gtihub.io at the moment. The ghpages renderings of the docs are great, but the flat listing of the schemas directory and the simple rendering of each schema (even though they're pretty printed) makes them harder to user. The fact that the RAML rendering doesn't handle $ref, often makes that less useful as well.

If we could render schemas so the reader could follow $ref that would help quite a bit. Adding some cross links from the docs would help too.

I don't know if we could use the "description" properties from the schemas in some other ways as well - e.g. a property index?

[peterbrightwell]

Raml2html doesn't show any signs of handling $refs in referenced schemas (after a failed attempt some time ago). Andrew pointed me to load_resolved_schema in the test suite, so I've now added this as a separate tool resolve-schema.py and modified the build scripts for nmos-template.

Effect is that the displayed schema in the RAML render includes the flattened $refs, and the schemas index gives the choice of flattened or with $refs.

If you can confirm this is ok, I'll add it to the other repos. IS-04 may be a bit more complex as it requires a different version of raml2html for the earliest versions. (Although it's less important to provide flattened schema for those now...)

[peterbrightwell]

Also done for IS-05 now. This can lead to some long schemas when flattened, e.g.:
https://amwa-tv.github.io/nmos-device-connection-management/tags/v1.1/html-APIs/ConnectionAPI.html#single_senders__senderid__constraints_get

[garethsb]

Good work trying out these different approaches. Hmm, I agree, the schemas probably end up too long if flattened and become hard to navigate for a different reason than before.

In my ideal scenario, I'd love to see each schema rendered unflattened as pretty-printed JSON with the $ref filenames made into links to the rendition of the referenced schema.

(Then in the API endpoint docs (RAML2HTML), the 'Type' could also just have the schema filename as a link to the schema, and just the examples embedded as now.)

[garethsb]

I knocked this up to show what I mean... (It's server-side C++, pretty-printing the JSON as HTML and linking the $refs live though, so I don't think the implementation is any help.)

[peterbrightwell]

Yes, that’s what I thought you meant, but wanted to try the flattened approach anyway as we had code. (As a byproduct of this Andrew has sorted installing python3 and python3-pip in the nodejs travis config). Another possibility might be to use something like http://caldwell.github.io/renderjson/ to render the flattened schemas to HTML a folding way. Possibly also merge that into the output of raml2html instead of the current schema view. P From: Gareth Sylvester-Bradley <notifications@github.com> Reply to: AMWA-TV/nmos-template <reply@reply.github.com> Date: Wednesday, 16 October 2019 at 13:19 To: AMWA-TV/nmos-template <nmos-template@noreply.github.com> Cc: Peter Brightwell <peter.brightwell@bbc.co.uk>, Comment <comment@noreply.github.com> Subject: Re: [AMWA-TV/nmos-template] Make it easier to navigate from docs to API/schemas and follow $ref (#3) I knocked this up to show what I mean... (It's C++, pretty-printing and linking the $refs live though, so I don't think the implementation is any help.) [Image removed by sender. SchemaRef]<https://user-images.githubusercontent.com/31761158/66918437-42f86580-f017-11e9-91f1-85856d09f3e2.gif> — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#3>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADV4FSFYNZI375Q5WRDGAULQO4BD7ANCNFSM4I3GJKOA>.

[peterbrightwell]

Tried experimenting with renderjson but it's quite ugly. A colleague modified json-formatter.js -- which is used in a chrome extension -- to let us specify a list of paths to show collapsed, e.g. the properties.fec_... paths below. Needs more work but could be a nice way forward:

[peterbrightwell]

Now rendering the schema and example pages using json-formatter.js. More work needed on getting the folding parameters sensible for our requirements, but better than before.

Still need to tackle what is shown in the RAML render -- which may just be a case of combining the outputs of raml2html and json-formatter.

[peterbrightwell]

Did the above for is-05 also, so you can see it on longer json docs.

[peterbrightwell]

Decided that trying to embed the folding JSON->HTML render into to the modal boxes in the RAML render wasn't going to be difficult (at least for now). Instead have replaced the embedded schema (but not the example) with a link to the folding schema page.

E.g. https://amwa-tv.github.io/nmos-device-connection-management/tags/v1.1/html-APIs/ConnectionAPI.html#single_receivers__receiverid__constraints_get

@garethsb-sony what do you think? Should we try harder to get the folding schemas into the modal view? Or go to a non-modal rendering such as https://developer.officernd.com/#offices? Or something else?

[garethsb]

I think that's a big improvement personally.

String values seem not to be wrapped, so the description values disappear off the right side of even my very wide browser window.

It might be nice to have cross-links between the flattened and unflattened versions? (I like that the Type in the rendered RAML links to the flattened/resolved version.)

Thanks, Pete, I think this is definitely worth having. 👍

[peterbrightwell]

Added those cross-links, e.g.
https://amwa-tv.github.io/nmos-template/branches/v1.0-dev/html-APIs/schemas/with-refs/exampleapi-base.html

https://github.com/richski has added long line wrap to his fork of json-formatter, and we've also managed to demonstrate folding within the modal, but I'll need to modify the output of raml2html to make that useful.

[peterbrightwell]

Luckily raml2html has themes, so I've done a raml2html-nmos-theme (which may get its own repo at some point) and put Rich's folding code in there, so don't need to modify raml2html itself.

See https://amwa-tv.github.io/nmos-template/branches/v1.0-dev/html-APIs/TemplateAPI.html#_get

We can use themes for other things in the future also :)

[peterbrightwell]

Did same for IS-05 (e.g. https://amwa-tv.github.io/nmos-device-connection-management/tags/v1.1/html-APIs/ConnectionAPI.html#single_senders__senderid__constraints_get)

This shows long line wrapping, and I also added a build date and time stamp.

Hopefully this won't break the nightly CI builds, but lets see...

[peterbrightwell]

Doing this led to discovering AMWA-TV/is-07#64. Including linting in the CI will help this in future (I've made a start with .ci-scripts/lint.sh), but it does now mean that broken schemas in already-released versions will now fail to render. I could work round this by not doing the folding render where linting fails. This will need a bit of work though, so how important is it?

[garethsb]

This seems to be mostly working really well now, so I suggest closing this issue and adding follow-up ones for any little extensions. 👍 Thanks, Pete!

[peterbrightwell]

Closing as suggested