API Reference documentation

[fvanroie]

By popular request I have documented the API calls for inclusion in the OPNsense documentation. This is the first draft, but should be a good starting point as there is currently no comprehensive list of API calls available.

The list is based on the source code and auto-generated by a script. I have reviewed it and use the same list already in my project. I present this PR as a starting point for discussing the API documentation and I'm open to any comments.

[fichtner]

Awesome, thanks for this! ❤️

Handing over to @AdSchellevis for review/merge.

[AdSchellevis]

@fvanroie do you mind if we split the file and move it around in the menu structure? I think this is a good starting point. I can merge this in a make some suggestions, if you're willing to keep working on it.

[fvanroie]

No problem @AdSchellevis. I figured the How-to's location wasn't the best place for it. As long as the page(s) retains the csv-table data format, I will be able to parse, re-import and update it without any problems.

Ideally the API reference should be updated on new PRs in source, but I know from experience this gets overlooked quite easily. I have some PowerShell scripts that do the heavy lifting already, so it's not much effort on my part. If anyone is interested, I don't mind sharing that too.

[AdSchellevis]

@fvanroie I've moved all to a top level section "API reference" in the development area (d9f2b20)

I don't use PowerShell myself, but am certainly interested in the scripting, maybe we can rewrite it at some point to python to fit it in our build tools.

Shall I publish what we have now? Maybe that inspires others to write some additional documentation around the described endpoints. (If I understand your work sofar, your scripting replaces the table sections and leaves the rest of the content intact).

[fvanroie]

I pulled the changes down and rendered it in Sphinx and that looks nice :-) I think you can publish it like it is and we can go from there.

For the moment I only have a generator script. Now that a final file structure is in place, I can rework it as an updater script. It can then check if an API call is in the documentation or not and add it if necessary...

[AdSchellevis]

let's do that, nice progress this. https://docs.opnsense.org/development/api.html

[fichtner]

love it, thanks to you both! :)

[fabianfrz]

@fvanroie If you can look into the function body and find a single call to a method with base in the name, the call is for interaction with the model. You would be able to add the API result as well because you can read the data format from the model.

[fvanroie]

@fabianfrz I'll have a look at that, allthough I already have a function that parses the model.xml files to create classes for CRUD items. So getting the info is not really a problem, but presenting it in a concise manner using RST is more of a challenge for me.

Does RST have a collapsable codeblock option? That way this info could be included whithout causing too much clutter on the page.

[fabianfrz]

@fvanroie sorry, I don't know that, but it is the job of the renderer anyway (maybe the template needs to be adopted to do that).

Collapse does not work with PDF so it probably cannot be in the table anyway because the HTML has limited width too.

[fvanroie]

@AdSchellevis I reworked the API extraction script in Python here fvanroie/opnsense_core@24d8644. Most of it is git, grep and regex to pull the data in.

[AdSchellevis]

@fvanroie cool! Are you planning to update the doc pages with this script at some point too?

[fvanroie]

@AdSchellevis Yes, that's still the plan. I'm just looking for the right tool for the job. My PowerShell is better than my Python skills. In the meantime I have found a nice API reference written in Sphinx that looks great. I'll look into those RST pages for some stlying guidelines.

I've also been looking at the OpenApi documentation tools provided by Swagger. I adjusted the output to produce OAS code instead of RST, including the data models for CRUD items. A preview can be found here.

[AdSchellevis]

@fvanroie I've tried slate some time ago with another project, which was quite practical, but gathering all in Sphinx into the same documentation base certainly has it advantages. If you help on the python part, just let me know. It would be cool if we could generate the basics and annotate the details in these documents.

[fvanroie]

I published the initial opnsense_MakeDocs PowerShell Module on github. It's the build script from source 2 rST and works on Windows and Linux. There is a dependency on some handcrafted json files (included in the module) for now... I'm in favor of removing this dependancy for of an OpenAPI specification file as the single source of thruth. That format seems to be the de-facto standard for APIs and widely supported...

On that note, I also built my first OPNsense plugin called SwaggerUI. It's an interactive API Reference viewer built on top of Swagger-UI-dist. The viewer can be installed in OPNsense, kind of what @juggie suggested in opnsense/core#1646.

The proof-of-concept: