Doc: add reference doc for submission json schema (Infra) (#872)

* Doc: add sphinx extension for json schema * ignore schema doc from spelling check * update cross-references per comments * add note in PR template for checking schema updates * mark outdated tutorials as orphan pages * Update .github/pull_request_template.md Co-authored-by: Matias Piipari <matias.piipari@gmail.com> --------- Co-authored-by: Matias Piipari <matias.piipari@gmail.com>
canonical · Dec 7, 2023 · 27d1974 · 27d1974
1 parent 601f7ce
commit 27d1974
Show file tree

Hide file tree

Showing 10 changed files with 35 additions and 11 deletions.
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -14,6 +14,8 @@ Pick one of the following:
 
 If your change is to providers it can only be (Infra, BugFix or New).
 
+If your change impacts the submission format in Checkbox test reports, ensure that `submission-schema/schema.json` is updated and relevant fields are documented.
+
 Signed commits are required.
   - See CONTRIBUTING.md (https://github.com/canonical/checkbox/blob/main/CONTRIBUTING.md#signed-commits-required) for further instructions.
   - If you are posting your first pull request from a fork of the repository, a Checkbox maintainer (someone with contributor / maintainer / admin rights) will be required to enable CI checks in the repo to be executed.

diff --git a/.vscode/settings.json b/.vscode/settings.json
diff --git a/docs/.sphinx/requirements.txt b/docs/.sphinx/requirements.txt
@@ -13,3 +13,4 @@ lxd-sphinx-extensions
 sphinx-copybutton
 myst-parser
 setuptools-scm
+sphinx-jsonschema
diff --git a/docs/.sphinx/spellingcheck.yaml b/docs/.sphinx/spellingcheck.yaml
@@ -8,7 +8,7 @@ matrix:
     - .sphinx/wordslist.txt
     output: .sphinx/.wordlist.dic
   sources:
-  - _build/**/*.html
+  - _build/**/*.html|!_build/reference/submission-schema/*
   pipeline:
   - pyspelling.filters.html:
       comments: false

diff --git a/docs/conf.py b/docs/conf.py
@@ -52,7 +52,8 @@
     'terminal-output',
     'sphinx_copybutton',
     'sphinxext.opengraph',
-    'myst_parser'
+    'myst_parser',
+    'sphinx-jsonschema'
     ]
 
 myst_enable_extensions = [

diff --git a/docs/reference/index.rst b/docs/reference/index.rst
@@ -10,3 +10,4 @@ Reference
    launcher
    units/index
    snaps
+   submission-schema
diff --git a/docs/reference/submission-schema.rst b/docs/reference/submission-schema.rst
@@ -0,0 +1,16 @@
+.. _submission_schema:
+
+Submission JSON Schema
+=======================
+
+Checkbox :ref:`submissions <submission-files>` contain reports of the tests and in-depth information that helps analysing the test results. 
+
+This document describes the schema of the ``submission.json`` files as part of the submission. To get the latest JSON schema file, go to the Checkbox `GitHub repository <https://github.com/canonical/checkbox/blob/main/submission-schema/schema.json>`_.
+
+.. important:: 
+
+    The schema described in this document is work-in-progress and being reviewed. If you need assistance in validating the schema, please contact the Checkbox team.
+
+.. jsonschema:: ../../submission-schema/schema.json
+    :lift_definitions:
+    :auto_reference:
diff --git a/docs/tutorial/tutorial.rst b/docs/tutorial/tutorial.rst
@@ -1,3 +1,5 @@
+:orphan: This document is built but not included in the table of content.
+
 .. _tutorials:
 
 Checkbox tutorials

diff --git a/docs/tutorial/using-checkbox/test-output.rst b/docs/tutorial/using-checkbox/test-output.rst
@@ -1,3 +1,5 @@
+:orphan: This document is built but not included in the table of content.
+
 .. _test_output:
 
 ============================

diff --git a/docs/tutorial/using-checkbox/test-report.rst b/docs/tutorial/using-checkbox/test-report.rst
@@ -17,6 +17,10 @@ respective contents.
     exporter to customize the reports. See :doc:`launcher<../../reference/launcher>` 
     and :doc:`exporter<../../reference/units/exporter>` for more details.
 
+.. note:: 
+
+    For a detailed description of the ``submission.json`` file, see the :doc:`../../reference/submission-schema` reference.
+
 .. _text-summary:
 
 Text Summary
@@ -122,8 +126,8 @@ in the search bar, you can view all test results related to audio testing.
     An example of beginning of a HTML report
 
 ``json``
-    JSON files store session exports that are compatible for submission to 
-    Certification Website.
+    JSON files store session exports that are generated using the pre-defined
+    :doc:`../../reference/submission-schema`.
 
 ``junit``
     `JUnit XML <https://windyroad.com.au/dl/Open%20Source/JUnit.xsd>`_ files 
@@ -133,8 +137,8 @@ in the search bar, you can view all test results related to audio testing.
     The xz compressed tarball is a comprehensive archive that includes the 
     aforementioned reports and all associated attachments, such as I/O logs 
     and binary files. You can extract the tarball with ``tar -xf 
-    sumbission.tar.xz -C /path/to/destination``.
+    submission.tar.xz -C /path/to/destination``.
 
     Certification Website only accepts submissions tarballs, from which it 
     extracts the ``submission.json`` file to create a new test report in the 
-    database. 
+    database.