Documentation Refactor (apache#9203)

* Documentation Refactor - Stage 1 RFC: https://github.com/apache/tvm-rfcs/blob/main/rfcs/0027-formalize-documentation-organization.md Tracking Issue: apache#8987 Stage 1 of the documentation refactor reorganizes the docs structure, moving files (without content changes) and adding new scaffolding to generate the proper document tree. It does not address naming, style, content, links, or other existing content in documents that were moved. State 2 will address fixing these issues with existing content. Major changes include but are not limited to: * Dividing the existing tutorials into two sections: * Tutorials * How Tos * Moving all of the existing tutorials out of the `/tutorial` directory and into the more general `/gallery` directory. * Breaking up how-tos into individual sections for more flexibility and more consistent rendering. * Moving content into new classifications: * `/docs/arch` for architecture guides * `/docs/reference` for API guides and other reference material * `/docs/topic` for topic specific guides such as microTVM and VTA * Restructuring `/docs/dev` * Adding a table of contents to the doc index * Adding instructions on how to install using third-party tlcpack * Documentation Refactor - Stage 2 RFC: https://github.com/apache/tvm-rfcs/blob/main/rfcs/0027-formalize-documentation-organization.md Tracking Issue: apache#8987 Stage 2 of the documentation refactor fixes naming and links in the documentation to be consistent with the overall structure. Major changes include: * an update to how to contribute to docs. * several updated index pages with title changes to match the organization style and bring consistency to the sections * expanded descriptions of some page collections * fixed links * Documentation Refactor - Stage 3 RFC: https://github.com/apache/tvm-rfcs/blob/main/rfcs/0027-formalize-documentation-organization.md Tracking Issue: apache#8987 Stage 3 of the documentation refactor adjusts CI for the new structure. The CI build script takes into account the new gallery format. It also prevents deleting existing documents, and takes advantage of the `_staging` and `_build` directories to clean out previous builds.
ylc · Jan 13, 2022 · 487c1cc · 487c1cc
1 parent e847ee9
commit 487c1cc
Show file tree

Hide file tree

Showing 173 changed files with 464 additions and 163 deletions.
diff --git a/docs/dev/benchmark.rst → docs/arch/benchmark.rst b/docs/dev/benchmark.rst → docs/arch/benchmark.rst
diff --git a/docs/dev/convert_layout.rst → docs/arch/convert_layout.rst b/docs/dev/convert_layout.rst → docs/arch/convert_layout.rst
diff --git a/docs/dev/debugger.rst → docs/arch/debugger.rst b/docs/dev/debugger.rst → docs/arch/debugger.rst
diff --git a/docs/dev/device_target_interactions.rst → docs/arch/device_target_interactions.rst b/docs/dev/device_target_interactions.rst → docs/arch/device_target_interactions.rst
diff --git a/docs/dev/frontend/tensorflow.rst → docs/arch/frontend/tensorflow.rst b/docs/dev/frontend/tensorflow.rst → docs/arch/frontend/tensorflow.rst
diff --git a/docs/dev/hybrid_script.rst → docs/arch/hybrid_script.rst b/docs/dev/hybrid_script.rst → docs/arch/hybrid_script.rst
diff --git a/docs/dev/index.rst → docs/arch/index.rst b/docs/dev/index.rst → docs/arch/index.rst
diff --git a/docs/dev/inferbound.rst → docs/arch/inferbound.rst b/docs/dev/inferbound.rst → docs/arch/inferbound.rst
diff --git a/.../introduction_to_module_serialization.rst → .../introduction_to_module_serialization.rst b/.../introduction_to_module_serialization.rst → .../introduction_to_module_serialization.rst
diff --git a/docs/dev/microtvm_design.rst → docs/arch/microtvm_design.rst b/docs/dev/microtvm_design.rst → docs/arch/microtvm_design.rst
@@ -13,6 +13,8 @@
     specific language governing permissions and limitations
     under the License.
 
+.. _microtvm-design:
+
 **************************
 microTVM Design Document
 **************************

diff --git a/docs/dev/model_library_format.rst → docs/arch/model_library_format.rst b/docs/dev/model_library_format.rst → docs/arch/model_library_format.rst
diff --git a/docs/dev/pass_infra.rst → docs/arch/pass_infra.rst b/docs/dev/pass_infra.rst → docs/arch/pass_infra.rst
diff --git a/docs/dev/relay_intro.rst → docs/arch/relay_intro.rst b/docs/dev/relay_intro.rst → docs/arch/relay_intro.rst
diff --git a/docs/dev/relay_op_strategy.rst → docs/arch/relay_op_strategy.rst b/docs/dev/relay_op_strategy.rst → docs/arch/relay_op_strategy.rst
diff --git a/docs/dev/runtime.rst → docs/arch/runtime.rst b/docs/dev/runtime.rst → docs/arch/runtime.rst
diff --git a/docs/dev/runtimes/vulkan.rst → docs/arch/runtimes/vulkan.rst b/docs/dev/runtimes/vulkan.rst → docs/arch/runtimes/vulkan.rst
diff --git a/docs/dev/security.rst → docs/arch/security.rst b/docs/dev/security.rst → docs/arch/security.rst
diff --git a/docs/dev/virtual_machine.rst → docs/arch/virtual_machine.rst b/docs/dev/virtual_machine.rst → docs/arch/virtual_machine.rst
diff --git a/docs/conf.py b/docs/conf.py
@@ -51,9 +51,8 @@
 else:
     tvm_path = Path(os.pardir)
 
-
-sys.path.insert(0, str(tvm_path / "python"))
-sys.path.insert(0, str(tvm_path / "vta" / "python"))
+sys.path.insert(0, str(tvm_path.resolve() / "python"))
+sys.path.insert(0, str(tvm_path.resolve() / "vta" / "python"))
 
 # -- General configuration ------------------------------------------------
 
@@ -209,22 +208,37 @@ def git_describe_version(original_version):
 
 from sphinx_gallery.sorting import ExplicitOrder
 
-examples_dirs = [tvm_path.joinpath("tutorials"), tvm_path.joinpath("vta", "tutorials")]
-gallery_dirs = ["tutorials", "vta/tutorials"]
+examples_dirs = [
+    tvm_path.joinpath("gallery", "tutorial"),
+    tvm_path.joinpath("gallery", "how_to", "compile_models"),
+    tvm_path.joinpath("gallery", "how_to", "deploy_models"),
+    tvm_path.joinpath("gallery", "how_to", "work_with_relay"),
+    tvm_path.joinpath("gallery", "how_to", "work_with_schedules"),
+    tvm_path.joinpath("gallery", "how_to", "optimize_operators"),
+    tvm_path.joinpath("gallery", "how_to", "tune_with_autotvm"),
+    tvm_path.joinpath("gallery", "how_to", "tune_with_autoscheduler"),
+    tvm_path.joinpath("gallery", "how_to", "work_with_microtvm"),
+    tvm_path.joinpath("gallery", "how_to", "extend_tvm"),
+    tvm_path.joinpath("vta", "tutorials"),
+]
+
+gallery_dirs = [
+    "tutorial",
+    "how_to/compile_models",
+    "how_to/deploy_models",
+    "how_to/work_with_relay",
+    "how_to/work_with_schedules",
+    "how_to/optimize_operators",
+    "how_to/tune_with_autotvm",
+    "how_to/tune_with_autoscheduler",
+    "how_to/work_with_microtvm",
+    "how_to/extend_tvm",
+    "topic/vta/tutorials",
+]
 
 subsection_order = ExplicitOrder(
     str(p)
     for p in [
-        tvm_path / "tutorials" / "get_started",
-        tvm_path / "tutorials" / "frontend",
-        tvm_path / "tutorials" / "language",
-        tvm_path / "tutorials" / "optimize",
-        tvm_path / "tutorials" / "autotvm",
-        tvm_path / "tutorials" / "auto_scheduler",
-        tvm_path / "tutorials" / "dev",
-        tvm_path / "tutorials" / "topi",
-        tvm_path / "tutorials" / "deployment",
-        tvm_path / "tutorials" / "micro",
         tvm_path / "vta" / "tutorials" / "frontend",
         tvm_path / "vta" / "tutorials" / "optimize",
         tvm_path / "vta" / "tutorials" / "autotvm",
@@ -236,18 +250,19 @@ def git_describe_version(original_version):
 # The unlisted files are sorted by filenames.
 # The unlisted files always appear after listed files.
 within_subsection_order = {
-    "get_started": [
+    "tutorial": [
         "introduction.py",
         "install.py",
         "tvmc_command_line_driver.py",
         "autotvm_relay_x86.py",
         "tensor_expr_get_started.py",
         "autotvm_matmul_x86.py",
         "auto_scheduler_matmul_x86.py",
+        "topi.pi",
         "cross_compilation_and_rpc.py",
         "relay_quick_start.py",
     ],
-    "frontend": [
+    "compile_models": [
         "from_pytorch.py",
         "from_tensorflow.py",
         "from_mxnet.py",
@@ -259,7 +274,7 @@ def git_describe_version(original_version):
         "from_caffe2.py",
         "from_paddle.py",
     ],
-    "language": [
+    "work_with_schedules": [
         "schedule_primitives.py",
         "reduction.py",
         "intrin_math.py",
@@ -269,26 +284,25 @@ def git_describe_version(original_version):
         "tuple_inputs.py",
         "tedd.py",
     ],
-    "optimize": [
+    "optimize_operators": [
         "opt_gemm.py",
         "opt_conv_cuda.py",
         "opt_conv_tensorcore.py",
     ],
-    "autotvm": [
-        "tune_simple_template.py",
+    "tune_with_autotvm": [
         "tune_conv2d_cuda.py",
         "tune_relay_cuda.py",
         "tune_relay_x86.py",
         "tune_relay_arm.py",
         "tune_relay_mobile_gpu.py",
     ],
-    "auto_scheduler": [
+    "tune_with_autoscheduler": [
         "tune_matmul_x86.py",
         "tune_conv2d_layer_cuda.py",
         "tune_network_x86.py",
         "tune_network_cuda.py",
     ],
-    "dev": [
+    "extend_tvm": [
         "low_level_custom_pass.py",
         "use_pass_infra.py",
         "use_pass_instrument.py",

diff --git a/docs/contribute/document.rst b/docs/contribute/document.rst
@@ -17,22 +17,125 @@
 
 .. _doc_guide:
 
-Write Document and Tutorials
-============================
+Write Documentation for TVM
+===========================
+
+TVM documentation loosely follows the `formal documentation style described by
+Divio <https://documentation.divio.com>`_. This system has been chosen because
+it is a "simple, comprehensive and nearly universally-applicable scheme. It is
+proven in practice across a wide variety of fields and applications."
+
+This document describes the organization of TVM documentation, and how to write
+new documentation.
+
+The Four Document Types
+***********************
+
+Introductory Tutorials
+----------------------
+
+These are step by step guides to introduce new users to a project. An
+introductory tutorial is designed to get a user engaged with the software
+without necessarily explaining why the software works the way it does. Those
+explanations can be saved for other document types. An introductory tutorial
+focuses on a successful first experience. These are the most important docs to
+turning newcomers into new users and developers. A fully end-to-end
+tutorial&mdash; from installing TVM and supporting ML software, to creating and
+training a model, to compiling to different architectures&mdash;will give a new
+user the opportunity to use TVM in the most efficient way possible. A tutorial
+teaches a beginner something they need to know. This is in contrast with a
+how-to, which is meant to be an answer to a question that a user with some
+experience would ask.
+
+Tutorials need to be repeatable and reliable, because the lack of success means
+a user will look for other solutions.
+
+How-to Guides
+-------------
+
+These are step by step guides on how to solve particular problems. The user can
+ask meaningful questions, and the documents provide answers. An examples of
+this type of document might be, "how do I compile an optimized model for ARM
+architecture?" or "how do I compile and optimize a TensorFlow model?" These
+documents should be open enough that a user could see how to apply it to a new
+use case. Practical usability is more important than completeness. The title
+should tell the user what problem the how-to is solving.
+
+How are tutorials different from how-tos? A tutorial is oriented towards the
+new developer, and focuses on successfully introducing them to the software and
+community. A how-to, in contrast, focuses on accomplishing a specific task
+within the context of basic understanding. A tutorial helps to on-board and
+assumes no prior knowledge. A how-to assumes minimum knowledge, and is meant to
+guide someone to accomplish a specific task.
+
+Reference
+---------
+
+Reference documentation describes how the software is configured and operated.
+APIs, key functions, commands, and interfaces are all candidates for reference
+documentation. These are the technical manuals that let users build their own
+interfaces and programs. They are information oriented, focused on lists and
+descriptions. You can assume that the audience has a grasp on how the software
+works and is looking for specific answers to specific questions. Ideally, the
+reference documentation should have the same structure as the code base and be
+generated automatically as much as possible.
+
+Architecture Guides
+-------------------
+
+Architecture Guides are explanations are background material on a topic. These
+documents help to illuminate and understand the application environment. Why
+are things the way they are? What were the design decisions, what alternatives
+were considered, what are the RFCs describing the existing system? This
+includes academic papers and links to publications relevant to the software.
+Within these documents you can explore contradictory and conflicting position,
+and help the reader make sense of how and why the software was built the way it
+is. It's not the place for how-tos and descriptions on how to accomplish tasks.
+They instead focus on higher level concepts that help with the understanding of
+the project.  Generally these are written by the architects and developers of
+the project, but can useful to help both users and developers to have a deeper
+understanding of why the software works the way it does, and how to contribute
+to it in ways that are consistent with the underlying design principles.
+
+Special considerations for TVM
+------------------------------
+
+The TVM community has some special considerations that require deviation from
+the simple docs style outlined by Divio. The first consideration is that there
+is frequently overlap between the user and developer communities. Many projects
+document the developer and user experience with separate systems, but it is
+appropriate to consider both in this system, with differentiations where
+appropriate. As a result the tutorials and how-tos will be divided between
+"User Guides" that focus on the user experience, and "Developer Guides" that
+focus on the developer experience.
+
+The next consideration is that there are special topics within the TVM
+community that benefit from additional attention. These topics include, but are
+not limited to, microTVM and VTA. Special "Topic Guides" can be created to
+index existing material, and provide context on how to navigate that material
+most effectively.
+
+To facilitate newcomers, a special "Getting Started" section with installation
+instructions, a overview of why to use TVM, and other first-experience
+documents will be produced.
+
+
+Technical Details
+*****************
 
 We use the `Sphinx <http://sphinx-doc.org>`_ for the main documentation.
-Sphinx support both the reStructuredText and markdown.
-When possible, we encourage to use reStructuredText as it has richer features.
-Note that the python doc-string and tutorials allow you to embed reStructuredText syntax.
+Sphinx support both the reStructuredText and markdown.  When possible, we
+encourage to use reStructuredText as it has richer features.  Note that the
+python doc-string and tutorials allow you to embed reStructuredText syntax.
 
 
-Document Python
----------------
-We use `numpydoc <https://numpydoc.readthedocs.io/en/latest/>`_
-format to document the function and classes.
-The following snippet gives an example docstring.
-We always document all the public functions,
-when necessary, provide an usage example of the features we support(as shown below).
+Python Reference Documentation
+------------------------------
+
+We use `numpydoc <https://numpydoc.readthedocs.io/en/latest/>`_ format to
+document the function and classes.  The following snippet gives an example
+docstring.  We always document all the public functions, when necessary,
+provide an usage example of the features we support(as shown below).
 
 .. code:: python
 
@@ -64,18 +167,20 @@ when necessary, provide an usage example of the features we support(as shown bel
         """
         return rv1
 
-Be careful to leave blank lines between sections of your documents.
-In the above case, there has to be a blank line before `Parameters`, `Returns` and `Examples`
-in order for the doc to be built correctly. To add a new function to the doc,
-we need to add the `sphinx.autodoc <http://www.sphinx-doc.org/en/master/ext/autodoc.html>`_
-rules to the `docs/api/python <https://github.com/apache/tvm/tree/main/docs/api/python>`_).
-You can refer to the existing files under this folder on how to add the functions.
+Be careful to leave blank lines between sections of your documents.  In the
+above case, there has to be a blank line before `Parameters`, `Returns` and
+`Examples` in order for the doc to be built correctly. To add a new function to
+the doc, we need to add the `sphinx.autodoc
+<http://www.sphinx-doc.org/en/master/ext/autodoc.html>`_ rules to the
+`docs/api/python <https://github.com/apache/tvm/tree/main/docs/api/python>`_).
+You can refer to the existing files under this folder on how to add the
+functions.
 
+C++ Reference Documentation
+---------------------------
 
-Document C++
-------------
-We use the doxgen format to document c++ functions.
-The following snippet shows an example of c++ docstring.
+We use the doxgen format to document c++ functions.  The following snippet
+shows an example of c++ docstring.
 
 .. code:: c++
 
@@ -89,21 +194,27 @@ The following snippet shows an example of c++ docstring.
       // When necessary, also add comment to clarify internal logics
     }
 
-Besides documenting function usages, we also highly recommend contributors
-to add comments about code logics to improve readability.
+Besides documenting function usages, we also highly recommend contributors to
+add comments about code logics to improve readability.
 
+Sphinx Gallery How-Tos
+----------------------
 
-Write Tutorials
----------------
-We use the `sphinx-gallery <https://sphinx-gallery.github.io/>`_ to build python tutorials.
-You can find the source code under `tutorials <https://github.com/apache/tvm/tree/main/tutorials>`_ quite self explanatory.
-One thing that worth noting is that the comment blocks are written in reStructuredText instead of markdown so be aware of the syntax.
+We use the `sphinx-gallery <https://sphinx-gallery.github.io/>`_ to build many
+python how-tos.  You can find the source code under `gallery
+<https://github.com/apache/tvm/tree/main/gallery>`_ quite self explanatory.
+One thing that worth noting is that the comment blocks are written in
+reStructuredText instead of markdown so be aware of the syntax.
 
-The tutorial code will run on our build server to generate the document page.
-So we may have a restriction like not being able to access a remote Raspberry Pi,
-in such case add a flag variable to the tutorial (e.g. `use_rasp`) and allow users to easily switch to the real device by changing one flag.
-Then use the existing environment to demonstrate the usage.
+The how-to code will run on our build server to generate the document page.  So
+we may have a restriction like not being able to access a remote Raspberry Pi,
+in such case add a flag variable to the tutorial (e.g. `use_rasp`) and allow
+users to easily switch to the real device by changing one flag. Then use the
+existing environment to demonstrate the usage.
 
+If you add a new categorization of how-to, you will need to add references to
+`conf.py <https://github.com/apache/tvm/tree/main/docs/conf.py>`_ and the
+`how-to index <https://github.com/apache/tvm/tree/main/docs/how-to/index.rst>`_
 
 Refer to Another Location in the Document
 -----------------------------------------

diff --git a/docs/contribute/index.rst b/docs/contribute/index.rst
@@ -15,7 +15,7 @@
     specific language governing permissions and limitations
     under the License.
 
-Contribute to TVM
+Contributor Guide
 =================
 
 TVM has been developed by community members.

diff --git a/docs/dev/how_to.rst → docs/dev/how_to/how_to.rst b/docs/dev/how_to.rst → docs/dev/how_to/how_to.rst
@@ -28,5 +28,4 @@ various areas of the TVM stack.
    relay_add_op
    relay_add_pass
    relay_bring_your_own_codegen
-   codebase_walkthrough
    pytest_target_parametrization
diff --git a/docs/dev/pytest_target_parametrization.rst → .../how_to/pytest_target_parametrization.rst b/docs/dev/pytest_target_parametrization.rst → .../how_to/pytest_target_parametrization.rst
diff --git a/docs/dev/relay_add_op.rst → docs/dev/how_to/relay_add_op.rst b/docs/dev/relay_add_op.rst → docs/dev/how_to/relay_add_op.rst
diff --git a/docs/dev/relay_add_pass.rst → docs/dev/how_to/relay_add_pass.rst b/docs/dev/relay_add_pass.rst → docs/dev/how_to/relay_add_pass.rst
diff --git a/docs/dev/relay_bring_your_own_codegen.rst → ...v/how_to/relay_bring_your_own_codegen.rst b/docs/dev/relay_bring_your_own_codegen.rst → ...v/how_to/relay_bring_your_own_codegen.rst
diff --git a/docs/dev/codebase_walkthrough.rst → docs/dev/tutorial/codebase_walkthrough.rst b/docs/dev/codebase_walkthrough.rst → docs/dev/tutorial/codebase_walkthrough.rst
diff --git a/docs/dev/tutorial/index.rst b/docs/dev/tutorial/index.rst
@@ -0,0 +1,29 @@
+..  Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+..    http://www.apache.org/licenses/LICENSE-2.0
+
+..  Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+.. _dev-tutorial:
+
+Developer Tutorial
+==================
+
+This section is a guide to the TVM codebase, and an introduction on how to
+contribute to different parts of the platform.
+
+.. toctree::
+   :maxdepth: 1
+
+   codebase_walkthrough
diff --git a/docs/errors.rst b/docs/errors.rst
@@ -16,8 +16,8 @@
     under the License.
 
 
-What do to when encountering TVM Errors
-=======================================
+Handle TVM Errors
+=================
 
 When running TVM, you may encounter an error message like:
 

diff --git a/docs/deploy/android.rst → docs/how_to/deploy/android.rst b/docs/deploy/android.rst → docs/how_to/deploy/android.rst
diff --git a/docs/deploy/arm_compute_lib.rst → docs/how_to/deploy/arm_compute_lib.rst b/docs/deploy/arm_compute_lib.rst → docs/how_to/deploy/arm_compute_lib.rst
diff --git a/docs/deploy/bnns.rst → docs/how_to/deploy/bnns.rst b/docs/deploy/bnns.rst → docs/how_to/deploy/bnns.rst
diff --git a/docs/deploy/cpp_deploy.rst → docs/how_to/deploy/cpp_deploy.rst b/docs/deploy/cpp_deploy.rst → docs/how_to/deploy/cpp_deploy.rst
diff --git a/docs/deploy/hls.rst → docs/how_to/deploy/hls.rst b/docs/deploy/hls.rst → docs/how_to/deploy/hls.rst