Merge pull request #399 from steve-s/ss/small-docs-improvs

Small documentation improvements & quickstart guide

Makefile

@@ -73,5 +73,6 @@ docs-examples-tests:
 	cd docs/examples/simple-example  && python3 setup.py --hpy-abi=universal install
 	cd docs/examples/mixed-example   && python3 setup.py install
 	cd docs/examples/snippets        && python3 setup.py --hpy-abi=universal install
+	cd docs/examples/quickstart      && python3 setup.py --hpy-abi=universal install
 	cd docs/examples/hpytype-example && python3 setup.py --hpy-abi=universal install
 	python3 -m pytest docs/examples/tests.py ${TEST_ARGS}

docs/api-reference/inline-helpers.rst

@@ -5,5 +5,10 @@ Inline Helpers
 Those functions are usually small convenience functions that everyone could
 write but in order to avoid duplicated effort, they are defined by HPy.
 
+One category of inline helpers are functions that convert the commonly used
+but not fixed width C types, such as ``int``, or ``long long``, to HPy API.
+The HPy API always uses well-defined fixed width types like ``int32`` or
+``unsigned int8``.
+
 .. autocmodule:: hpy/inline_helpers.h
    :members:

docs/api.rst

@@ -157,6 +157,7 @@ Moreover, ``HPyContext`` is used by the :term:`HPy Universal ABI` to contain a
 sort of virtual function table which is used by the C extensions to call back
 into the Python interpreter.
 
+.. _simple example:
 
 A simple example
 -----------------
@@ -242,9 +243,11 @@ are still written using the ``Python.h`` API.
 
 Note that the HPy module does not specify its name. HPy does not support the legacy
 single phase module initialization and the only module initialization approach is
-the multi-phase initialization (PEP 451). With multi-phase module initialization,
-the name of the module is always taken from the ``ModuleSpec``, i.e., most likely
-from the name used in the ``import {{name}}`` statement that imported your module.
+the multi-phase initialization (`PEP 489 <https://peps.python.org/pep-0489/>`_).
+With multi-phase module initialization,
+the name of the module is always taken from the ``ModuleSpec`` (`PEP 451 <https://peps.python.org/pep-0451/>`_)
+, i.e., most likely from the name used in the ``import {{name}}`` statement that
+imported your module.
 
 This is the only difference stemming from multi-phase module initialization in this
 simple example.
@@ -320,30 +323,6 @@ table, which now becomes:
   :start-after: // BEGIN: methodsdef
   :end-before: // END: methodsdef
 
-More Examples
--------------
-
-HPy usually has tests for each API function. This means that there is lots of
-examples available by looking at the tests. However, the test source uses
-many macros and is hard to read. To overcome this we supply a utility to
-export clean C sources for the tests. Since the HPy tests are not shipped by
-default, you need to clone the HPy repository from GitHub:
-
-.. code-block:: console
-
-    > git clone https://github.com/hpyproject/hpy.git
-
-After that, install all test requirements and dump the sources:
-
-.. code-block:: console
-
-    > cd hpy
-    > python3 -m pip install pytest filelock
-    > python3 -m pytest --dump-dir=test_sources test/
-
-This will dump the generated test sources into folder ``test_sources``. Note,
-that the tests won't be executed but skipped with an appropriate message.
-
 Creating types in HPy
 ---------------------
 
@@ -477,7 +456,7 @@ A type with ``.legacy_slots != NULL`` is required to have
 ``HPyType_BuiltinShape_Legacy`` and to include ``PyObject_HEAD`` at the start of
 its struct. It would be easy to relax this requirement on CPython (where the
 ``PyObject_HEAD`` fields are always present) but a large burden on other
-implementations (e.g. PyPy, GraalPython) where a struct starting with
+implementations (e.g. PyPy, GraalPy) where a struct starting with
 ``PyObject_HEAD`` might not exist.
 
 Types created via the old Python C API are automatically legacy types.
@@ -567,3 +546,40 @@ be considered in three places:
 For more information about the built-in shape and for a technical explanation
 for why it is required, see :c:member:`HPyType_Spec.builtin_shape` and
 :c:enum:`HPyType_BuiltinShape`.
+
+More Examples
+-------------
+
+The :doc:`porting-example/index` shows another complete example
+of HPy extension ported from Python/C API.
+
+The `HPy project space <https://github.com/hpyproject/>`_ on GitHub
+contains forks of some popular Python extensions ported to HPy as
+a proof of concept/feasibility studies, such as the
+`Kiwi solver <https://github.com/hpyproject/kiwi-hpy>`_.
+Note that those forks may not be up to date with their upstream projects
+or with the upstream HPy changes.
+
+HPy unit tests
+~~~~~~~~~~~~~~
+
+HPy usually has tests for each API function. This means that there is lots of
+examples available by looking at the tests. However, the test source uses
+many macros and is hard to read. To overcome this we supply a utility to
+export clean C sources for the tests. Since the HPy tests are not shipped by
+default, you need to clone the HPy repository from GitHub:
+
+.. code-block:: console
+
+    > git clone https://github.com/hpyproject/hpy.git
+
+After that, install all test requirements and dump the sources:
+
+.. code-block:: console
+
+    > cd hpy
+    > python3 -m pip install pytest filelock
+    > python3 -m pytest --dump-dir=test_sources test/
+
+This will dump the generated test sources into folder ``test_sources``. Note,
+that the tests won't be executed but skipped with an appropriate message.

docs/debug-mode.rst

@@ -43,7 +43,7 @@ Debug mode works *only* for extensions built with HPy universal ABI.
 To enable debug mode, use environment variable ``HPY``. If ``HPY=debug``, then
 all HPy modules are loaded with the trace context. Alternatively, it is also
 possible to specify the mode per module like this:
-``HPY=modA:debug,modB=debug``.
+``HPY=modA:debug,modB:debug``.
 
 In order to verify that your extension is being loaded in debug mode, use
 environment variable ``HPY_LOG``. If this variable is set, then all HPy

docs/examples/hpytype-example/builtin_type.c

@@ -82,7 +82,7 @@ static void make_Language(HPyContext *ctx, HPy module)
 }
 
 HPyDef_SLOT(simple_exec, HPy_mod_exec)
-int simple_exec_impl(HPyContext *ctx, HPy m) {
+static int simple_exec_impl(HPyContext *ctx, HPy m) {
     make_Dummy(ctx, m);
     if (HPyErr_Occurred(ctx))
         return -1;

docs/examples/hpytype-example/simple_type.c

@@ -80,7 +80,7 @@ static HPyType_Spec Point_spec = {
 
 // BEGIN: add_type
 HPyDef_SLOT(simple_exec, HPy_mod_exec)
-int simple_exec_impl(HPyContext *ctx, HPy m) {
+static int simple_exec_impl(HPyContext *ctx, HPy m) {
     if (!HPyHelpers_AddType(ctx, m, "Point", &Point_spec, NULL)) {
         return -1;
     }

docs/examples/quickstart/quickstart.c

@@ -0,0 +1,39 @@
+// quickstart.c
+
+// This header file is the entrypoint to the HPy API:
+#include "hpy.h"
+
+// HPy method: the HPyDef_METH macro generates some boilerplate code,
+// the same code can be also written manually if desired
+HPyDef_METH(say_hello, "say_hello", HPyFunc_NOARGS)
+static HPy say_hello_impl(HPyContext *ctx, HPy self)
+{
+    // Methods take HPyContext, which must be passed as the first argument to
+    // all HPy API functions. Other than that HPyUnicode_FromString does the
+    // same thing as PyUnicode_FromString.
+    //
+    // HPy type represents a "handle" to a Python object, but may not be
+    // a pointer to the object itself. It should be fully "opaque" to the
+    // users. Try uncommenting the following two lines to see the difference
+    // from PyObject*:
+    //
+    // if (self == self)
+    //      HPyUnicode_FromString(ctx, "Surprise? Try HPy_Is(ctx, self, self)");
+
+    return HPyUnicode_FromString(ctx, "Hello world");
+}
+
+static HPyDef *QuickstartMethods[] = {
+    &say_hello, // 'say_hello' generated for us by the HPyDef_METH macro
+    NULL,
+};
+
+static HPyModuleDef quickstart_def = {
+    .doc = "HPy Quickstart Example",
+    .defines = QuickstartMethods,
+};
+
+// The Python interpreter will create the module for us from the
+// HPyModuleDef specification. Additional initialization can be
+// done in the HPy_mod_execute slot
+HPy_MODINIT(quickstart, quickstart_def)

docs/examples/quickstart/setup.py

@@ -0,0 +1,13 @@
+# setup.py
+
+from setuptools import setup, Extension
+from os import path
+
+DIR = path.dirname(__file__)
+setup(
+    name="hpy-quickstart",
+    hpy_ext_modules=[
+        Extension('quickstart', sources=[path.join(DIR, 'quickstart.c')]),
+    ],
+    setup_requires=['hpy'],
+)

docs/examples/tests.py

@@ -43,6 +43,12 @@ def test_simple_type():
     assert p.z == 2000
 
 
+def test_quickstart():
+    import quickstart
+    assert quickstart.say_hello() == "Hello world"
+# END: test_quickstart
+
+
 def test_builtin_type():
     obj = builtin_type.Dummy("hello")
     assert obj == "hello"

docs/index.rst

@@ -8,36 +8,58 @@ HPy: a better API for Python
 
 HPy provides a new API for extending Python in C.
 
+There are several advantages to writing C extensions in HPy:
+
+  - **Speed**: it runs much faster on PyPy, GraalPy, and at native speed on CPython
+
+  - **Deployment**: it is possible to compile a single binary which runs unmodified on all
+    supported Python implementations and versions -- think "stable ABI" on steroids
+
+  - **Simplicity**: it is simpler and more manageable than the ``Python.h`` API, both for
+    the users and the Pythons implementing it
+
+  - **Debugging**: it provides an improved debugging experience. Debug mode can be turned
+    on at runtime without the need to recompile the extension or the Python running it.
+    HPy design is more suitable for automated checks.
+
 The official `Python/C API <https://docs.python.org/3/c-api/index.html>`_,
 also informally known as ``#include <Python.h>``, is
 specific to the current implementation of CPython: it exposes a lot of
-internal details which makes it hard:
+internal details which makes it hard to:
+
+  - implement it for other Python implementations (e.g. PyPy, GraalPy,
+    Jython, ...)
+
+  - experiment with new approaches inside CPython itself, for example:
 
-  - to implement it for other Python implementations (e.g. PyPy, GraalPython,
-    Jython, IronPython, etc.)
+    - use a tracing garbage collection instead of reference counting
+    - remove the global interpreter lock (GIL) to take full advantage of multicore architectures
+    - use tagged pointers to reduce memory footprint
 
-  - to experiment with new things inside CPython itself: e.g. using a GC
-    instead of refcounting, or to remove the GIL.
+Where to go next:
+-----------------
 
-There are several advantages to write your C extension in HPy:
+  - Show me the code:
 
-  - it runs much faster on PyPy, GraalPython, and at native speed on CPython
+      - :doc:`Quickstart<quickstart>`
+      - :ref:`Simple documented HPy extension example<simple example>`
+      - :doc:`Tutorial: porting Python/C API extension to HPy<porting-example/index>`
 
-  - it is possible to compile a single binary which runs unmodified on all
-    supported Python implementations and versions
+  - Details:
 
-  - it is simpler and more manageable than the ``Python.h`` API
+      - :doc:`HPy overview: motivation, goals, current status<overview>`
+      - :doc:`HPy API concepts introduction<api>`
+      - :doc:`Python/C API to HPy Porting guide<porting-guide>`
+      - :doc:`HPy API reference<api-reference/index>`
 
-  - it provides an improved debugging experience: in "debug mode", HPy
-    actively checks for many common mistakes such as reference leaks and
-    invalid usage of objects after they have been deleted. It is possible to
-    turn the "debug mode" on at startup time, without needing to recompile
-    Python or the extension itself
 
+Full table of contents:
+-----------------------
 
 .. toctree::
    :maxdepth: 2
 
+   quickstart
    overview
    api
    porting-guide

docs/misc/index.rst

@@ -2,8 +2,6 @@ Misc notes
 ==========
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 1
 
-   str-builder-api
    embedding
-   protocols