Users learn from example. So, whether you're writing a docstring or a user guide, include examples that illustrate the relevant APIs. Your examples should run out-of-the-box so that users can copy them and adapt them to their own needs.
This page describes how to write code snippets so that they're tested in CI.
Note
The examples in this guide use reStructuredText. If you're writing Markdown, use MyST syntax. To learn more, read the MyST documentation.
There are three types of examples: doctest-style, code-output-style, and literalinclude.
doctest-style examples mimic interactive Python sessions. :
.. doctest::
>>> def is_even(x):
... return (x % 2) == 0
>>> is_even(0)
True
>>> is_even(1)
FalseThey're rendered like this:
>>> def is_even(x): ... return (x % 2) == 0 >>> is_even(0) True >>> is_even(1) False
Tip
If you're writing docstrings, exclude .. doctest:: to simplify your code. :
Example:
>>> def is_even(x):
... return (x % 2) == 0
>>> is_even(0)
True
>>> is_even(1)
Falsecode-output-style examples contain ordinary Python code. :
.. testcode::
def is_even(x):
return (x % 2) == 0
print(is_even(0))
print(is_even(1))
.. testoutput::
True
FalseThey're rendered like this:
- def is_even(x):
return (x % 2) == 0
print(is_even(0)) print(is_even(1))
True False
literalinclude examples display Python modules. :
.. literalinclude:: ./doc_code/example_module.py
:language: python
:start-after: __is_even_begin__
:end-before: __is_even_end__./doc_code/example_module.py
They're rendered like this:
./doc_code/example_module.py
There's no hard rule about which style you should use. Choose the style that best illustrates your API.
Tip
If you're not sure which style to use, use code-block-style.
If you're writing a small example that emphasizes object representations, or if you want to print intermediate objects, use doctest-style. :
.. doctest::
>>> import ray
>>> ds = ray.data.range(100)
>>> ds.schema()
Column Type
------ ----
id int64
>>> ds.take(5)
[{'id': 0}, {'id': 1}, {'id': 2}, {'id': 3}, {'id': 4}]If you're writing a longer example, or if object representations aren't relevant to your example, use code-block-style. :
.. testcode::
from typing import Dict
import numpy as np
import ray
ds = ray.data.read_csv("s3://anonymous@air-example-data/iris.csv")
# Compute a "petal area" attribute.
def transform_batch(batch: Dict[str, np.ndarray]) -> Dict[str, np.ndarray]:
vec_a = batch["petal length (cm)"]
vec_b = batch["petal width (cm)"]
batch["petal area (cm^2)"] = vec_a * vec_b
return batch
transformed_ds = ds.map_batches(transform_batch)
print(transformed_ds.materialize())
.. testoutput::
MaterializedDataset(
num_blocks=...,
num_rows=150,
schema={
sepal length (cm): double,
sepal width (cm): double,
petal length (cm): double,
petal width (cm): double,
target: int64,
petal area (cm^2): double
}
)If you're writing an end-to-end examples and your examples doesn't contain outputs, use literalinclude.
You don't need to test examples that depend on external systems like Weights and Biases.
To skip a doctest-style example, append # doctest: +SKIP to your Python code. :
.. doctest::
>>> import ray
>>> ray.data.read_images("s3://private-bucket") # doctest: +SKIPTo skip a code-block-style example, add :skipif: True to the testoutput block. :
.. testcode::
:skipif: True
from ray.air.integrations.wandb import WandbLoggerCallback
callback = WandbLoggerCallback(
project="Optimization_Project",
api_key_file=...,
log_config=True
)If your Python code is non-deterministic, or if your output is excessively long, you may want to skip all or part of an output.
To ignore parts of a doctest-style output, replace problematic sections with ellipses. :
>>> import ray
>>> ray.data.read_images("s3://anonymous@ray-example-data/image-datasets/simple")
Dataset(
num_blocks=...,
num_rows=...,
schema={image: numpy.ndarray(shape=(32, 32, 3), dtype=uint8)}
)To ignore an output altogether, write a code-block-style snippet. Don't use # doctest: +SKIP.
If parts of your output are long or non-deterministic, replace problematic sections with ellipses. :
.. testcode::
import ray
ds = ray.data.read_images("s3://anonymous@ray-example-data/image-datasets/simple")
print(ds)
.. testoutput::
Dataset(
num_blocks=...,
num_rows=...,
schema={image: numpy.ndarray(shape=(32, 32, 3), dtype=uint8)}
)If your output is nondeterministic and you want to display a sample output, add :options: +MOCK. :
.. testcode::
import random
print(random.random())
.. testoutput::
:options: +MOCK
0.969461416250246If your output is hard to test and you don't want to display a sample output, exclude the testoutput. :
.. testcode::
print("This output is hidden and untested")To configure Bazel to run an example with GPUs, complete the following steps:
- Open the corresponding
BUILDfile. If your example is in thedoc/folder, opendoc/BUILD. If your example is in thepython/folder, open a file likepython/ray/train/BUILD. Locate the
doctestrule. It looks like this: :doctest( files = glob( include=["source/**/*.rst"], ), size = "large", tags = ["team:none"] )Add the file that contains your example to the list of excluded files. :
doctest( files = glob( include=["source/**/*.rst"], exclude=["source/data/requires-gpus.rst"] ), tags = ["team:none"] )If it doesn't already exist, create a
doctestrule withgpuset toTrue. :doctest( files = [], tags = ["team:none"], gpu = True )Add the file that contains your example to the GPU rule. :
doctest( files = ["source/data/requires-gpus.rst"] size = "large", tags = ["team:none"], gpu = True )
For a practical example, see doc/BUILD or python/ray/train/BUILD.
To locally test examples, install the Ray fork of pytest-sphinx.
pip install git+https://github.com/ray-project/pytest-sphinxThen, run pytest on a module, docstring, or user guide.
pytest --doctest-modules python/ray/data/read_api.py
pytest --doctest-modules python/ray/data/read_api.py::ray.data.read_api.range
pytest --doctest-modules doc/source/data/getting-started.rst