Skip to content

Commit

Permalink
Merge pull request google#358 from google/docs-contributing-section
Browse files Browse the repository at this point in the history
add contributing section to docs
  • Loading branch information
ianspektor committed Jan 30, 2024
2 parents 1005a5e + 6ef914d commit b585065
Show file tree
Hide file tree
Showing 9 changed files with 109 additions and 72 deletions.
5 changes: 4 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -118,5 +118,8 @@
"python.analysis.typeCheckingMode": "basic",
"jupyter.notebookFileRoot": "${fileDirname}",
"C_Cpp.clang_format_fallbackStyle": "Google",
"editor.defaultFormatter": "ms-python.black-formatter"
"editor.defaultFormatter": "ms-python.black-formatter",
"[markdown]": {
"editor.defaultFormatter": "esbenp.prettier-vscode"
}
}
84 changes: 47 additions & 37 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,19 @@
# How to Contribute
# Contributing

This guide describes how to contribute to Temporian, and will help you set up your environment and create your first submission.
Temporian is an open source project - we warmly welcome and appreciate any kind of contribution! 🤝🏼

## Contributor License Agreement
This guide describes how to contribute, and will help you set up your environment and create your first submission.

Contributions to this project must be accompanied by a Contributor License Agreement.
Check out the open [GitHub issues](https://github.com/google/temporian/issues) to see what we're working on, and what we need help with.
Look especially for the `good first issue` label, which indicates issues that are suitable for new contributors.

You (or your employer) retain the copyright to your contribution, this simply gives us permission to use and redistribute your contributions as part of the project. Head over to <https://cla.developers.google.com/> to see your current agreements on file or sign a new one.

You generally only need to submit a CLA once, so if you've already submitted one (even if it was for a different project), you probably won't need to do it again.
If you'd like help or additional guidance to contribute, please join our [Discord](https://discord.gg/x8rm9YYy).

## Code reviews

All submissions, including submissions by project members, require review. We use GitHub Pull Requests for this purpose. Consult [GitHub Help](https://help.github.com/articles/about-pull-requests/) for more information on using pull requests.

All new contributions must pass all the tests and checks performed by GitHub actions, and any changes to docstrings must respect the [docstring guidelines](docs/docstring_guidelines.md).
All new contributions must pass all the tests and checks performed by GitHub actions, and any changes to docstrings must respect the [docstring guidelines](tools/docstring_guidelines.md).

## Development

Expand Down Expand Up @@ -50,7 +49,7 @@ Before installing the package you need to install [bazel](https://bazel.build/in
brew install bazelisk
```

Navigate to the project's root and run:
Navigate to the project's root and create a Poetry environment with the correct PyEnv version and all needed dependencies with:

```shell
pyenv which python | xargs poetry env use
Expand All @@ -65,6 +64,8 @@ poetry shell

### Testing

All tests must pass for your contribution to be accepted.

Run all tests with bazel:

```shell
Expand All @@ -78,6 +79,14 @@ If developing and testing C++ code, the `--compilation_mode=dbg` flag enables ad
Note that these tests also include docstring examples, using the builtin `doctest` module.
See the [Adding code examples](#adding-code-examples) section for more information.

### Running the documentation server

Live preview your local changes to the documentation with

```shell
mkdocs serve -f docs/mkdocs.yml
```

### Benchmarking and profiling

Benchmarking and profiling of pre-configured scripts is available as follow:
Expand All @@ -101,14 +110,36 @@ consumption.
bazel run -c opt --config=linux //benchmark:benchmark_time
```

### Running docs server
### Developing a new operator

Live preview your local changes to the documentation with
We provide a utility script that generates placeholder files, modifies existing ones, and prints needed modifications to develop and make available a new operator. From the project's root, run:

```shell
mkdocs serve -f docs/mkdocs.yml
tools/create_operator.py --operator <name>
```

so for example, to create the `EventSet.map()` operator, you'd run `tools/create_operator.py --operator map`.

Here are some key files you'll need to modify (and write the operator's logic in):

- [temporian/core/event_set_ops.py](temporian/core/event_set_ops.py) or [temporian/**init**.py](temporian/__init__.py), depending on if the operator is available in the `EventSet` class (like `EventSet.since_last()`) or in the global `tp` module (like `tp.glue()`).
- Write the operator's core logic in `temporian/core/operators/<name>.py`.
- The core logic is that related to the operator's definition in the graph, checks, and normalization during initialization. It doesn't interact with the actual data contained within the `EventSet`.
- Example: [temporian/core/operators/since_last.py](temporian/core/operators/since_last.py).
- Write the operator's implementation in `temporian/implementation/numpy/operators/<name>.py`.
- The implementation is what actually executes the operator's logic on an `EventSet`'s data.
- Example of a Python-only operator: [temporian/implementation/numpy/operators/since_last.py](temporian/implementation/numpy/operators/since_last.py).
- Example of a C++ operator: [temporian/implementation/numpy/operators/resample.py](temporian/implementation/numpy/operators/resample.py) and [temporian/implementation/numpy_cc/operators/resample.cc](temporian/implementation/numpy_cc/operators/resample.cc).
- Write unit tests for the operator in `temporian/core/operators/test/test_<name>.py`.
- Example: [temporian/core/operators/test/test_since_last.py](temporian/core/operators/test/test_since_last.py).
- Add the operator to the docs in `docs/src/reference/temporian/operators/<name>.md`.
- The docs are generated automatically by [mkdocstrings](https://mkdocstrings.github.io/python/) from the operator's docstring.
- Example: [docs/src/reference/temporian/operators/since_last.md](docs/src/reference/temporian/operators/since_last.md).

Groups of operator with a similar implementation as grouped together. For instance, `temporian/core/operators/window` contains moving window operators (e.g., `EventSet.simple_moving_average()`) and `temporian/core/operators/binary` contains operators taking two features as input (e.g. `EventSet.subtract()`).

Read the script's output to see in detail all other files that need to be modified to finish setting up the operator!

### Adding code examples

Any doctest code examples in `temporian/*.py` or `docs/*.md`, will be executed and tested using the python's built-in [doctest](https://docs.python.org/3/library/doctest.html) module.
Expand Down Expand Up @@ -166,31 +197,10 @@ bazel test --config=linux //temporian/test:doc_test --test_output=streamed

In case of unexpected outputs, the result is printed and compared to the expected values, so that they can be fixed.

### Developing a new operator

We provide a utility script that generates placeholder files, modifies existing ones, and prints needed modifications to develop and make available a new operator. From the project's root, run:

```shell
tools/create_operator.py --operator <name>
```

so for example, to create the `EventSet.map()` operator, you'd run `tools/create_operator.py --operator map`.
## Contributor License Agreement

Here are some key files you'll need to modify (and write the operator's logic in):
- [temporian/core/event_set_ops.py](temporian/core/event_set_ops.py) or [temporian/__init__.py](temporian/__init__.py), depending on if the operator is available in the `EventSet` class (like `EventSet.since_last()`) or in the global `tp` module (like `tp.glue()`).
- Write the operator's core logic in `temporian/core/operators/<name>.py`.
- The core logic is that related to the operator's definition in the graph, checks, and normalization during initialization. It doesn't interact with the actual data contained within the `EventSet`.
- Example: [temporian/core/operators/since_last.py](temporian/core/operators/since_last.py).
- Write the operator's implementation in `temporian/implementation/numpy/operators/<name>.py`.
- The implementation is what actually executes the operator's logic on an `EventSet`'s data.
- Example of a Python-only operator: [temporian/implementation/numpy/operators/since_last.py](temporian/implementation/numpy/operators/since_last.py).
- Example of a C++ operator: [temporian/implementation/numpy/operators/resample.py](temporian/implementation/numpy/operators/resample.py) and [temporian/implementation/numpy_cc/operators/resample.cc](temporian/implementation/numpy_cc/operators/resample.cc).
- Write unit tests for the operator in `temporian/core/operators/test/test_<name>.py`.
- Example: [temporian/core/operators/test/test_since_last.py](temporian/core/operators/test/test_since_last.py).
- Add the operator to the docs in `docs/src/reference/temporian/operators/<name>.md`.
- The docs are generated automatically by [mkdocstrings](https://mkdocstrings.github.io/python/) from the operator's docstring.
- Example: [docs/src/reference/temporian/operators/since_last.md](docs/src/reference/temporian/operators/since_last.md).
Contributions to this project must be accompanied by a Contributor License Agreement.

Groups of operator with a similar implementation as grouped together. For instance, `temporian/core/operators/window` contains moving window operators (e.g., `EventSet.simple_moving_average()`) and `temporian/core/operators/binary` contains operators taking two features as input (e.g. `EventSet.subtract()`).
You (or your employer) retain the copyright to your contribution, this simply gives us permission to use and redistribute your contributions as part of the project. Head over to <https://cla.developers.google.com/> to see your current agreements on file or sign a new one.

Read the script's output to see in detail all other files that need to be modified to finish setting up the operator!
You generally only need to submit a CLA once, so if you've already submitted one (even if it was for a different project), you probably won't need to do it again.
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Temporian is to [temporal data](https://temporian.readthedocs.io/en/stable/user_
Python, on large datasets using [Apache Beam](https://beam.apache.org/).
-->

## QuickStart
## Quickstart

### Installation

Expand Down Expand Up @@ -120,13 +120,15 @@ the major concepts, operators, conventions, and practices of Temporian. For a
hands-on learning experience, work through the [Tutorials](https://temporian.readthedocs.io/en/stable/tutorials/) or refer to the [API
reference](https://temporian.readthedocs.io/en/stable/reference/).

If you need help, have a question, want to contribute, or just want to be a part of the Temporian community, we encourage you to join our [Discord](https://discord.gg/x8rm9YYy) server! 🤝🏼

## Documentation

The documentation 📚 is available at [temporian.readthedocs.io](https://temporian.readthedocs.io/en/stable/). The [Getting Started guide](https://temporian.readthedocs.io/en/stable/getting_started/) is the best way to start.

## Contributing

Contributions to Temporian are welcome! Check out the [contributing guide](CONTRIBUTING.md) to get started.
Contributions to Temporian are welcome! Check out the [Contributing guide](https://temporian.readthedocs.io/en/stable/contributing/) to get started.

## Credits

Expand Down
25 changes: 25 additions & 0 deletions docs/gen_contributing.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
"""
Generates a docs/contributing.md on the fly from the CONTRIBUTING.md and fixes some of the links.
Source: https://github.com/tryolabs/norfair/blob/master/docs/gen_index.py
"""

import re

import mkdocs_gen_files

# read CONTRIBUTING on the root of the repo
with open("CONTRIBUTING.md", "r", encoding="utf-8") as f:
content = f.read()

# fix links to relative paths under temporian/, benchmark/, docs/ and tools/
for dir in ["temporian", "benchmark", "docs", "tools"]:
content = re.sub(
rf"\({dir}\/([\w\/-]+)",
rf"(https://github.com/google/temporian/blob/main/{dir}/\1",
content,
)

# write the file
with mkdocs_gen_files.open("contributing.md", "w") as fd: #
print(content, file=fd)
1 change: 1 addition & 0 deletions docs/gen_index.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
Source: https://github.com/tryolabs/norfair/blob/master/docs/gen_index.py
"""

import re

import mkdocs_gen_files
Expand Down
10 changes: 8 additions & 2 deletions docs/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,10 @@ nav:
- Home: index.md
- Getting Started: getting_started.ipynb
- User Guide: user_guide.ipynb
- Recipes: recipes/
- Tutorials: tutorials/
- Recipes: recipes/
- API Reference: reference/ # generated by gen-files + literate-nav
- Contributing: contributing.md
- Release notes: CHANGELOG.md

extra:
Expand Down Expand Up @@ -89,6 +90,8 @@ plugins:
scripts:
# Generate the index page from the README.md.
- gen_index.py
# Generate the contributing page from the CONTRIBUTING.md.
- gen_contributing.py
# Generate the ref pages and navigation.
- gen_ref_pages.py
- literate-nav:
Expand Down Expand Up @@ -150,9 +153,12 @@ markdown_extensions:
- name: mermaid
class: mermaid
format: !!python/name:pymdownx.superfences.fence_code_format
- mdx_truly_sane_lists # recognize 2 spaces as indentation inside lists

watch:
- ../README.md
- ../temporian
- ./gen_ref_pages.py
- ../README.md
- ./gen_index.py
- ../CONTRIBUTING.md
- ./gen_contributing.py
9 changes: 4 additions & 5 deletions docs/src/css/custom.css
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
[data-md-color-scheme="temporian"] {
--md-primary-fg-color--light: #24201D;
--md-primary-fg-color--dark: #24201D;
background-color: red;
--md-default-bg-color: red;
[data-md-color-scheme="default"] {
}

/* Change background color when dark mode */
[data-md-color-scheme="slate"] {
--md-default-bg-color: #121212;
--md-code-bg-color: #212121;
/* make font a bit brighter than mkdocs-material's default */
--md-typeset-color: #f0f0f0;
--md-code-fg-color: #f0f0f0;
}

/* Hide Temporian text in header. Uncomment if we swap the logo in the header for the full-text one.
Expand Down
Loading

0 comments on commit b585065

Please sign in to comment.