Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs (Closes #110) #111

Merged
merged 30 commits into from
Nov 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
210ee74
Use Python 3.10 in pre-commit to prevent mismatch.
hmgaudecker Oct 18, 2022
29d3208
Fix paths.
hmgaudecker Oct 18, 2022
420634c
Split up getting started section of docs.
hmgaudecker Oct 18, 2022
f726ed3
Use pytask .pre-commit-config.yaml
timmens Oct 18, 2022
b8f5b33
Remove empty sections from getting_started
timmens Oct 18, 2022
9441924
Change order of sections.
hmgaudecker Oct 19, 2022
121c185
Smaller updates and cleanup.
hmgaudecker Oct 19, 2022
9c6503a
Update documentation and cookiecutter.json
timmens Oct 20, 2022
58c449b
Update documentation and cookiecutter.json
timmens Oct 20, 2022
037452f
Adjust README so that most tests run again.
hmgaudecker Oct 21, 2022
a229f1c
Better warnings in cookiecutter-dialogue
timmens Oct 21, 2022
c3ca691
Update documentation build
timmens Oct 27, 2022
d0b6cd5
Fix Python version error in GHA
timmens Oct 27, 2022
97c782c
Fix Python version error in GHA
timmens Oct 27, 2022
34e4bb5
Fix error in ci workflow yaml
timmens Oct 27, 2022
67ddb8d
Delete figures that are created automatically
timmens Oct 28, 2022
9bae04d
Complete environment for figures, let pytask fail upon first error, p…
hmgaudecker Nov 8, 2022
b51ae0f
Continued with docs.
hmgaudecker Nov 11, 2022
a40bfcb
More updates to docs & additions to directories defined in config.
hmgaudecker Nov 12, 2022
51c12cb
Use sensible LaTeX pipeline for typical installations, remove changin…
hmgaudecker Nov 12, 2022
762f149
Mostly done with text.
hmgaudecker Nov 12, 2022
543b152
Make data local
timmens Nov 15, 2022
2531716
Use explicit latex dependencies on GA
timmens Nov 15, 2022
5e74309
Update paper and environments
timmens Nov 15, 2022
2de481d
Move to Python 3.11 (#114)
timmens Nov 15, 2022
1e35c85
Add generated figures
timmens Nov 15, 2022
e3a0af1
Merge branch 'update-docs' of https://github.com/OpenSourceEconomics/…
timmens Nov 15, 2022
e756513
Move figure generation into conf.py
timmens Nov 15, 2022
fc0ddb0
Upload generated figures
timmens Nov 15, 2022
aff2e67
Move pip-installed package to the very end, else it would not get ins…
hmgaudecker Nov 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 10 additions & 14 deletions .github/workflows/continuous-integration-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,16 @@ concurrency:
group: ${{ github.head_ref || github.run_id }}
cancel-in-progress: true


on:
push:
branches:
- master
- master
pull_request:
branches:
- '*'
- "*"

jobs:

run-tests:

name: Run tests for ${{ matrix.os }} on ${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
env:
Expand All @@ -26,8 +23,8 @@ jobs:
strategy:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest', 'windows-latest']
python-version: ['3.8', '3.9', '3.10']
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
python-version: ["3.9", "3.10", "3.11"]

steps:
- uses: actions/checkout@v2
Expand All @@ -37,6 +34,8 @@ jobs:
- uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
channels: conda-forge,defaults
channel-priority: true
python-version: ${{ matrix.python-version }}

- name: Install core dependencies.
Expand All @@ -48,33 +47,29 @@ jobs:
run: tox -e pytest -- tests -m end_to_end --cov=./ --cov-report=xml -n auto

- name: Upload coverage reports of end-to-end tests.
if: runner.os == 'Linux' && matrix.python-version == '3.9'
if: runner.os == 'Linux' && matrix.python-version == '3.11'
shell: bash -l {0}
run: bash <(curl -s https://codecov.io/bash) -F end_to_end -c


pre-commit:

name: Run pre-commit.
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- name: Set up Python 3.9
- name: Set up Python 3.11
uses: actions/setup-python@v1
with:
python-version: 3.9
python-version: 3.11

- name: Install dependencies
run: pip install tox

- name: Run pre-commit
run: tox -e pre-commit


docs:

name: Run documentation.
runs-on: ubuntu-latest

Expand All @@ -83,6 +78,7 @@ jobs:
- uses: conda-incubator/setup-miniconda@v2
with:
auto-update-conda: true
python-version: 3.9

- name: Install core dependencies.
shell: bash -l {0}
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
docs/scripts/latex/*.png
*-blx.bib
*.asv
*.aux
Expand Down
161 changes: 111 additions & 50 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,63 +1,124 @@
# Exclude every file or folder starting with a dot.
exclude: ^(\.|\{\{cookiecutter\.project_slug\}\}/.+)
exclude: ^(\.|{{cookiecutter\.project_slug}}/.+)

repos:
- repo: https://github.com/asottile/reorder_python_imports
rev: v3.8.5
hooks:
- id: reorder-python-imports
args: [--unclassifiable-application-module=src]
- repo: https://github.com/pre-commit/pre-commit-hooks
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.3.0
hooks:
- id: check-yaml
- id: check-added-large-files
args: ["--maxkb=100000"] # Set to max file size in Github
- id: check-byte-order-marker
types: [text]
- id: check-merge-conflict
- id: check-json
- id: pretty-format-json
args: [--autofix, --no-sort-keys, --no-ensure-ascii]
- id: trailing-whitespace
- repo: https://github.com/asottile/pyupgrade
rev: v3.1.0
hooks:
- id: pyupgrade
- id: check-added-large-files
args: ['--maxkb=25']
- id: check-case-conflict
- id: check-merge-conflict
- id: check-vcs-permalinks
- id: check-yaml
exclude: |
(?x)^(
{{cookiecutter.project_slug}}/environment.yml|
{{cookiecutter.project_slug}}/.pre-commit-config.yaml
)$
- id: debug-statements
- id: end-of-file-fixer
- id: fix-byte-order-marker
- id: mixed-line-ending
- id: no-commit-to-branch
args: [--branch, main]
- id: trailing-whitespace
- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.9.0
hooks:
- id: python-check-blanket-noqa
- id: python-check-mock-methods
- id: python-no-eval
- id: python-no-log-warn
- id: python-use-type-annotations
- id: text-unicode-replacement-char
- repo: https://github.com/asottile/pyupgrade
rev: v2.37.1
hooks:
- id: pyupgrade
args: [--py37-plus]
- repo: https://github.com/asottile/blacken-docs
rev: v1.12.1
hooks:
- id: blacken-docs
additional_dependencies: [black]
files: '(\.md|\.rst)$'
- repo: https://github.com/psf/black
rev: 22.10.0
hooks:
- id: black
language_version: python3.9
- repo: https://github.com/PyCQA/flake8
rev: 5.0.4
hooks:
- id: flake8
exclude: docs/bld/example/r/r_example/src/documentation/conf.py
additional_dependencies:
[
- repo: https://github.com/asottile/reorder_python_imports
rev: v3.8.1
hooks:
- id: reorder-python-imports
- repo: https://github.com/psf/black
rev: 22.6.0
hooks:
- id: black
- repo: https://github.com/myint/docformatter
rev: v1.3.1
hooks:
- id: docformatter
args: [--in-place, --wrap-summaries, "88", --wrap-descriptions, "88", --blank]
- repo: https://github.com/PyCQA/flake8
rev: 4.0.1
hooks:
- id: flake8
additional_dependencies: [
flake8-alfred,
flake8-bugbear,
flake8-builtins,
flake8-comprehensions,
flake8-deprecated,
flake8-docstrings,
flake8-eradicate,
flake8-print,
flake8-rst-docstrings,
flake8-pytest-style,
flake8-todo,
flake8-typing-imports,
flake8-unused-arguments,
pep8-naming,
pydocstyle,
]
# Harmonizing flake8 and black
args:
[
"--max-line-length=88",
"--ignore=E203,E402,E501,E800,W503",
"--select=B,C,E,F,W,T4,B9",
]
Pygments,
]
args: [--exclude, "conf.py, docs/"]
- repo: https://github.com/econchick/interrogate
rev: 1.5.0
hooks:
- id: interrogate
args: [
-v,
--fail-under=40,
--config,
pyproject.toml,
--exclude,
"{{cookiecutter.project_slug}}/"
]
pass_filenames: false
- repo: https://github.com/executablebooks/mdformat
rev: 0.7.14
hooks:
- id: mdformat
additional_dependencies: [
mdformat-gfm,
mdformat-black,
]
args: [--wrap, "88"]
files: (README\.md)
- repo: https://github.com/executablebooks/mdformat
rev: 0.7.14
hooks:
- id: mdformat
additional_dependencies: [
mdformat-myst,
mdformat-black,
]
args: [--wrap, "88"]
files: (docs/.)
# Exclude files with admonitions.
# exclude: |
# (?x)^(
# path/to/file.py
# )$
- repo: https://github.com/codespell-project/codespell
rev: v2.1.0
hooks:
- id: codespell
- repo: https://github.com/guilatrova/tryceratops
rev: v1.1.0
hooks:
- id: tryceratops
exclude: "{{cookiecutter.project_slug}}/pyproject.toml"
- repo: meta
hooks:
- id: check-hooks-apply
- id: check-useless-excludes
# - id: identity # Prints all files passed to pre-commits. Debugging.
7 changes: 2 additions & 5 deletions cookiecutter.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@
"github_email": "{{ cookiecutter.email }}",
"git_remote_url": "",
"version": "0.0.1",
"python_version": "3.8",
"python_version": "3.11",
"is_ci": ["no", "yes"],
"add_tox": [
"yes",
"no"
Expand All @@ -22,10 +23,6 @@
"yes",
"no"
],
"add_readthedocs": [
"yes",
"no"
],
"add_codecov": [
"yes",
"no"
Expand Down
2 changes: 1 addition & 1 deletion docs/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Minimal makefile for Sphinx documentation
#


# You can set these variables from the command line, and also
# from the environment for the first two.
Expand Down
2 changes: 1 addition & 1 deletion docs/rtd_environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ channels:
- nodefaults

dependencies:
- python=3.9
- python==3.9
- pip
- setuptools_scm
- toml
Expand Down
18 changes: 11 additions & 7 deletions docs/scripts/task_create_visualization.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@
import pytask

SRC = Path(__file__).parent.resolve() # root/docs/scripts
FIGURES = SRC.parent.resolve().joinpath("source").joinpath("figures").resolve()
FIGURES = SRC.parent.joinpath("source", "figures", "generated").resolve()
sys.path.append(str(SRC))

from visualization_functions import visualize_organisational_steps
from visualization_functions import visualize_organisational_steps # noqa: E402


for case in ["model_steps_full", "model_steps_select", "steps_only_full"]:
Expand All @@ -28,16 +28,20 @@ def task_visualize_organisational_steps(produces, case):

for tex_file in ["root_bld_src", "src"]:

@pytask.mark.depends_on(SRC.joinpath("latex").joinpath(f"{tex_file}.tex"))
@pytask.mark.produces(SRC.joinpath(f"{tex_file}.png"))
@pytask.mark.depends_on(SRC.joinpath("latex", f"{tex_file}.tex"))
@pytask.mark.produces(SRC.joinpath("latex", f"{tex_file}.png"))
@pytask.mark.task(id=tex_file)
def task_compile_latex(depends_on):
subprocess.Popen(["pdflatex", "--shell-escape", str(depends_on)])
subprocess.run(
("pdflatex", "--shell-escape", depends_on.name), cwd=depends_on.parent
)
time.sleep(1)
subprocess.Popen(["pdflatex", "--shell-escape", str(depends_on)])
subprocess.run(
("pdflatex", "--shell-escape", depends_on.name), cwd=depends_on.parent
)
time.sleep(1)

@pytask.mark.depends_on(SRC.joinpath(f"{tex_file}.png"))
@pytask.mark.depends_on(SRC.joinpath("latex", f"{tex_file}.png"))
@pytask.mark.produces(FIGURES.joinpath(f"{tex_file}.png"))
@pytask.mark.task(id=tex_file)
def task_copy_png_to_figures(depends_on, produces):
Expand Down
5 changes: 3 additions & 2 deletions docs/scripts/visualization_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ def visualize_organisational_steps(case):
"""
fig = go.Figure()

# define coordinate dimenions
# define coordinate dimensions
fig.add_trace(
go.Scatter(
x=[0.75, 4.3],
Expand Down Expand Up @@ -85,6 +85,7 @@ def update_fig_with_shape(case, fig):
)
else:
raise ValueError(
"Case must be in {'steps_only_full', 'model_steps_full', 'model_steps_select'}"
"Case must be in {'steps_only_full', 'model_steps_full', "
"'model_steps_select'}"
)
return fig
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,4 @@ different specifications. As a case in point, consider the DAG for an early vers

Do you want to keep those dependencies in your head? Or would it be useful to
specify them once and for all in order to have more time for thinking about
research? The next section shows you how to do that.
research? The next section shows you how to do that.
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,7 @@ The design of the project templates is guided by the following main thoughts:
#. **Re-use of code and data instead of copying and pasting:** Else you will forget the
copy & paste step at some point down the road. At best, this leads to errors; at
worst, to misinterpreting the results.

..
comment:: What is the difference between re-use code and copy-paste?

#. **Be as language-agnostic as possible:** Make it easy to use the best tool for a
particular task and to mix tools in a project.
#. **Separation of inputs and outputs:** Required to find your way around in a complex
project.

I will not touch upon the last point until the section on how to organise the workflow
below. The remainder of this page introduces an example and a general concept of how to
think about the first four points.
Loading