Skip to content

[M3-P1] Configure Jupytext Paired Sync for Pilot Notebooks #975

@Gorkowski

Description

@Gorkowski

Description

Set up the Jupytext paired sync infrastructure for the pilot notebooks in Activity/ and Gas_Phase/ directories. This phase creates the configuration files needed before converting notebooks to paired format.

Context

Part of: M3: Jupytext Notebook Sync Migration (Pilot)

This is Phase 1 of the Jupytext pilot migration. The goal is to enable LLM-friendly notebook editing by pairing .ipynb files with .py:percent files that can be linted, type-checked, and have readable diffs.

Value:

  • Enable ruff/mypy linting on example code
  • Clean PR diffs instead of JSON noise
  • Standard git merge conflict resolution
  • Foundation for pilot notebook conversion

Scope

Estimated Lines of Code: ~30 lines of configuration changes
Complexity: Low (configuration only, no code)

Files to Create:

  • jupytext.toml (~5 LOC)

Files to Modify:

  • pyproject.toml (~15 LOC changes to [tool.ruff] section)
  • mkdocs.yml (~2 LOC addition to exclude_docs)
  • .gitignore (~2 LOC addition)

Acceptance Criteria

Core Implementation

  • Create jupytext.toml at repository root with pairing rules:

    # Jupytext configuration for particula
    # Pairs notebooks in docs/Examples with percent-format Python scripts
    
    [formats]
    "docs/Examples/**/*.ipynb" = "ipynb,py:percent"
  • Update pyproject.toml [tool.ruff] section:

    # UPDATE src to include docs/Examples
    src = ["particula", "docs/Examples"]
    include = ["particula/**/*.py", "docs/Examples/**/*.py"]
    
    # Keep existing extend-exclude unchanged:
    extend-exclude = [
      "**/*.ipynb",
    ]
  • Add per-file-ignores for example scripts in pyproject.toml:

    [tool.ruff.lint.per-file-ignores]
    "*_test.py" = ["S101", "E721", "B008"]
    "docs/Examples/**/*.py" = ["D100", "D103", "INP001"]

    Note: D100 = missing module docstring, D103 = missing function docstring, INP001 = implicit namespace package (no __init__.py)

  • Update mkdocs.yml exclude_docs section:

    exclude_docs: |
      .assets/
      Examples/**/*.py
  • Add to .gitignore (after line 103, Jupyter Notebook section):

    # Jupyter notebook backups (created by run_notebook tool)
    *.ipynb.bak

Validation (No separate tests - configuration validation)

  • jupytext.toml exists and is valid TOML
  • ruff check particula/ still passes (existing code not affected)
  • ruff check docs/Examples/Activity/ docs/Examples/Gas_Phase/ runs without config errors (may have linting issues in existing .py files - that's expected)
  • mkdocs build succeeds without errors

Technical Notes

Why These Specific Ignores?

The docs/Examples/**/*.py ignores are intentional:

  • D100 (missing module docstring): Example scripts don't need module docstrings
  • D103 (missing function docstring): Example code prioritizes brevity
  • INP001 (implicit namespace package): Examples aren't importable packages

Current pyproject.toml State

Currently (lines 57-86):

[tool.ruff]
line-length = 80
fix = true
src = ["particula"]
include = ["particula/**/*.py"]
extend-exclude = [
  "**/*.ipynb",
]

[tool.ruff.lint.per-file-ignores]
"*_test.py" = ["S101", "E721", "B008"]

Current mkdocs.yml State

Currently (lines 8-9):

exclude_docs: |
  .assets/

Current .gitignore State

Line 102-103 has:

# Jupyter Notebook
.ipynb_checkpoints

Add backup entry after this section.

Validation Commands

# Verify jupytext.toml is valid
python -c "import tomllib; tomllib.load(open('jupytext.toml', 'rb'))"

# Verify ruff config works
ruff check particula/ --select=E --max-errors=0 || echo "Expected: passes"
ruff check docs/Examples/Activity/ --select=E || echo "May have linting issues"

# Verify mkdocs builds
mkdocs build --strict 2>&1 | head -20

References

Maintenance Plan:

  • adw-docs/dev-plans/maintenance/M3-jupytext-notebook-sync.md - Full migration plan

Configuration References:

  • pyproject.toml (lines 57-86) - Current ruff config
  • mkdocs.yml (lines 8-9) - Current exclude_docs
  • .gitignore (lines 102-103) - Jupyter section

Documentation:

  • adw-docs/documentation_guide.md (lines 341-534) - Jupytext workflow docs

Metadata

Metadata

Assignees

No one assigned

    Labels

    agentCreated or managed by ADW automationmaintenanceMaintenance work - refactoring, technical debt, testingmodel:defaultUse base/sonnet tier (workflow default)type:patchQuick patch workflow (plan → build → ship)

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions