Skip to content

fix: split a docstring that follows code into its own notebook cell - #214

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/remove-finish-docstring-hack
Jul 30, 2026
Merged

fix: split a docstring that follows code into its own notebook cell#214
Jammy2211 merged 1 commit into
mainfrom
feature/remove-finish-docstring-hack

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Closes the generator half of #211.

What was broken

ipynb-py-convert's py2nb splits cells on the literal "\n\n# %%\n" — the marker must be preceded by a blank line. The docstring-opener branch of add_notebook_quotes emitted "# %%\n" after a single newline, so a column-0 docstring opened on the line immediately after code never split. The marker and both delimiters ended up inside the preceding code cell as literal text:

np.save(
    file=dataset_path / f"nufft_precision_operator_{mask_radius}.npy",
    arr=nufft_precision_operator,
    allow_pickle=False,
)
# %%
'''
To load the `nufft_precision_operator` matrix from hard-disk in your model-fit, you can use the code:
'''

That cell is a SyntaxError if a user runs it. 13 committed notebook code cells were broken this way, across autolens_workspace (4), autogalaxy_workspace (3), autocti_workspace (4) and HowToLens (2).

The fix

Emit the separator when it is missing, under two constraints — both load-bearing, both verified:

  1. Never when out is empty. py2nb strips a leading # %%\n header; a leading blank line defeats that strip and yields a spurious empty first code cell.
  2. Never when the output already ends blank. Emitting it unconditionally appends a trailing blank line to every code cell in every generated notebook. Verified empirically: regenerating autofit_workspace with this change produces a zero-byte diff, and HowToLens changes in exactly its 2 broken notebooks and nothing else.

Also raises on a hand-written column-0 # %% in a source script. This function is what inserts cell markers, so an authored one collides with the generated one and corrupts the conversion the same way — guides/hpc/example_cpu_and_gpu.py in two workspaces carried these for years. Loud beats laundering it into a broken artefact.

Tests

test pins
test_docstring_immediately_after_code_is_its_own_markdown_cell the bug. Fails on main: ['markdown', 'code', 'code'] instead of ['markdown', 'code', 'markdown', 'code']
test_hand_written_cell_marker_in_source_raises the new guard
test_script_ending_in_code_keeps_a_complete_final_code_cell that a script ending in code converts intact — the pin that makes the workspaces' trailing """Finish.""" crutch provably unnecessary
test_leading_docstring_does_not_produce_an_empty_first_code_cell constraint 1

The two pre-existing adjacent-docstring tests pass unchanged.

6 passed in 0.43s

Merge order

This must merge before the workspace PRs in #211 — workspaces invoke ../PyAutoHands/autohands/generate.py from the local checkout, so their regenerated notebooks are only correct against this version.

🤖 Generated with Claude Code

https://claude.ai/code/session_013XzXdQMU3KV2tyZaMNPvsM

`ipynb-py-convert`'s `py2nb` splits cells on the literal "\n\n# %%\n" — the
marker must be preceded by a blank line. The docstring-opener branch of
`add_notebook_quotes` emitted "# %%\n" after a single newline, so a column-0
docstring opened on the line immediately after code never split: the marker and
both delimiters ended up inside the preceding code cell as literal text, making
that cell a SyntaxError for anyone who ran it. 13 committed notebook cells
across four workspace repos were broken this way.

Emit the separator when it is missing, under two constraints. Never when `out`
is empty: `py2nb` strips a leading "# %%\n" header, and a leading blank line
defeats that strip and yields a spurious empty first code cell. Never when the
output already ends blank: emitting it unconditionally would append a trailing
blank line to every code cell in every generated notebook.

Also raise on a hand-written column-0 `# %%` in a source script. This function
is what inserts cell markers, so an authored one collides with the generated
one and corrupts the conversion the same way; two workspace scripts carried
these for years. Loud beats laundering it into a broken artefact.

Tests cover the docstring-after-code split, the stray-marker raise, and — as
the pin that makes the workspaces' trailing `"""Finish."""` crutch provably
unnecessary — that a script ending in a code cell converts intact.

Refs #211

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XzXdQMU3KV2tyZaMNPvsM
@Jammy2211
Jammy2211 force-pushed the feature/remove-finish-docstring-hack branch from 26a690e to 774067c Compare July 30, 2026 10:10
Jammy2211 pushed a commit to PyAutoLabs/autolens_workspace that referenced this pull request Jul 30, 2026
61 occurrences: 40 trailing blocks, 17 in-block lines, 2 sentence-leading, 1 empty `__Finish__` header, 1 commented-out block.

These trailing docstring blocks were a crutch against notebook generation
"cutting off weird" when a script's last cell was not a docstring. That shape
converts correctly and is now pinned by a regression test in PyAutoHands
(PyAutoLabs/PyAutoHands#214), so the crutch is dead weight — it rendered as a
pointless final markdown cell in every generated notebook.

Refs PyAutoLabs/PyAutoHands#211

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XzXdQMU3KV2tyZaMNPvsM
Jammy2211 pushed a commit to PyAutoLabs/autogalaxy_workspace that referenced this pull request Jul 30, 2026
22 occurrences: 9 trailing blocks, 12 in-block lines, 1 empty `__Finish__` header.

These trailing docstring blocks were a crutch against notebook generation
"cutting off weird" when a script's last cell was not a docstring. That shape
converts correctly and is now pinned by a regression test in PyAutoHands
(PyAutoLabs/PyAutoHands#214), so the crutch is dead weight — it rendered as a
pointless final markdown cell in every generated notebook.

Refs PyAutoLabs/PyAutoHands#211

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XzXdQMU3KV2tyZaMNPvsM
@Jammy2211
Jammy2211 merged commit 2844d0f into main Jul 30, 2026
Jammy2211 pushed a commit to PyAutoLabs/autocti_workspace that referenced this pull request Jul 30, 2026
35 occurrences: 34 trailing blocks, 1 in-block line.

These trailing docstring blocks were a crutch against notebook generation
"cutting off weird" when a script's last cell was not a docstring. That shape
converts correctly and is now pinned by a regression test in PyAutoHands
(PyAutoLabs/PyAutoHands#214, merged), so the crutch is dead weight.

notebooks/ is NOT regenerated: generate.py rejects project 'autocti' (absent
from COLAB_PROJECTS and PyAutoNerves _PROJECTS). Filed separately.

Rebuilt on top of #15 rather than rebased — that PR normalised
scripts/dataset_1d/advanced/database/examples/data_fitting.py and
scripts/plot/plotters/ImagingCIPlotter.py from CRLF to LF, which made both files
whole-file conflicts. This commit follows main's line endings.

Refs PyAutoLabs/PyAutoHands#211

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XzXdQMU3KV2tyZaMNPvsM
@Jammy2211
Jammy2211 deleted the feature/remove-finish-docstring-hack branch July 30, 2026 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant