Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add processing of literal blocks in try examples directive #134

Merged
merged 3 commits into from
Feb 13, 2024
Merged

Add processing of literal blocks in try examples directive #134

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f5f9eef
Add processing of literal blocks in RST
steppi Feb 2, 2024
6ccf11c
Fix edge case in literal block processing
steppi Feb 2, 2024
b1586fa
Black fix code
steppi Feb 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
46 changes: 45 additions & 1 deletion jupyterlite_sphinx/_try_examples.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ def _append_code_cell_and_clear_lines(code_lines, output_lines, notebook):
def _append_markdown_cell_and_clear_lines(markdown_lines, notebook):
"""Append new markdown cell to notebook, clearing lines."""
markdown_text = "\n".join(markdown_lines)
# Convert blocks of LaTeX equations
markdown_text = _process_literal_blocks(markdown_text)
markdown_text = _process_latex(markdown_text)
markdown_text = _strip_ref_identifiers(markdown_text)
markdown_text = _convert_links(markdown_text)
Expand Down Expand Up @@ -216,6 +216,50 @@ def _process_latex(md_text):
return "\n".join(wrapped_lines)


def _process_literal_blocks(md_text):
md_lines = md_text.split("\n")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dumb questions, but how will this behave on windows where the line separator may be \r\n ? Or is it always \n in sphinx ? maybe:

Suggested change
md_lines = md_text.split("\n")
md_lines = md_text.splitlines()

I know there are other differences like if there is a terminal \n, or empty string.

Copy link
Collaborator Author

@steppi steppi Feb 13, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good question. Basically yes, I think we can always assume it's '\n' in Sphinx. Changing to splitlines won't affect the outcome, but it seems like a good change to make just because it better demonstrates intent.

The details: Python's open function has a newline kwarg whose default behavior is to normalize all line breaks to '\n', and this is what is used in Sphinx. So in Sphinx it's always '\n', regardless of the convention used by the RST files themselves. From the docs for open:

newline determines how to parse newline characters from the stream. It can be None, '', '\n', '\r', and '\r\n'. It works as follows:

When reading input from the stream, if newline is None, universal newlines mode is enabled. Lines in the input can end in '\n', '\r', or '\r\n', and these are translated into '\n' before being returned to the caller. If it is '', universal newlines mode is enabled, but line endings are returned to the caller untranslated. If it has any of the other legal values, input lines are only terminated by the given string, and the line ending is returned to the caller untranslated.*

When writing output to the stream, if newline is None, any '\n' characters written are translated to the system default line separator, os.linesep. If newline is '' or '\n', no translation takes place. If newline is any of the other legal values, any '\n' characters written are translated to the given string.

I think the only difference we'd see using splitlines() vs split('\n') is that, like you said, the former ignores terminal newlines, and I don't think this actually would make a difference. I think Jupyter just ignores trailing newlines in markdown cells.

new_lines = []
in_literal_block = False
literal_block_accumulator = []

for line in md_lines:
indent_level = len(line) - len(line.lstrip())

if in_literal_block and (indent_level > 0 or line.strip() == ""):
literal_block_accumulator.append(line.lstrip())
elif in_literal_block:
new_lines.extend(["```"] + literal_block_accumulator + ["```"])
literal_block_accumulator = []
if line.endswith("::"):
# If the line endswith ::, a new literal block is starting.
line = line[:-2] # Strip off the :: from the end
if not line:
# If the line contains only ::, we ignore it.
continue
else:
# Only set in_literal_block to False if not starting new
# literal block.
in_literal_block = False
# We've appended the entire literal block which just ended, but
# still need to append the current line.
new_lines.append(line)
else:
if line.endswith("::"):
# A literal block is starting.
in_literal_block = True
line = line[:-2]
if not line:
# As above, if the line contains only ::, ignore it.
continue
new_lines.append(line)

if literal_block_accumulator:
# Handle case where a literal block ends the markdown cell.
new_lines.extend(["```"] + literal_block_accumulator + ["```"])

return "\n".join(new_lines)


# https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html#docstring-sections
_non_example_docstring_section_headers = (
"Args",
Expand Down