Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
7922a74
Address review feedback for wrapping and footnotes
leynos Apr 18, 2026
5eb0acc
Tighten review follow-ups for tables and footnotes
leynos Apr 18, 2026
68dfaf7
Finish review fixes for renumber and wrap tests
leynos Apr 18, 2026
df08d31
Split renumber helpers and parameterize table tests
leynos Apr 18, 2026
07000dd
Simplify DefinitionParts path in renumber helpers
leynos Apr 18, 2026
c99f456
Avoid panic on missing footnote regex match
leynos Apr 18, 2026
31a27a7
Clarify architecture documentation
leynos May 25, 2026
a781071
Add direct coverage for refactored helpers
leynos May 25, 2026
4ac48af
Normalize whitespace-only lines during wrap
leynos May 30, 2026
6d88138
Address review follow-ups for renumber and wrap
leynos May 30, 2026
e454f0e
Address additional review follow-ups
leynos May 31, 2026
9a32b54
Tighten review follow-ups across wrap, footnotes, and docs
leynos May 31, 2026
a292ccb
Address review follow-ups for renumber and wrap
leynos May 31, 2026
690fe53
Reduce wrap test duplication and renumber complexity
leynos May 31, 2026
112524d
Clarify definition reorder warning docs
leynos May 31, 2026
1802832
Document test modules and trace conversions
leynos Jun 1, 2026
c12022d
Split oversized test modules
leynos Jun 1, 2026
4769fc3
Tidy HTML docs and rebalance tests
leynos Jun 1, 2026
a1accb5
Remove duplicate HTML conversion docs
leynos Jun 1, 2026
650d4e8
Anchor HTML table block detection
leynos Jun 1, 2026
1d362fc
Fix Markdown blank-line lint
leynos Jun 1, 2026
bb88c80
Tighten footnote docs and tests
leynos Jun 1, 2026
1874003
Add state-machine properties and tracing
leynos Jun 1, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@
labels, URLs, and optional titles are never collapsed into prose or split
across lines.
([`#292`](https://github.com/leynos/mdtablefix/issues/292))
- Normalize whitespace-only artefacts during wrapping by rebalancing atomic
tails.
- Preserve trailing spaces on the final line when wrapping Markdown, retaining
hard break semantics. See [trailing spaces](docs/trailing-spaces.md) for
details. ([#65](https://github.com/leynos/mdtablefix/issues/65))
Expand All @@ -50,3 +52,12 @@
[#252](https://github.com/leynos/mdtablefix/issues/252))
- Avoid converting numeric references in ATX heading text (including headings in
blockquotes and list items) when the `--footnotes` option is enabled.
- Compute continuation-line indentation from Unicode display width (via
`UnicodeWidthStr::width`) rather than byte or character count, so prefixes
containing full-width characters no longer misalign wrapped output.
- Convert `<table>...</table>` blocks that span multiple lines and carry
leading indentation, leaving surrounding non-table lines at the same
indentation level untouched.
- Correct the `cargo-binstall` Linux GNU `bin-dir` override so binaries are
installed from the archive's current directory (`.`) rather than a derived
`{ bin }{ binary-ext }` path, restoring `cargo binstall` on Linux.
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ the `--wrap` option is used, it also wraps paragraphs and list items to 80 colum

[package.metadata.binstall.overrides.'cfg(all(target_os = "linux", any(target_arch = "x86_64", target_arch = "aarch64"), target_env = "gnu"))']
pkg-url = "{ repo }/releases/download/v{ version }/{ name }-{ version }-{ target }.tar.gz"
bin-dir = "{ bin }{ binary-ext }"
bin-dir = "."
pkg-fmt = "tgz"

[dependencies]
Expand Down
20 changes: 20 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,26 @@ provides streaming helpers that combine the lower-level functions. The `io`
module handles filesystem operations, delegating the text processing to
`process`.

### Stateful helpers

`ParagraphWriter` owns paragraph buffering and flush boundaries for wrapping.
It keeps the current indent, emits wrapped or verbatim lines into the caller's
output buffer, and leaves inline fitting to the wrapping helpers.

`HtmlTableState` buffers candidate HTML table lines until the surrounding
table closes. Its depth counter tracks nested `<table>` blocks, so only the
outermost table is converted at once, while incomplete input can still be
flushed back verbatim.

The `footnotes::renumber::definitions` submodule owns definition scanning and
rewriting. `DefinitionScanState` coordinates the number mapping, collects
already-parsed definitions, and stages numeric candidates for later
conversion without cluttering the top-level renumber flow.

`ListState` tracks the active indentation stack and per-indent counters for
ordered list renumbering. It resets on headings and thematic breaks, and it
uses paragraph boundaries to decide when numbering should restart.

### Tokenizer flow

The inline tokenizer still iterates over the source string lazily, so no
Expand Down
45 changes: 45 additions & 0 deletions docs/developers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -567,6 +567,51 @@ committing. Snapshot churn across many cases usually means the fixture is too
broad or a shared transform changed behaviour; inspect the labelled case, mode,
and arguments before accepting the new output.

## Stateful pipeline helpers

Three internal types centralize the buffered state used by the conversion
pipeline. Each owns one slice of pipeline behaviour, so the surrounding
functions remain focused on traversal.

### `HtmlTableState` (`src/html.rs`)

`HtmlTableState` buffers the lines belonging to an HTML `<table>…</table>`
block and tracks the current nesting depth. `in_html()` returns `true`
whenever the buffer is non-empty, so the caller knows a table is still being
accumulated. `push_html_line` appends the supplied line, increments `depth`
once for every `<table>` start tag found on the trimmed line, and decrements
it once for every `</table>` end tag on the same trimmed line. When `depth`
returns to zero, the buffered lines are converted by `table_lines_to_markdown`
and the buffer is cleared. `flush_raw` exists for the fenced-block escape
path: it emits the buffered lines verbatim without conversion, so raw HTML
inside a fenced code block is preserved unchanged.

### `DefinitionScanState` (`src/footnotes/renumber/definitions.rs`)

`DefinitionScanState` accumulates the footnote-definition rewrite plan
during a single scan over the input. It borrows the shared `(original →
new)` mapping and the `next_number` counter so renumbering decisions stay
consistent with explicit reference rewrites. Explicit `[^n]:` headers are
appended to `definitions` as soon as they are encountered, producing a
`DefinitionLine` per header in scan order. Ordered-list items that look like
candidate footnote definitions are buffered as `NumericCandidate` entries
during the scan and finalized at the end via `finalize_numeric_candidates`,
which drains the buffer in reverse, so the assigned numbers reflect
bottom-up ordering rather than the order in which the candidates were
discovered.

### `ListState` (`src/lists.rs`)

`ListState` maintains an indent stack and a per-indent counter map for
ordered-list renumbering. `next_number(indent)` first prunes indent levels
deeper than `indent` (their counters disappear so a future deeper level
restarts at 1), pushes `indent` onto the stack if it is new, and returns the
next sequential number for that level — incrementing the counter, so the next
call at the same indent receives the following integer. `reset()` clears
both the stack and the counter map; the renumbering pass invokes it when a
heading or thematic break is encountered, so the next list starts numbering
from 1 again.

## Test infrastructure

### `tests/support/` module
Expand Down
24 changes: 24 additions & 0 deletions docs/users-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,30 @@ Two trailing spaces at the end of a line produce a hard line break in rendered
Markdown. `mdtablefix --wrap` preserves those trailing spaces on the final
wrapped line, so hard-break semantics are not lost after reformatting.

Lines that consist entirely of whitespace — spaces, tabs, or any mixture — are
normalized to empty strings during wrapping. Such lines act as paragraph
boundaries and are never passed through with their original whitespace
content, so the output uses a single uniform separator between paragraphs
regardless of the input's incidental indentation.

When computing the indentation width for continuation lines in prefixed
contexts (blockquotes, lists, and footnote definitions), `mdtablefix` measures
the prefix using Unicode display width (`UnicodeWidthStr::width`) rather than
byte or character count. Continuation lines therefore stay correctly aligned
when the prefix contains full-width characters such as ideographic spaces or
CJK punctuation.

## HTML table conversion

`mdtablefix` converts `<table>…</table>` blocks that span multiple lines and
carry leading indentation into Markdown pipe tables. The leading indentation
is preserved on every emitted row, so the converted table sits at the same
indentation level as the original HTML. Surrounding non-table lines at that
same indentation level are passed through unchanged. Nested `<table>` tags
are tracked by depth, so the buffered structure is converted only once the
outermost `</table>` is reached and never split into two separate
conversions.

## Fence normalization

Pass `--fences` to normalize fenced code blocks before later processing. Safe
Expand Down
11 changes: 11 additions & 0 deletions src/code_emphasis.rs
Original file line number Diff line number Diff line change
Expand Up @@ -261,4 +261,15 @@ mod tests {
let input = vec!["before `code` after".to_string()];
assert_eq!(fix_code_emphasis(&input), input);
}

#[test]
fn consume_code_affixes_clears_mixed_pending_prefix() {
let mut tokens = vec![Token::Text("*lead*tail")].into_iter().peekable();
let mut pending = "**";

let (prefix, suffix, modified) = consume_code_affixes(&mut tokens, &mut pending);

assert_eq!((prefix, suffix, modified), ("", "", true));
assert_eq!(tokens.next(), Some(Token::Text("lead*tail")));
}
}
Loading
Loading