Document SEP_RE constant and ADR for table reflow pipeline

[leynos]

Summary

• Documents SEP_RE constant with a comprehensive doc comment detailing its purpose, matching rules, and edge cases
• Introduces Architecture Decision Record (ADR 0001) for the table reflow pipeline, including Unicode width handling and continuation-row preservation
• Updates architecture and developer/user guides to describe the reflow pipeline and behavior
• Adds tests covering continuation rows, ellipsis handling, and Unicode width alignment

Changes

Documentation

• SEP_RE: Added documentation in code comments describing the detection of Markdown separator lines and escape behavior
• ADR 0001: Added docs/adrs/0001-table-reflow-pipeline.md outlining the table reflow pipeline decisions
• Updated docs/architecture.md to describe the table reflow pipeline, Unicode width usage, and minimum separator widths
• Updated docs/developers-guide.md detailing internal API surface for the reflow pipeline
• Added docs/users-guide.md describing table reflow, continuation rows, and ellipsis handling

Tests

• Added tests/table_continuations.rs covering continuation-row preservation and width alignment
• Extended tests to verify ellipsis handling and Unicode-aware widths

Test Plan

• Run cargo test to verify all tests pass
• Specifically run the new tests in tests/table_continuations.rs
• Manually verify that tables with continuation rows align and maintain column structure

Potential Impact

• Improves reliability for Markdown tables with continuation rows and ellipsis
• Documentation-focused change with no user-facing API changes; maintains backward compatibility
• Unicode-aware widths ensure proper alignment for non-ASCII content

📎 Task: https://www.devboxer.com/task/bdf32d63-a27d-41fc-8a83-b480791b5ddc
📎 Task: https://www.devboxer.com/task/76f086fe-74e1-4c40-b366-aa9edf3940ad
📎 Task: https://www.devboxer.com/task/d99b9fbc-b00d-4c00-aab1-1b14e4c9a375
📎 Task: https://www.devboxer.com/task/22be2748-8bcc-4708-8429-541b372e777e

📎 Task: https://www.devboxer.com/task/e2d0b3a9-732f-498e-ba08-625f76da0beb

[coderabbitai]

Introduces Table Reflow Pipeline with Unicode Width, Continuation-Row Preservation, Ellipsis Ordering, and Comprehensive Tests & Docs

This PR implements a staged Markdown table reflow pipeline with Unicode display-width alignment, explicit handling for leading-empty continuation rows, ellipsis-before-reflow ordering, separator validation, and broad unit/integration test coverage. The changes are documented by ADR 0001 and covered in updated architecture, developer, user guides and an updated execplan document.

Key Behavioural Changes

• Continuation rows: leading-empty cells are protected with a private LEADING_EMPTY_CELL_MARKER by protect_leading_empty_cells before sentinel-based splitting and restored by clean_rows after parsing so continuation rows retain original column positions (indentation/prefix preservation intentionally out-of-scope).
• Unicode-aware widths: column widths use UnicodeWidthStr display widths (via emitted_cell_width) rather than byte/len; pad_cell_to_width pads cells after escaping literal pipes to ensure alignment for wide glyphs and emojis.
• Ellipsis ordering: ProcessBuffer now exposes an ellipsis flag; when enabled, ellipsis replacement (… for ...) is applied to buffered table lines before reflow so width calculations reflect the final emitted glyphs; when disabled literal "..." are preserved (covered by tests).
• Pipe escaping: restored non-leading cells have literal '|' re-escaped (as |) where required to preserve correct splitting on subsequent parses.
• Separator handling: SEP_RE improves separator detection; separator columns are clamped to a minimum width of three dashes when present. format_separator_cells validates separator formatting; on mismatch reflow_table falls back to returning the original lines (defensive behaviour).
• Robust parsing: parse_rows now inserts protection markers prior to splitting; clean_rows restores markers and removes fully-empty rows. split_cells and separator handling were refactored to support the pipeline.

Utilities & API (internal / pub(crate))

• New/modified internal helpers and utilities: LEADING_EMPTY_CELL_MARKER, protect_leading_empty_cells, escape_literal_pipes (implicit in protection flow), emitted_cell_width, pad_cell_to_width, format_separator_cells, and SEP_RE (LazyLock regex).
• Signatures of key internal functions unchanged, but behaviour updated: parse_rows, clean_rows, calculate_widths, format_rows. calculate_and_format now returns Option<Vec> to allow safe fallback on separator mismatch.

Tests

• New and expanded tests:
  • tests/table_continuations.rs: continuity-row behaviour, escaped pipes + continuation rows, ellipsis enabled/disabled integration tests, and parameterised Unicode-width alignment tests.
  • src/reflow/tests.rs: unit tests for marker protection, escape handling, emitted_cell_width, pad_cell_to_width, calculate_widths and format_rows.
  • Additional regression/unit tests ensure ellipsis ordering, separator mismatch fallback, and continuation-row preservation.
• Tests assert widened spacing/alignment changes and cover both ellipsis paths.

Documentation

• ADR: docs/adrs/0001-table-reflow-pipeline.md (design rationale and ordering constraints).
• Architecture: docs/architecture.md updated with a "Table reflow pipeline" section documenting staged processing and ordering-sensitive steps (including re-escaping pipes and ellipsis-before-reflow).
• Developer & User guides: docs/developers-guide.md and docs/users-guide.md added/updated with API responsibilities and user-facing behaviour (continuation rows, Unicode alignment, escaped pipes, --ellipsis).
• Execplan doc: docs/execplans/yaml-frontmatter.md updated (make nixie requirement now unconditional; minor formatting fixes).
• Minor docs: module-level doctest/example import path updated (src/wrap/tokenize/mod.rs).

Backward Compatibility & Fallbacks

• Tables without continuation rows remain compatible.
• When separator rows don't match computed columns or separator formatting fails, reflow_table preserves original input lines rather than producing an invalid table.

Notes from Review Thread

• Reviewer concerns about preserving original leading indentation when reconstructing protected rows were acknowledged but intentionally marked out-of-scope for this PR; continuation-row column positions are preserved via the marker strategy and covered by tests.
• Requested unit tests and documentation gaps from CI were addressed: helper unit tests, pipe-escaping behaviour, ADR and docs were added/expanded.

CI / Developer guidance

• Recommended to run cargo test and verify against markdownlint; the PR includes tests exercising the behavioral changes.

Walkthrough

Augment the Markdown table reflow pipeline to protect leading-empty continuation cells with a marker, compute column widths using Unicode display width, enforce a minimum separator width of three dashes, and apply ellipsis replacement to buffered table lines before reflow. Update process buffering, parsing/cleaning, separator formatting, tests, and documentation.

Changes

Cohort / File(s)	Summary
Process & entrypoint src/process.rs	Add ellipsis flag to ProcessBuffer; flush now takes buffered lines once, applies ellipsis replacement for table buffers before reflow, and routes through table or passthrough paths.
Reflow helpers src/reflow.rs	Protect leading empty cells with a marker before sentinel splitting; restore markers and drop fully-empty rows in clean_rows; compute widths using UnicodeWidthStr::width (escaped-pipe-aware); add pad_cell_to_width and helpers for pipe-escaping and emitted-width calculation; guard separator insertion on empty output.
Table formatting src/table.rs	Add format_separator_cells to widen separator dashes to computed column widths (min 3) while preserving : markers; make calculate_and_format return Option<Vec<String>> and fall back to original input when separators are incompatible; reposition #[must_use] on split_cells.
Reflow & table tests src/reflow/tests.rs, tests/table_continuations.rs	Add unit and integration tests for marker protection, pipe escaping, padding/width calculations (including Unicode/CJK/emoji), separator marker preservation, continuation-row behaviour, ellipsis ordering, and fallback on invalid separators.
Documentation & ADRs docs/adrs/0001-table-reflow-pipeline.md, docs/architecture.md, docs/developers-guide.md, docs/users-guide.md	Document marker-based continuation protection, sentinel-safe parsing, escape/restoration rules, Unicode display-width measurement, minimum separator width policy, and ellipsis-before-reflow ordering; add ADR and developer/user guidance.
Misc docs & examples docs/execplans/yaml-frontmatter.md, src/wrap/tokenize/mod.rs	Update make nixie requirement text and minor punctuation; fix example import path from crate::wrap to mdtablefix::wrap.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Process Entry
    participant Buffer as ProcessBuffer
    participant Protector as protect_leading_empty_cells()
    participant Parser as parse_rows()
    participant Cleaner as clean_rows()
    participant WidthCalc as calculate_widths()
    participant Formatter as format_rows()
    participant TableFmt as format_separator_cells()
    participant Emitter as Output

    Client->>Buffer: call flush(buffered_lines, ellipsis?)
    alt Detected as table block
        Buffer->>Buffer: replace_ellipsis(buffered) if ellipsis enabled
        Buffer->>Protector: insert leading-empty markers per line
        Protector->>Parser: feed protected lines
        Parser->>Cleaner: parsed rows + split flag
        Cleaner->>WidthCalc: rows (markers restored, empty rows removed)
        WidthCalc->>Formatter: column widths (unicode display width)
        Formatter->>TableFmt: separator cells + widths
        TableFmt->>Formatter: formatted separator (or signal invalid)
        alt Separator valid
            Formatter->>Emitter: emit reflowed table lines
        else Separator invalid
            Formatter->>Emitter: emit original input lines (fallback)
        end
    else Not a table
        Buffer->>Emitter: emit buffered lines unchanged
    end

Loading

Poem

Guard the gaps with markers bright,
Escape the pipes to hold their right,
Swap three dots for single glyph,
Stretch separators, three dashes stiff,
Tables align and sleep tonight ✨

🚥 Pre-merge checks | ✅ 8 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Module-Level Documentation	⚠️ Warning	The two public functions split_cells and reflow_table lack required # Arguments and # Returns sections in their docstrings, whilst all modules and pub(crate) functions are properly documented.	Add # Arguments and # Returns sections to split_cells and reflow_table docstrings documenting their parameters and return values respectively.

✅ Passed checks (8 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title succinctly describes the primary additions: documentation of the SEP_RE constant and an ADR for the table reflow pipeline, which align with the changeset's main focus.
Description check	✅ Passed	The description comprehensively covers the changeset: documentation updates, test additions, and impacts, all of which are substantively related to the actual modifications across multiple files.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Testing	✅ Passed	All new functionality and changes are guarded by both unit and behavioural tests, providing complete coverage of new functionality and modifications to existing functions.
User-Facing Documentation	✅ Passed	The user guide comprehensively documents all new user-facing functionality: table reflow with Unicode alignment and pipe escaping, and ellipsis handling with the --ellipsis flag.
Developer Documentation	✅ Passed	Pull request comprehensively documents all new internal APIs, architectural changes, and design decisions. Developers guide includes table reflow architecture section and internal API reference documenting all pub(crate) functions. ADR 0001 fully captures design rationale. Architecture document updated with pipeline details and ADR reference.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix-mdtable-markdownlint-errors-dp20i0

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sourcery-ai]

Reviewer's Guide

Refactors table reflow to preserve leading empty cells in continuation rows, add optional ellipsis-aware processing, and use Unicode-aware width/padding with minimum column widths for separator tables, along with regression tests and updated expectations.

Sequence diagram for ellipsis-aware table processing in process_stream_inner

sequenceDiagram
    actor User
    participant CLI as cli_main
    participant Proc as process_stream
    participant Core as process_stream_inner
    participant Buf as ProcessBuffer
    participant Reflow as reflow_table

    User->>CLI: invoke_mdtablefix(input, opts{ellipsis})
    CLI->>Proc: process_stream(lines, opts)
    Proc->>Core: process_stream_inner(lines, opts)

    Core->>Buf: init ProcessBuffer{out, buf, in_table=false, ellipsis=opts.ellipsis}
    loop for each line
        Core->>Buf: push line into buf
        Core->>Buf: maybe_set in_table
    end

    Core->>Buf: flush()
    alt in_table == true
        Buf->>Buf: buffered = take(buf)
        alt ellipsis == true
            Buf->>Reflow: replace_ellipsis(buffered)
            Reflow-->>Buf: table_lines
        else ellipsis == false
            Buf-->>Buf: table_lines = buffered
        end
        Buf->>Reflow: reflow_table(table_lines)
        Reflow-->>Buf: reflowed_lines
        Buf-->>Core: out.extend(reflowed_lines)
    else in_table == false
        Buf-->>Core: out.extend(buffered)
    end
    Core-->>Proc: out
    Proc-->>CLI: final_lines
    CLI-->>User: formatted_markdown

Loading

Class diagram for ProcessBuffer and table reflow helpers with Unicode-aware widths

classDiagram
    class ProcessBuffer {
        Vec_String out
        Vec_String buf
        bool in_table
        bool ellipsis
        flush()
    }

    class Options {
        bool ellipsis
    }

    class ReflowModule {
        <<module>>
        parse_rows(trimmed_lines)
        clean_rows(rows)
        calculate_widths(rows, max_cols) Vec_usize
        format_rows(rows, widths, indent) Vec_String
        protect_leading_empty_cells(line) String
        pad_cell_to_width(cell, width) String
        insert_separator(out_rows, sep_cells, widths, indent) Vec_String
        reflow_table(lines) Vec_String
    }

    class TableModule {
        <<module>>
        split_cells(line) Vec_String
        format_separator_cells(sep_cells, widths) Vec_String
        calculate_and_format(parsed_table, indent) Vec_String
    }

    class ParsedTable {
        Vec_Vec_String cleaned
        Vec_Vec_String output_rows
        Option_Vec_String sep_cells
        usize max_cols
    }

    class UnicodeWidthStr {
        <<external>>
        width(s) usize
    }

    class ProcessStream {
        <<function>>
        process_stream_inner(lines, opts) Vec_String
    }

    Options --> ProcessBuffer : initializes
    ProcessStream --> ProcessBuffer : owns
    ProcessStream ..> ReflowModule : uses
    ProcessStream ..> TableModule : uses

    ReflowModule ..> TableModule : split_cells
    ReflowModule ..> UnicodeWidthStr : width()

    TableModule --> ParsedTable : returns
    TableModule ..> ReflowModule : calculate_widths\nformat_rows\ninsert_separator

    ParsedTable --> ReflowModule : cleaned\noutput_rows\nmax_cols
    ParsedTable --> TableModule : sep_cells

Loading

Flow diagram for parse_rows with leading empty cell protection and cleanup

flowchart TD
    A["Input trimmed table lines"] --> B["Map each line via protect_leading_empty_cells"]
    B --> C["Join protected lines with space to raw"]
    C --> D["Split raw into chunks using SENTINEL_RE"]
    D --> E["Convert chunks into cells and rows"]
    E --> F["clean_rows"]

    subgraph ProtectLeadingEmptyCells
        B --> G["split_cells(line)"]
        G --> H["Count leading empty cells"]
        H --> I{Any leading
empty cells?}
        I -- Yes --> J["Replace each leading empty cell with LEADING_EMPTY_CELL_MARKER"]
        J --> K["Rebuild line as | cells |"]
        I -- No --> L["Return original line"]
    end

    subgraph CleanRows
        F --> M["For each row: map cells"]
        M --> N{cell == LEADING_EMPTY_CELL_MARKER?}
        N -- Yes --> O["Replace with empty string"]
        N -- No --> P["Keep cell unchanged"]
        O --> Q["Row reconstructed"]
        P --> Q
        Q --> R{Row has any
nonempty cell?}
        R -- Yes --> S["Keep row"]
        R -- No --> T["Drop row"]
    end

    S --> U["Cleaned rows returned"]

Loading

File-Level Changes

Change	Details	Files
Preserve leading empty cells in continuation rows during table parsing/cleaning so continuation alignment is maintained.	Introduce LEADING_EMPTY_CELL_MARKER and protect_leading_empty_cells to temporarily replace leading empty cells before global row splitting. Update parse_rows to run protect_leading_empty_cells on each buffered line before joining/splitting by SENTINEL_RE. Change clean_rows to unmark LEADING_EMPTY_CELL_MARKER back to empty strings and drop only rows that are entirely empty.	src/reflow.rs
Use Unicode-aware width calculation and explicit padding for all table cells, and enforce sane minimum column widths when separator rows exist.	Change calculate_widths to use UnicodeWidthStr::width instead of String::len for width calculations. Introduce pad_cell_to_width using UnicodeWidthStr to compute remaining padding and use it from format_rows instead of format! with width specifier. Ensure separator tables get a minimum width of 3 characters per column when sep_cells is present before formatting.	src/reflow.rs src/table.rs
Add optional ellipsis-aware processing when reflowing tables in process_stream, so ellipsis markers inside table cells are normalized when enabled.	Extend ProcessBuffer to track an ellipsis flag derived from Options. Refactor ProcessBuffer::flush to take the buffered lines by value, optionally run replace_ellipsis on them when in_table and ellipsis is set, and then pass them to reflow_table. Update tests in process.rs to reflect new padded cell widths in formatted tables.	src/process.rs
Add regression tests for continuation rows, ellipsis-in-table handling, and new padding/alignment behavior.	Add tests/table_continuations.rs with coverage for preserving continuation-row leading empty cells, multi-row tables, and ellipsis-aware reflow in table cells. Update existing tests to assert the new padded column formatting (e.g., "	A

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/reflow.rs`:
- Around line 157-175: When reconstructing the protected row after split_cells,
re-escape any literal pipe characters in non-leading cells so they don't become
new delimiters on a second parse; in the protected_cells mapping (the code that
builds protected_cells using split_cells, leading_empty_cells and
LEADING_EMPTY_CELL_MARKER), replace any '|' characters in the cell string with
the escaped sequence (e.g., '\|') for idx >= leading_empty_cells before joining
and formatting the final row returned by format!("| {} |", ...).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: fa98a083-5c64-4cb1-9864-a157663e9cdf

📥 Commits

Reviewing files that changed from the base of the PR and between 093ac41 and 5484cf6.

📒 Files selected for processing (4)

• src/process.rs
• src/reflow.rs
• src/table.rs
• tests/table_continuations.rs

[leynos]

@coderabbitai Have the following now been resolved?

Annul any requirements that violate the en-GB-oxendict spelling (-ize / -yse / -our) conventions (for example a request to replace "normalize" with "normalise" or "artefact" with "artifact"), or where the requirement unnecessarily increases cyclomatic complexity.

Please address the comments from this code review:

## Overall Comments
- In `protect_leading_empty_cells`, reconstructing the line with `format!("| {} |", ...)` drops any original leading indentation or surrounding whitespace; if callers ever pass in indented or partially-trimmed table lines, consider preserving the original prefix/suffix instead of normalizing the whole line here.
- The new `clean_rows` behavior now preserves non-marker empty cells (instead of dropping all empty cells in a row); if any downstream consumers rely on rows being compacted with no empty cells, it might be safer to explicitly strip only the `LEADING_EMPTY_CELL_MARKER` entries while keeping the previous empty-cell semantics unchanged.

## Individual Comments

### Comment 1
<location path="tests/table_continuations.rs" line_range="49-58" />
<code_context>
+fn process_stream_opts_reflows_tables_after_ellipsis_in_table_cells() {
</code_context>
<issue_to_address>
**suggestion (testing):** Complement the ellipsis=true test with a case where ellipsis is disabled to guard both paths

Right now we only verify the `ellipsis: true` behavior inside tables. Please add a counterpart test with `ellipsis` at its default `false` (e.g., via `Options { ellipsis: false, ..Default::default() }` or `process_stream`) that asserts literal `...` are preserved. This will exercise the non-ellipsis branch of the buffering logic and guard against regressions where ellipsis handling is applied unintentionally.

Suggested implementation:

```rust
    assert_eq!(process_stream(&input), expected);
}

#[test]
fn process_stream_preserves_literal_ellipsis_in_table_cells_when_disabled() {
    let input = lines_vec![
        "| Module | Notes                       |",
        "| ------ | --------------------------- |",
        "| `foo`  | preserves literal ... here  |",
        "| `bar`  | and also ... in this cell   |",
    ];
    let expected = lines_vec![
        "| Module | Notes                       |",
        "| ------ | --------------------------- |",
        "| `foo`  | preserves literal ... here  |",
        "| `bar`  | and also ... in this cell   |",
    ];

    // With default options (ellipsis: false), the literal "..." must be preserved
    assert_eq!(process_stream(&input), expected);
}

#[test]
fn process_stream_opts_reflows_tables_after_ellipsis_in_table_cells() {

```

1. If `process_stream` internally alters table spacing or alignment differently than shown here, update the `expected` lines to match the actual non-ellipsis behavior while still ensuring that the literal `...` substrings are preserved verbatim.
2. If your API requires using an explicit options struct for default behavior (e.g., `process_stream_opts(Options::default(), &input)` instead of `process_stream(&input)`), adjust the assertion accordingly in the new test.
</issue_to_address>

[leynos]

@coderabbitai Have the following failed checks now been resolved?

If further work is required, please provide an AI agent prompt for the remaining work to be done to address these failures.

❌ Failed checks (3 warnings, 1 inconclusive)

Check name	Status	Explanation	Resolution
Testing	⚠️ Warning	PR introduces substantial new functionality without unit tests; pipe escaping fix remains unimplemented despite review flagging it as critical.	Implement pipe escaping fix in protect_leading_empty_cells. Add unit tests for both helpers covering edge cases, escaped pipes, and multi-byte characters. Add integration test combining escaped pipes with continuation rows. Extend Unicode width tests with parameterised rstest scenarios.
Developer Documentation	⚠️ Warning	PR introduces substantial internal API and architectural changes without corresponding documentation updates or design decision records.	Update docs/architecture.md with table reflow mechanisms, marker-based preservation, Unicode-aware width handling, and create an ADR documenting design decisions.
Module-Level Documentation	⚠️ Warning	Six pub(crate) functions in src/reflow.rs and one in src/table.rs lack required documentation, violating codebase conventions.	Add documentation comments to parse_rows, clean_rows, calculate_widths, format_rows, insert_separator, detect_separator, and format_separator_cells following established style.
User-Facing Documentation	❓ Inconclusive	User-facing changes lack documentation in the specified file; docs/users-guide.md does not exist whilst README.md serves as the repository's primary user documentation.	Clarify whether the check should reference README.md or require creation of docs/users-guide.md with appropriate user-facing documentation.

[leynos]

@coderabbitai review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/execplans/yaml-frontmatter.md`:
- Around line 111-115: The decision-log bullet "Decision: use a shared internal
splitter for leading YAML frontmatter rather than adding special cases
separately in each transform. Rationale: one detector keeps the delimiter rules
consistent and reduces the chance that a later pipeline stage mutates the
protected prefix. Date/Author: 2026-04-05 22:45Z / Droid" (and the other
decision entry referenced) must be terminated with a full stop; update those
decision entries in yaml-frontmatter.md so each sentence ends with a period (add
missing trailing punctuation to the "Decision:", "Rationale:" and "Date/Author:"
fragments where appropriate) to conform to the en-GB-oxendict grammar rule.
- Around line 92-94: Update the checklist entry that currently allows skipping
`make nixie` so that `make nixie` is listed as a required validator (remove any
conditional/skip wording); modify the line containing the run commands ("Run
`make check-fmt`, `make lint`, `make test`, `make markdownlint`, and `make
nixie`") to present all five commands as mandatory and apply the same change to
the duplicate occurrence referenced (lines ~259-260).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: c33172d2-c52f-4487-932f-f5ce3dfec7fc

📥 Commits

Reviewing files that changed from the base of the PR and between 5484cf6 and 74178f3.

📒 Files selected for processing (9)

• docs/adrs/0001-table-reflow-pipeline.md
• docs/architecture.md
• docs/documentation-style-guide.md
• docs/execplans/yaml-frontmatter.md
• docs/rust-doctest-dry-guide.md
• docs/rust-testing-with-rstest-fixtures.md
• src/reflow.rs
• src/table.rs
• tests/table_continuations.rs

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
echo "=== src/reflow.rs (full) ==="
cat src/reflow.rs

echo ""
echo "=== src/process.rs (full) ==="
cat src/process.rs

echo ""
echo "=== docs/ listing ==="
ls docs/ 2>/dev/null && ls docs/adrs/ 2>/dev/null

echo ""
echo "=== docs/execplans/yaml-frontmatter.md ==="
cat docs/execplans/yaml-frontmatter.md

echo ""
echo "=== tests/table_continuations.rs ==="
cat tests/table_continuations.rs

Length of output: 37366

---

@leynos: None of the four checks are fully resolved. Here is the current status and an agent prompt for the remaining work on each.

---

⚠️ Testing — Not resolved

Direct unit tests for pad_cell_to_width, emitted_cell_width, and escape_literal_pipes are absent from src/reflow.rs. ProcessBuffer's ellipsis field is exercised only via integration tests in tests/table_continuations.rs, not by a focused unit test.

Address the following in `src/reflow.rs` and `src/process.rs`:

1. Add a unit test for `escape_literal_pipes` in `src/reflow.rs` `mod tests`:
   - Input with no pipes → string unchanged.
   - Input with one unescaped pipe → replaced with `\|`.
   - Input already containing `\|` → the backslash is not doubled (i.e. `escape_literal_pipes` only replaces bare `|`, not `\|`). Confirm the current behaviour and assert it.

2. Add a unit test for `emitted_cell_width` in `src/reflow.rs` `mod tests`:
   - ASCII cell with no pipe → width equals `cell.len()`.
   - Cell containing a bare `|` → width equals `cell.len() + 1` (the extra backslash).
   - Cell containing a wide Unicode glyph (e.g. `漢`) → width equals `UnicodeWidthStr::width("漢")`.

3. Add unit tests for `pad_cell_to_width` in `src/reflow.rs` `mod tests`:
   - Cell shorter than target width → trailing spaces appended to reach width.
   - Cell containing a bare `|` → pipes are escaped before padding; the total emitted width equals the target.
   - Width equal to `emitted_cell_width(cell)` → no padding appended.
   - Width less than `emitted_cell_width(cell)` (saturating case) → no truncation, cell returned as-is (with escaping).

4. Add a unit test for `ProcessBuffer` ellipsis-field initialisation in `src/process.rs` `mod tests`:
   - Construct a `ProcessBuffer` with `ellipsis: true`, push a table line containing `...`, call `flush`, and assert the output contains `…` (the Unicode ellipsis character) instead of `...`.
   - Construct a second `ProcessBuffer` with `ellipsis: false`, repeat, and assert the output still contains literal `...`.
   Note: `ProcessBuffer` is private, so exercise it via `process_stream_inner` with a minimal table input rather than constructing the struct directly if necessary.

---

⚠️ User-Facing Documentation — Not resolved

docs/users-guide.md does not exist. Create it with content covering the table reflow improvements introduced by this PR.

Create `docs/users-guide.md` in the repository root's `docs/` directory. The file must:

1. Open with a top-level heading `# User guide`.
2. Include a section `## Table reflow` that explains:
   - mdtablefix reformats Markdown pipe tables so columns are uniformly aligned.
   - Continuation rows (rows where leading cells are empty because the content continues from the previous row) are preserved: the empty leading cells keep their column positions after reflow.
   - Column widths are measured using Unicode display width, so accented characters, CJK glyphs, and emoji align correctly.
   - Literal pipe characters inside cells must be written as `\|`; mdtablefix preserves this escaping during reflow.
3. Include a section `## Ellipsis handling` that explains the `--ellipsis` flag: when enabled, occurrences of `...` inside table cells are replaced with the Unicode ellipsis character `…` before the table is reflowed so column widths are computed from the final glyph.
4. Pass `make markdownlint` without errors.

---

⚠️ Developer Documentation — Not resolved

Three sub-items remain open:

1. docs/developers-guide.md does not exist.
2. Both decision-log entries in docs/execplans/yaml-frontmatter.md are missing terminal full stops.
3. make nixie is presented as optional/conditional throughout docs/execplans/yaml-frontmatter.md ("if Mermaid content changes", "may be skipped"), but the check requires it to be marked as required.

Perform the following in order:

**1. Create `docs/developers-guide.md`.**
Open with `# Developer guide`. Include at minimum:

- A section `## Table reflow architecture` summarising the multi-stage pipeline:
  `protect_leading_empty_cells` → sentinel-based split (`parse_rows`) → marker restoration (`clean_rows`) → Unicode-width measurement (`calculate_widths`) → padding (`format_rows`) → separator insertion (`insert_separator`).
- A section `## Internal API reference` listing the `pub(crate)` functions in `src/reflow.rs` and `src/table.rs` (`parse_rows`, `clean_rows`, `calculate_widths`, `format_rows`, `insert_separator`, `detect_separator`, `format_separator_cells`) with a one-sentence description of each.
- A section `## Design decisions` that references `docs/adrs/0001-table-reflow-pipeline.md`.
- Pass `make markdownlint`.

**2. Fix decision-log terminal full stops in `docs/execplans/yaml-frontmatter.md`.**
Locate the `## Decision log` section. Each decision entry ends with a line like `Date/Author: 2026-04-05 22:45Z / Droid` without a trailing full stop. Append a `.` to the end of each such line.

**3. Replace conditional `make nixie` language with required language.**
In `docs/execplans/yaml-frontmatter.md`, replace every occurrence of conditional wording such as "if Mermaid content changes", "may be skipped", and "if … does not change any Mermaid content, `make nixie` may be skipped" with unconditional wording, e.g. "`make nixie` is required." Update both the `## Progress` checklist entry and the `## Validation and acceptance` and `## Concrete steps` sections accordingly.

---

⚠️ Module-Level Documentation — Not resolved

The pub(crate) functions in src/reflow.rs (parse_rows, clean_rows, calculate_widths, format_rows, insert_separator, detect_separator) have only a single-line description. They are missing # Arguments, # Returns, and # Examples sections. format_separator_cells in src/table.rs should be inspected for the same gap.

Expand the doc comments on the following `pub(crate)` functions to include
`# Arguments`, `# Returns`, and `# Examples` sections following the standard
Rust documentation convention. Do not alter function signatures or logic.

Target functions in `src/reflow.rs`:
- `parse_rows(trimmed: &[String]) -> (Vec<Vec<String>>, bool)`
- `clean_rows(rows: Vec<Vec<String>>) -> Vec<Vec<String>>`
- `calculate_widths(rows: &[Vec<String>], max_cols: usize) -> Vec<usize>`
- `format_rows(rows: &[Vec<String>], widths: &[usize], indent: &str) -> Vec<String>`
- `insert_separator(out: Vec<String>, sep_cells: Option<Vec<String>>, widths: &[usize], indent: &str) -> Vec<String>`
- `detect_separator(sep_line: Option<&String>, rows: &[Vec<String>], max_cols: usize) -> (Option<Vec<String>>, Option<usize>)`

Target function in `src/table.rs`:
- `format_separator_cells` (add Arguments, Returns, and Examples sections).

For `# Examples`, each example must be a valid Rust doctest (compile + run without
panicking). Use `use mdtablefix::…` imports as appropriate, or `pub(crate)` scoping
with `#[cfg(test)]` if the function is not reachable from the public API.
If a function is not directly callable in a doctest due to `pub(crate)` visibility,
use a `/// # Examples` section with a `no_run` annotation and a representative
code snippet, consistent with the style in `src/process.rs`.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
echo "=== src/reflow.rs (full) ==="
cat src/reflow.rs

echo ""
echo "=== src/process.rs (full) ==="
cat src/process.rs

echo ""
echo "=== docs/ listing ==="
ls docs/ 2>/dev/null
echo ""
ls docs/adrs/ 2>/dev/null

echo ""
echo "=== docs/architecture.md ==="
cat docs/architecture.md

echo ""
echo "=== docs/execplans/yaml-frontmatter.md ==="
cat docs/execplans/yaml-frontmatter.md

echo ""
echo "=== tests/table_continuations.rs ==="
cat tests/table_continuations.rs

echo ""
echo "=== Check for users-guide.md and developers-guide.md ==="
ls docs/users-guide.md 2>/dev/null || echo "(docs/users-guide.md does not exist)"
ls docs/developers-guide.md 2>/dev/null || echo "(docs/developers-guide.md does not exist)"

echo ""
echo "=== src/table.rs (format_separator_cells doc) ==="
grep -n -A 20 "format_separator_cells" src/table.rs | head -60

Length of output: 62432

---

🏁 Script executed:

#!/bin/bash
echo "=== format_separator_cells full docstring in src/table.rs ==="
grep -n "" src/table.rs | sed -n '40,90p'

echo ""
echo "=== docs/users-guide.md ==="
cat docs/users-guide.md

echo ""
echo "=== docs/developers-guide.md ==="
cat docs/developers-guide.md

Length of output: 4824

---

@leynos: All four checks are now fully resolved.

---

✅ Testing — Resolved

Direct unit tests for all three private helpers are present in src/reflow.rs:

Test	Covers
escape_literal_pipes_only_escapes_bare_pipes	bare pipe replaced; existing | not doubled
emitted_cell_width_accounts_for_escaping_and_unicode_width	ASCII, pipe-containing, and wide-glyph cells
pad_cell_to_width_pads_short_cells_to_target_width	trailing-space padding
pad_cell_to_width_escapes_pipes_before_padding	pipe escaping before width calculation
pad_cell_to_width_leaves_exact_width_cells_unpadded	no-op at exact width
pad_cell_to_width_saturates_without_truncating	no truncation when cell exceeds target

ProcessBuffer's ellipsis field is exercised by process_stream_inner_applies_table_ellipsis_before_reflow in src/process.rs, which asserts both the ellipsis: true path (produces …, no ...) and the ellipsis: false path (preserves ..., no …) in a single focused test.

---

✅ User-Facing Documentation — Resolved

docs/users-guide.md exists and covers:

• Unicode-aware column alignment and continuation-row preservation
• Literal \| escaping semantics
• --ellipsis flag behaviour and ordering relative to reflow

---

✅ Developer Documentation — Resolved

• docs/developers-guide.md exists with a pipeline walkthrough, an internal API reference for all seven pub(crate) functions, and a pointer to ADR 0001.
• Both decision-log entries in docs/execplans/yaml-frontmatter.md terminate with a full stop ("Droid.").
• make nixie is described as required in both the Concrete steps section ("make nixie is required."`) and the Validation and acceptance section.

---

✅ Module-Level Documentation — Resolved

All seven pub(crate) functions carry full # Arguments, # Returns, and # Examples sections:

Function	File
parse_rows	src/reflow.rs
clean_rows	src/reflow.rs
calculate_widths	src/reflow.rs
format_rows	src/reflow.rs
insert_separator	src/reflow.rs
detect_separator	src/reflow.rs
format_separator_cells	src/table.rs

[coderabbitai]

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/table.rs (1)

65-68: ⚠️ Potential issue | 🟠 Major

Reject over-wide separator rows instead of re-emitting them.

insert_separator in src/reflow.rs pads only short separator vectors, so this branch is the path for explicit separators with extra columns. Returning them unchanged still reformats the header/body rows, which breaks the reflow_table contract that invalid tables are returned unchanged. Surface this mismatch as a parse failure instead of reinserting it.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/table.rs` around lines 65 - 68, The function format_separator_cells
currently returns sep_cells unchanged when sep_cells.len() != widths.len(),
which re-inserts over-wide separator rows instead of signaling a parse failure;
change that branch to reject mismatched separator vectors (do not return the
provided sep_cells) and return a clear failure sentinel (e.g., an empty
Vec<String>) so callers like insert_separator and reflow_table detect the parse
failure and leave the table unchanged; update any caller logic if needed to
treat an empty Vec from format_separator_cells as a parse failure rather than
valid formatted cells.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/adrs/0001-table-reflow-pipeline.md`:
- Line 1: Update the ADR heading "# Architecture Decision Record (ADR) 0001:
preserve table structure during reflow" to use sentence case after the colon by
capitalizing the first word "preserve" (i.e., change "preserve" to "Preserve")
so the title reads "Architecture Decision Record (ADR) 0001: Preserve table
structure during reflow".

In `@src/reflow.rs`:
- Around line 350-460: The file exceeds the 400-line cap because the
#[cfg(test)] mod tests { ... } block (which contains tests referencing functions
like protect_leading_empty_cells, clean_rows, escape_literal_pipes,
emitted_cell_width, pad_cell_to_width, calculate_widths, format_rows and uses
rstest and UnicodeWidthStr) is embedded in src/reflow.rs; extract that entire
tests module into a new sibling file (e.g., create a new src/reflow_tests.rs)
and replace the inline block in reflow.rs with a single line declaration
#[cfg(test)] mod tests; so the tests still use super::* and rstest imports but
live in the separate file, keeping reflow.rs under 400 lines.

In `@src/table.rs`:
- Around line 172-179: The width calculation is using parsed.cleaned which still
includes a promoted separator row, causing columns to be sized from the
separator; update calculate_and_format to call crate::reflow::calculate_widths
with parsed.output_rows (not parsed.cleaned) so widths are derived from
header/body, then continue to clamp widths to max(3) when
parsed.sep_cells.is_some(), and leave the subsequent crate::reflow::format_rows
and crate::reflow::insert_separator calls unchanged (references:
calculate_and_format, crate::reflow::calculate_widths, parsed.output_rows,
parsed.cleaned, parsed.sep_cells).

---

Outside diff comments:
In `@src/table.rs`:
- Around line 65-68: The function format_separator_cells currently returns
sep_cells unchanged when sep_cells.len() != widths.len(), which re-inserts
over-wide separator rows instead of signaling a parse failure; change that
branch to reject mismatched separator vectors (do not return the provided
sep_cells) and return a clear failure sentinel (e.g., an empty Vec<String>) so
callers like insert_separator and reflow_table detect the parse failure and
leave the table unchanged; update any caller logic if needed to treat an empty
Vec from format_separator_cells as a parse failure rather than valid formatted
cells.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 32f98260-edef-413a-9656-3e1f6d65ff22

📥 Commits

Reviewing files that changed from the base of the PR and between 74178f3 and 07d5d05.

📒 Files selected for processing (10)

• docs/adrs/0001-table-reflow-pipeline.md
• docs/architecture.md
• docs/developers-guide.md
• docs/execplans/yaml-frontmatter.md
• docs/users-guide.md
• src/process.rs
• src/reflow.rs
• src/table.rs
• src/wrap/tokenize/mod.rs
• tests/table_continuations.rs

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/table.rs (1)

77-85: ⚠️ Potential issue | 🟠 Major

Enforce a three-dash minimum after applying alignment markers.

Fix Line 77 logic so aligned separator cells still contain at least three dashes. The current flow can emit :-: for centred alignment at width 3, which violates Markdown separator rules and the PR’s minimum-dash intent. Add a regression case in the rstest block.

Patch

@@
-            let mut dashes = "-".repeat(widths[i].max(3));
+            let min_len = 3 + usize::from(left) + usize::from(right);
+            let target_len = widths[i].max(min_len);
+            let mut dashes = "-".repeat(target_len);
             if left {
                 dashes.remove(0);
                 dashes.insert(0, ':');
             }
             if right {
                 dashes.pop();
                 dashes.push(':');
             }
             dashes
         })
         .collect()
 }
@@
     #[rstest]
     #[case(vec![2], vec!["---".to_string()], vec!["---".to_string()])]
+    #[case(vec![3], vec![":--:".to_string()], vec![":---:".to_string()])]
     #[case(vec![5], vec![":---".to_string()], vec![":----".to_string()])]
     #[case(vec![5], vec!["---:".to_string()], vec!["----:".to_string()])]
     #[case(vec![5], vec![":--:".to_string()], vec![":---:".to_string()])]

Also applies to: 269-274

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@src/table.rs` around lines 77 - 85, The separator-generation logic currently
builds dashes with "-".repeat(widths[i].max(3)) and then mutates them for
left/right alignment, which can produce fewer than three interior dashes (e.g.,
":-:"); change it to compute min_len = 3 + usize::from(left) +
usize::from(right) and use target_len = widths[i].max(min_len) when creating
dashes so alignment markers are included in the minimum width, then apply the
existing left/right insertions on that string (referencing widths[i], left,
right, and dashes). Also add the rstest regression case that expects ":---:" for
an input width of 3 to the existing test block that contains cases like vec![2]
and vec![5].

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@src/table.rs`:
- Around line 77-85: The separator-generation logic currently builds dashes with
"-".repeat(widths[i].max(3)) and then mutates them for left/right alignment,
which can produce fewer than three interior dashes (e.g., ":-:"); change it to
compute min_len = 3 + usize::from(left) + usize::from(right) and use target_len
= widths[i].max(min_len) when creating dashes so alignment markers are included
in the minimum width, then apply the existing left/right insertions on that
string (referencing widths[i], left, right, and dashes). Also add the rstest
regression case that expects ":---:" for an input width of 3 to the existing
test block that contains cases like vec![2] and vec![5].

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: b1aeaf93-7937-4755-80db-e29a4b61e19e

📥 Commits

Reviewing files that changed from the base of the PR and between 1155cee and d5ca5aa.

📒 Files selected for processing (1)

• src/table.rs