v4.6.0

ctrl-kd is the Python reference engine that reads WordStar 4–7 documents and
converts them to text, Markdown, HTML, RTF and PDF. This release is a large
accuracy pass on printed-page layout, measured page by page against real
WordStar 7 output, plus a set of Modern-view layout rules brought across from
the Soft Return app so that the app and the engines lay out the same lines.

Note before you upgrade: the RTF and HTML emitters do not yet carry the
new page-level behaviour described below. They are correct as text; the page
model is PDF and the on-screen views for now. That work is scheduled next.

Modern layout — the app's rules are now the engine's rules

The Soft Return app's "Modern" reading view had a number of typographic
refinements the engines did not share, so the same document could break lines
differently on screen and in an exported PDF. Those rules now live in the
engine:

• Verse and centred lines tighten up. Poetry, song lyrics and centred
  display lines now use the same reduced line height the app uses, with the
  matching bit of extra air above the first tightened line, instead of full
  body leading on every line.
• Definition lists hang properly. A definition-style row now hangs its
  label at the left margin with the body text and every continuation line
  aligned under the start of the body, across the full measure. Bulleted rows
  hang on the bullet's own cell.
• A one-sided left-margin change is no longer treated as a block indent.
  A .lm with no matching .rm used to push a paragraph a further inch and a
  half into the page in Modern; it now sits at the margin, the way the printed
  page shows it.
• The page baseline model matches the app. The first line of every Modern
  page now starts flush at the top margin rather than a full line below it, and
  a page fits while the line's box is inside the margin. On documents that mix
  type sizes this was worth a whole line per page.
• Line height is measured from the font's real design metrics — corrected
  natural line height for the body face, real vertical metrics from the font
  tables, and a blank line following a tightened line keeps body leading
  instead of inheriting the tightened value.
• Centring that a document implies rather than declares is now detected and
  centred.
• Rows that must not wrap no longer wrap — legend rows, chart rows and
  similar fixed-shape lines are kept on one line, and a colon inside such a row
  is no longer taken as a break opportunity.
• Line width is computed the same way the reference implementation computes
  it, so a line that fits in one engine fits in the other.
• Box and rule characters advance by the document's own fixed-pitch cell,
  not by the size of the face they happen to be drawn in — rules built from
  box-drawing characters now end flush at the margin instead of running off the
  page.
• Runs no font block covers keep the reader's face, and honour
  "proportional spacing off".
• Endnotes start a new page when the last page of the body already carries
  footnotes.
• Greek, mathematical and dingbat characters no longer fall back to a
  question mark in Modern PDF, and box characters inside a symbol-mapped run
  draw as real shapes.

The euro, and the peseta

Code 158 in the DOS character set is the peseta sign. Some WordStar 7 printer
definitions were patched in the late 1990s to print a euro there instead — a
well-known trick, documented in WordStar's own release notes. ctrl-kd now
models it the way WordStar did: if the document's last-used printer is one of
the three patched LaserJet definitions, code 158 prints as €; otherwise it
prints as the peseta ₧ — drawn as a real glyph where the base font set has no
slot for it. This replaces a blanket "always euro" fallback that turned genuine
peseta amounts in older documents into euros. The rule applies to body text and
to running heads and feet alike.

Printed-page layout, measured against real WordStar 7 printouts

• Newspaper columns (.co) are implemented in printed PDF layout: columns
  as wide as the block's right margin, filled down and then across, no
  balancing, and leaving a column region breaks the page — WordStar's own
  behaviour, verified against captures.
• Full justification (.oj on) is implemented, with the extra space split
  between gaps the way WordStar splits it.
• Even/odd page handling. .h1e/.h1o/.f1e/.f1o conditional headers
  and footers were never recognised at all (the pattern matched two of three
  letters); they now work, and the even/odd page offset .poe/.poo reaches
  the running head and foot, not just the body.
• Line numbering (.l#) is positional — a document can turn numbering on
  and off down the page and the gutter numbers follow — instead of one flat
  setting for the whole file.
• Landscape pages. A page length that does not match a named paper size no
  longer produces a square, clipped page; the document's own width is honoured.
• Leading. A paragraph style's own line height now outranks a stale
  document-wide setting; automatic leading inside a font block has a ceiling and
  never governs its own line; page capacity prices each line at its real height
  rather than the document default.
• Indents. A typed run of leading spaces indents by the natural width of a
  space when no paragraph-margin rate is in force, and by the paragraph margin
  when one is; a style-driven first-line indent no longer double-counts a typed
  one.
• Tabs. A literal tab byte round-trips and expands to the next column stop
  at layout time; a run of spaces before a tab shifts that stop correctly; a
  ruler line (.rr) resets left, paragraph and right margins, not only the tab
  stops, and consumes its own ruler-image line.
• Headers and footers honour an embedded paragraph-style selection for
  alignment, font and attributes, re-apply bullet substitution on the fast path,
  and track a page offset only when it is a real page-geometry change.
• Conditional text. .if/.el/.ei are evaluated for constant
  conditions, and a dot command immediately after a form feed is recognised.
• Page breaks. .cb/.cc/.cp are honoured in the footnote-aware
  paginator; a trailing .pa opens a new page only when real content follows;
  .fi prints nothing when its target cannot be resolved.
• Superscripts and subscripts in proportional text keep the surrounding
  block's pitch and change only the drawn size, matching WordStar.
• Pictures are included by default and sit at the right vertical offset,
  with the descent gap before the image WordStar leaves.

Reading printer output

The tooling that decodes captured printer streams now tracks symbol-set
selection, so curly quotes, the copyright sign and Symbol-font runs decode
correctly; it implements transparent print data; it unwraps a vertical-position
overflow WordStar itself emits; and page-size and paper-source commands eject a
page. Four more WordStar typeface identifiers are mapped.

Test corpus and manifests

• Test material that was never meant to ship no longer appears anywhere in the
  public test suite or tooling, and the public manifest no longer records stray
  files found in a corpus tree.
• Documents that cannot meaningfully be compared against a printout are now
  excluded by name and by class rather than silently failing: mail-merge data
  and format files, PostScript-driver documents, documents with form feeds
  turned off, duplicate copies of the same document, and one degenerate file
  that declares a zero-length page.
• Coverage of the printed-output comparison expanded to a much larger capture
  round, and the shared cross-engine answer key now covers the full corpus
  rather than a hand-picked subset.
• Tier-1 tests run on every push and pull request.

Other

• Plugin loading works again on Python 3.9.
• One environment variable per corpus, one shape each.

Known issues

• RTF and HTML exports do not yet carry the new page-layout rules. They are
  correct as text, but the page-level behaviour added in this release — default
  page numbers, even/odd page headers, page capacity, newspaper columns, tabbed
  tables and the Modern verse tightening — is not reflected in those two formats
  yet. PDF and the layout model are unaffected. Scheduled for the next release.
• The printed-output comparison set grew faster than it has been triaged.
  This release expanded the set of real WordStar 7 printouts we compare against
  from a few dozen pages to a few hundred. On a substantial number of the newly
  added documents the printed PDF still differs from the original printout, and
  those differences have not yet been sorted into "a substituted font is a
  different width" versus "we place something wrongly". They are measured and
  reported by name in the test output rather than hidden; classifying them is
  the next round of this work. The two engines agree with each other on
  every one of those documents.
• Documents that declare printer fonts we substitute — the LaserJet's
  scalable faces — can break lines slightly differently from the original
  printout, because the substituted face's character widths are not identical.
  Laying the page out with WordStar's own printer width tables is a planned
  improvement.
• On one page of one archived Sawyer test document, eight words sit about a
  third of a point to the left of where the original printer placed them. It is
  far below anything the eye resolves and is recorded rather than chased.