Skip to content

v2.4.0

Choose a tag to compare

@github-actions github-actions released this 20 Aug 16:02
· 21 commits to main since this release

A Fettler release, and a pass over the repository's own scripts.

fettle reads Excel and Word now, and searches inside them - along with the
PDFs it could already open. Not being able to read a workbook or a document
meant converting one by hand before an assistant was allowed to look at it,
which is a chore rather than a workflow.

And search can see inside all three. It could not see inside a PDF at all
before this, which made for the worst kind of gap: the file opened by name and
was invisible to every search, so a pattern that was plainly in the document
came back as a calm, confident zero.

Added

  • Spreadsheets are read. .xlsx and .xlsm come back a line per row,
    sheet by sheet, in the order the workbook lists its sheets rather than the
    order the files happen to sit in. Formulas are rendered as well as the
    values they last worked out to
    - a search for VLOOKUP is a question
    about how a sheet works and would find nothing at all in the cached
    results. Dates are rendered as dates, both calendars and the 1900 leap
    year Excel invented, because a workbook stores one as a day count and a
    workbook full of unsearchable numbers is not much better than a shut one.

  • Word documents are read. .docx and .docm come back as their
    paragraphs and tables, with headings kept as headings - and the headings are
    what a hit is cited under, since Word does not store where its pages break.
    Tables render a row at a time, footnotes and endnotes are included because
    they carry real content, and text removed by a tracked change is left out:
    nobody reading the document can see it, so nothing should find it.

  • Neither adds a dependency. Both formats are a zip full of XML, and the
    zip and XML readers were already here in the shared framework. The one
    package fettle bundles is still PdfPig, still only for PDF.

  • Documents are searched, not skipped. search renders a document to text
    and searches that, like any other file. Hits carry the page, sheet or
    heading they were found under, so a result can be turned back into somewhere
    to look. Nothing is unpacked to disk.

  • A reader mechanism, so the list can grow. A document type is one small
    class saying which extensions it claims and how to turn bytes into text plus
    landmarks; read, search and the kind reported for a file all follow from
    that one registration. It is a table compiled in, not a plugin loader:
    running a declared, named command already needs an explicit execute
    grant, so loading unnamed code in process would be a hole straight through
    the boundary the tool exists to draw.

  • write, edit and replace refuse a document by name. What comes back
    for one is a rendering rather than the file, with the sheets, the formulas
    and the structure left behind in the making of it. Writing that text back
    would not edit the document, it would replace it with a fraction of itself
    still wearing a name that says it is a document. The check sits in the one
    function every verb that writes text passes through, so there is no verb to
    forget; extract is deliberately not caught by it, since that writes a
    document's own bytes.

  • Search says when it did not look. An answer reports how many documents
    it left shut, in the text form and as documents_skipped in JSON. A file
    that was never opened is not a file with nothing in it, and the two used to
    be spelled the same way. --no-documents (no_documents over MCP) asks for
    the old behaviour deliberately; documents are searched by default. Anything
    over 32 MB is skipped and counted rather than rendered.

  • A declared task's command line can name a value the configuration
    supplies.
    A task still takes no arguments from its caller - that is the
    whole point of one - but writing {like-this} in its run line fills it
    from a replacements block, and an undeclared name is refused rather than
    passed through. The value can live in .fettler.local.json, which is not
    checked in, so the committed file declares the task every checkout shares
    and the overlay supplies what changes per person and per moment. A value
    containing spaces stays one argument: the command line is split before it
    is filled, never after.

  • scripts/shoddy-display.ps1 / .sh runs the three suites the gate is
    not allowed to run - seedscribbler, seedturtle, seedplotter. They open
    a real window, and GLFW needs a platform to create one on, which a hosted
    runner does not have. The script reads the same TST_EXCLUSIONS list the
    gate excludes them by, so a fourth windowed suite cannot be added there and
    forgotten here, and it refuses on a machine with no display rather than
    failing somewhere inside GLFW.

  • scripts/shoddy-commit.ps1 / .sh stages and commits. It refuses on
    main, which is the point of it rather than a courtesy, and -Force is the
    exception that has to be typed out.

  • scripts/verify-twins.js, now a preflight step, proves every script in
    the repository has its twin and that both offer the same verbs. The pair
    had drifted before - -y against -Yes - while the documentation went on
    calling them interchangeable.

  • tutorials/spiro/build.ps1, the twin that was missing. The tutorial is
    meant to be copied out of the tree, so it finds the mill by itself rather
    than borrowing the repository's shared helper.

Fixed

  • Non-ASCII text sent to fettle was corrupted on Windows. Console.In
    decodes stdin with the console's input code page, 437 or 850 on a default
    install, so the three UTF-8 bytes of an em dash were read as three separate
    characters and written back out as UTF-8 in their own right: six bytes where
    there should be three. The round trip was self-consistent, so the hash
    matched and nothing downstream could tell. It reached further than writing:
    a search for a non-ASCII pattern matched nothing and said so calmly.

  • A new .sh was committed without its executable bit. verify-permissions
    walks tracked files, so the window between writing a script and adding it was
    invisible to it - which is how v1.8.0 shipped four scripts that way and one of
    them took the Release workflow down with Permission denied. shoddy-commit
    now stages an unseen shell script with git add --chmod=+x.

  • shoddy-release.ps1 tested the wrong thing for "am I at the repo root".
    It compared paths that Git for Windows spells differently; it now asks git
    directly.

  • shoddy-release.ps1 wrote Directory.Build.props through Windows
    PowerShell's -Encoding utf8, which emits a BOM.
    The version bump now
    writes UTF-8 without one.

  • The shell twins accepted only -y where the PowerShell scripts document
    -Yes,
    and only as the first argument. Both twins now take either
    spelling, anywhere in the line.

  • sed -i in shoddy-release.sh meant something different on BSD sed
    from GNU sed. The version bump no longer uses it.

Changed

  • The gate runs its PowerShell steps under PowerShell 7. Windows
    PowerShell 5.1 turns anything a native program writes to stderr into a
    terminating error the moment a caller captures both streams, and writes a
    BOM where it is told not to. pwsh is probed once and used everywhere, with
    a fall back to powershell where it is absent.

Documentation

  • WORKFLOW.md and RELEASING.md pointed at shoddy-release.ps1 as the way
    to release.
    It is the primitive underneath, not the front door: it builds,
    tests, tags and pushes, while the doc gates, three of the four test lanes and
    the green-receipt check are all skipped. Both pages now name
    scripts/shoddy.ps1 release X.Y.Z, with a table of exactly what the wrapper
    adds.

  • The display-suite gap is written down in WORKFLOW.md beside the command
    that covers it, rather than being an exclusion list nobody reads.