Skip to content

Releases: mahope/clean-copy-cli

v1.5.0 — CSV mode

Choose a tag to compare

@mahope mahope released this 25 Aug 09:54

New: -v / --csv flag. HTML tables become comma-separated CSV rows (RFC 4180 quoting), ready for Excel/Google Sheets. No tables? Falls back to cleaned plain text.

v1.4.6

Choose a tag to compare

@mahope mahope released this 24 Aug 18:22

Fixed: intermittent empty extraction on some platforms (e.g. Wix) — the CLI now retries when a server returns an empty page body.

Tests: replaced the joshwcomeau.com live test (site now 403s non-browser clients) with an astro.build test. 40 passed, 0 failed.

v1.4.5 - Ghost/404media extraction fix

Choose a tag to compare

@mahope mahope released this 24 Aug 16:55

Fixes URL extraction on Ghost-based news sites (404media.co):

  • Depth-matched boilerplate/ad container removal replaces the fragile tempered-dot regex (a dangling aside close tag used to swallow main and leak ad chrome into the output).
  • New ad-container strip (.ad, .ad-leaderboard, ...) removes ad blocks embedded inside the article container.
  • Token-exact class matching: has-sidebar on the whole article no longer causes the article to be removed.
  • Live regression tests added for ghost.org and 404media.co - 32 passing.

v1.4.4

Choose a tag to compare

@mahope mahope released this 24 Aug 16:30

Iterative content-block refinement: after scoring candidate containers, the extractor now repeatedly descends into the best nested block (while it keeps >=60% of the parent score). This peels page chrome that large CMS wrappers (Squarespace, Wix, Shopify) carry around the article. Wider boilerplate class exclusions (hero, breadcrumbs, lead magnets, subscribe forms, bylines, menus). 28 tests pass including live Wikipedia/Shopify/Wix/Squarespace URL checks.

v1.4.3

Choose a tag to compare

@mahope mahope released this 24 Aug 16:22

Microdata (itemprop=articleBody) extraction + article score boost: Shopify/Wix blogs now extract clean prose instead of promo banners. Live tests extended to Shopify and Wix — 26 passed.

v1.4.2 — Real-world URL extraction

Choose a tag to compare

@mahope mahope released this 24 Aug 15:55

clean-copy CLI v1.4.2 improves URL → Markdown extraction against real websites:

Changes

  • Strips ARIA/role boilerplate (role=navigation/banner/contentinfo/complementary)
  • Removes MediaWiki maintenance boxes (ambox/ombox), navboxes, edit links and print-only cruft
  • Link-density scoring: navigation-heavy wrappers no longer outrank prose blocks
  • <section> added to candidate containers; hidden elements skipped
  • Prefers JSON-LD articleBody when present (news sites)
  • New live test: Wikipedia extraction verified in npm test

Verify

node test.js — 22 passed, 0 failed.

v1.4.1 — Table alignment

Choose a tag to compare

@mahope mahope released this 24 Aug 15:15
  • Tables now preserve column alignment from text-align styles and legacy align attributes: :--- left, :---: center, ---: right
  • colspan cells pad correctly and can carry alignment
  • includes language-tagged fenced code blocks from v1.4.0

v1.3.10

Choose a tag to compare

@mahope mahope released this 24 Aug 13:28

Strip invisible U+2060-U+2064 characters (Guardian word joiners); unwrap image-only wrapper links and dead #fragment anchors; core synced across all surfaces.

v1.3.9

Choose a tag to compare

@mahope mahope released this 24 Aug 13:15

Fenced-code entity fix (MDN) + depth-matched container extraction (Wikipedia). Tests 18/18.

v1.3.8

Choose a tag to compare

@mahope mahope released this 24 Aug 12:56

stripTagsSafe: tags with > inside quoted attribute values (e.g. Wikipedia data-mw JSON) no longer leak raw markup into output. Regression test included.