Skip to content

CMMNS 3.10.0

Latest

Choose a tag to compare

@HarperZ9 HarperZ9 released this 14 Nov 07:28
· 1 commit to main since this release
16b4ae0

Any page can be saved from the right-click menu, a YouTube video can be saved in full with its
transcript and comments, Content Credentials get a validation report, and a Python analysis layer
ships beside the extension. No store build saves YouTube video files.

Added

  • Save page, on any site (project-docs/SIDECAR-FORMAT.md, "A page save"). A right-click entry, and
    what Download, the hover control's shortcut and "Download media here" now do when no picture or video is
    under the pointer. On a page with no site reader it saves the text of the selection, of the block under
    the pointer or of the article, the title, author and date the page states, the canonical address, the
    page's Open Graph, JSON-LD and citation tags, and each media element's address, as a note and a .json
    record with adapter: "article" or "generic". Files the page offers as a plain address are saved
    beside it, 24 at most; a stream, a player's in-memory source and protected media are listed with
    not saved (reason) and never fetched. With source protection on, no file is asked for.

  • Every note names the reader that made it (adapter), and says media_file: "saved" or
    "not saved (reason)" where that is recorded. A YouTube reference lists the caption tracks the page's
    own data names.

  • Save video in full (project-docs/SIDECAR-FORMAT.md, "A video saved in full"). On YouTube the hover
    control saves the reference with the transcript as timestamped lines and as plain paragraphs, the
    chapters and metadata (<stem>.video.json, cmmns.video/1), the comment tree with replies and like
    counts, and <stem>.SOURCE.txt and <stem>.SHA256SUMS. The comment tree now joins what the watch page
    drew with the comments in the page's own data, filed per video so a save never carries the previous
    video's comments. The media file is asked on each save; builds with the command line hand-off copy the
    command that saves it beside the bundle. Store builds still do not save YouTube video files.

  • cmmns media fetch <url> --full writes the same bundle with yt-dlp: media (or --no-video), caption
    tracks and both transcripts, chapters and metadata, up to --comments N comments with replies (600 by
    default, sorted by top), the note, SOURCE.txt and SHA256SUMS, in a folder of its own. --name <stem>
    saves a plain fetch beside an extension save. The media file is a local research copy, never uploaded.

  • cmmns media fetch --full without ffmpeg keeps the best video and the best audio as two files, both
    hashed, where it asked yt-dlp to merge them before. YouTube no longer offers a file that holds both. The
    bundle folder is named after the video's title, not the first words of its description.

  • analysis/signalchain: one measurement API for a signal and a transform (measure(x, medium),
    through(x, y)): spectrum, entropy, signal-to-noise, envelope, resonance, decay, gain and loss, plus
    non-stationary measures. Each measure passed a 40-seed recovery gate against a seed-shuffled null, and
    four false-success controls must fail it. numpy only.

Fixed

  • Download on a text post or a text page saved nothing. "Download media here" and the keyboard
    shortcut answered "No picture, GIF or video was found under the pointer" on a text-only post. They now
    save the post as a note, or the page.
  • A note on a site without a reader was refused ("Select some text, or point at a post") unless text
    was selected. It now reads the article or main block.
  • YouTube references after the site's own navigation. The description was the page's cut-off text
    ending in its "more" control, not marked as incomplete, and the date was missing or was the date of the
    first video the tab loaded. The reference now reads the player's own record of the current video for
    the whole description, the date and the caption list, uses the page's date tags only for the video they
    were written for, and marks a cut description text_complete: false.
  • A text note now writes its .json record when that setting is on.

Added (Content Credentials)

  • Validation report and badge (src/core/c2pa-report.js, project-docs/C2PA-REPORT.md, schema
    schema/c2pa-report.v1.json). A checked file now gets one field per check: signature, the chain
    against trust anchors you supply, revocation at signing (stapled OCSP) and now (CRLs or OCSP
    responses you supply, else "unknown"), the manifest fields present or absent with the AI-origin
    and generator fields named, the hard and soft binding, and the C2PA 2.5 weaknesses that apply,
    each with its clause. The window's first row is the badge: the strongest statement the checks
    support, such as "certificate revoked since 2026-06-01" or "AI-origin field absent, which the spec
    does not require". The API adds credentialReport (API 2.1.0). Nothing new is sent anywhere.
    User page: docs/guide/CONTENT-CREDENTIALS.md.

Fixed

  • Addresses without share tracking (src/core/url-clean.js, project-docs/SIDECAR-FORMAT.md,
    "Addresses without share tracking"). A sidecar's post and page addresses, its link list, its link
    cards and the quoted or parent post's address lose share and campaign parameters: utm_* and click
    ids on any host, and the short names only where a host uses them for tracking (YouTube si, X s and
    t, Instagram igsh). These parameters can name the person who shared the link. A redd.it media
    signature (s=), a YouTube time (t=), the media address and the post's own text stay as they were.
    Video bundles store the watch address and SOURCE.txt line the same way.
  • YouTube thumbnails saved off a watch page keep their video. A thumbnail saved from the home
    page, a channel or a search page recorded only page: "https://www.youtube.com/"; the sidecar now
    takes post from the thumbnail's address (i.ytimg.com/vi/<id>/).
  • What a thread file claims (project-docs/SIDECAR-FORMAT.md, "The thread file"; format
    cmmns.capture-thread/2). A thread with no comments no longer reads complete: true: it has its own
    state, "empty", and no longer tells you to open the post's page for comments that do not exist. X's
    reply count covers only replies directly under the post, so "Captured 5 of 1" now reads "Captured 1 of
    1 direct reply, and 4 more deeper in the thread", in separate fields. The same mix-up marked some short
    captures complete and could stop the opening of more early; both now follow the direct count. The post
    author's own continuation is labelled and counted apart from comments. readCaptureThread reads files
    in the first format.

Added

  • Python analysis layer (analysis/). analysis/formulas (exact binomial bound with family
    correction, two-lines guard, shareability) and analysis/latent (hidden-variable instrument with
    planted and pure-noise controls) move in from the author's workstation tools. A CC0 statistics
    vector file, analysis/vectors/stats_vectors.json, is the one contract for Holm, Bonferroni,
    Benjamini-Hochberg, the binomial upper tail and the Wilson interval; Python and
    src/core/discourse/stats.js are both tested against it. node scripts/ci.mjs analysis runs it,
    and CI has a new analysis job. captureCanonical is now tested against the canonical-record/1
    vectors from Flywheel (RFC 8785); it agrees on every valid case.
  • Whole saves (project-docs/SIDECAR-FORMAT.md, "The thread file"). A saved post brings its whole
    body and its parts. Its comments come with it as <name>.thread.json, a tree with each comment's
    handle, words, score, time, depth and link, and a completeness block: the site's count, how many
    were captured, and whether the cap stopped the opening. On the post's own page CMMNS opens "more
    replies" one at a time, 1.2 to 3 s apart, up to a cap you set (200 by default), with Stop on the
    notice; source protection holds that off. Reddit reads the rendered comment tree (never the
    .json view); X reads the replies X sent the page and writes t.co links as the addresses they
    stand for, with the quoted post's pictures and alt text; YouTube references carry the comments the
    page drew. Instagram saves read the post data Instagram's own app received: every carousel item
    at full size, videos, alt text, caption, time, counts, location and comments. A likes-page
    thumbnail, whose page holds only a 240 px picture, opens its post in a background tab and is saved
    from there; if that fails, the thumbnail is saved with a "Not the whole post" note saying why, and
    the profile's page description is no longer taken for the post's words. Sidecars list every file
    of a post with its own alt text under ## Media. Instagram saves now land in <save folder>/instagram/,
    beside x and reddit.
  • Save all likes on X (project-docs/SAVE-ALL-LIKES.md). On your likes list a bar collects every
    liked post from the responses X already sent the page and lists each one once in a manifest
    (Downloads/CMMNS/library/x-likes/). cmmns likes fetch --watch fetches the files from X's media
    hosts over reused HTTP/2 connections, 8 at once, with retries, 429 handling and resumable ranges, into
    CMMNS/x/ beside your earlier saves, each post with its note (from: "x-likes"). Save in the
    browser does the same through the page. Posts already saved are skipped before any request, by post
    id, and a file whose SHA-256 matches one already saved is not kept twice. The run survives a reload
    or a restart and offers Resume; Pause and Stop keep the record. Import like.js from your X archive as
    the checklist; cmmns likes missing lists what is not saved yet. Auto-scroll, paced with jitter
    (careful by default) and stopping on X's first error or rate-limit message, ships only in the
    unpacked and self-distributed builds. The Boards page gains From my likes only and a posted-date range.
    A post whose video X listed only as a stream counts as saved without video, on the bar, in the run's
    record and in cmmns likes fetch and missing; its note names the stream, and on the command line it stops counting once the video is saved from the post's page.
  • Faster saves. A picture is asked for once, where it was asked for twice (one request, read on when
    the words go inside the file). A library saves four posts at once, keeps index.jsonl in the page's
    order, and writes its record every 10 posts. The saved list is written once for each batch of files.
    Measured in Chromium against a local host with a 60 ms delay: one picture 352 to 275 ms, a four-file
    gallery 481 to 370 ms, a 20-post library 5.1 to 2.1 s, 200 posts 41 s to 18 s.
  • Engagement over time for saved posts (project-docs/ENGAGEMENT-SERIES.md). Each post saved on X,
    Reddit or YouTube is tracked by id, and its counts are written down again whenever the page's own
    data carries it, at most once per post per 6 hours, with the post's age. A daily alarm works out
    which readings are due at 1, 3, 7, 14, 30 and 90 days and records the missed ones; with the
    reminder on, the toolbar badge shows the count and the collections page lists the posts, with Next
    opening the first in the same tab. An optional seen-post log (off by default, 90 days, no post
    text) keeps every post the page loaded. "Export engagement series" writes JSONL and a manifest with
    each file's SHA-256. No request is made for any of it; the new alarms permission only counts.
    Source protection forces the seen-post log off and writes no entry while on; the options page shows
    the switch locked with the reason, and the protection page offers a two-step delete of entries
    kept from before.
  • Reply timeline in the thread report (src/core/reply-timeline.js, src/ext/thread-tools.js): a
    strip of bars showing when the loaded replies arrived, with the stretches the timing-burst test
    marked drawn in the sign colour and underlined. Never finer than the times behind it (times good
    to the hour get hour bars). The strip carries one sentence as its accessible name, printed under
    it too, and names no account. Nothing moves.
  • Save thread and Save as JSONL in the thread report (src/core/thread-save.js): the replies
    the page has loaded, each with its author, time, text, badges (with what each does not prove) and
    its same-text group or burst, as one Markdown note or as JSON Lines (cmmns.thread/1, the thread
    first, then the post, then one line per reply). Written through the same save path as other
    saves; nothing is sent. No extremity score or rhetoric marker is kept per reply.
  • Library capture (project-docs/LIBRARY-CAPTURE.md). A Save page as library bar on X profile,
    list, search and hashtag pages and Reddit user and subreddit pages. Every post the page loads is saved
    once, by post id, with its media and its daylight.sidecar/1 note, into
    CMMNS/library/<site>-<name>/, with index.jsonl (each post's files and their SHA-256) and
    LIBRARY.md (source pages, sessions, counts, what was skipped and why, and the evidence-only scope
    note). Each post keeps every field the page delivers in the clear: full text, media at the largest
    size served, quoted and parent posts, link targets, engagement, and from X's post data the language,
    posting app, earlier versions, Community Note and paid-partnership mark (fields in the index; one-click
    saves now also list each link's target). Reopening the page continues the same library.
    An account's library keeps its own posts, its reposts (kind: "repost", with the original's author
    and id) and, on a Reddit user page, its comments (kind: "comment", with the parent post and
    subreddit), so the account's own posts can still be filtered; context posts and adverts are counted.
  • Scroll to the end for a library, with a cap (500 by default), at a reading pace with jitter; only
    in the unpacked and self-distributed builds, never in the store packages (tests/store-build.test.js).
  • cmmns case add-library <folder> [--copy-media]: every post of a library becomes a case source,
    each file's hash checked against the index, so timeline, claim, trace and read run over it.
  • Catch rate and tendency in the window. Under each measured sign's precision, "Catch rate" says
    how much of the labelled test text it found ("finds about 1 in 47 posts calling for violence in
    testing") and "Tendency" how readily it fires ("rarely fires; when it does, it is usually right").
    A "Sensitivity" row gives d' with its 95% interval ("d' 1.00 (0.82 to 1.20); 0 is chance, 2 is
    good"), or "not measured" under the 20-hit rule.
  • One test set per sign in the window. Precision, catch rate, d' and tendency now all come from
    one test set, named in the "Measured" line: the sign's own label on Measuring Hate Speech where it
    has one (dehumanising, violence, stereotype), else Measuring Hate Speech's general hostility label
    (slurs). The window's precision for dehumanising language moves from 93% (pooled, against general
    hostility) to 60% (against its own label), call for violence from 88% to 64%, group stereotype from
    84% to 73% and slurs from 91% to 95%. The pooled figures stay on the measurement page, labelled.
  • Signal detection measures in the evaluation (scripts/eval/sdt.mjs, node scripts/eval-sdt.mjs).
    Hit rate, false-alarm rate, sensitivity d' and bias c with 95% bootstrap intervals, per sign and
    corpus, in eval-results.json, EVALUATION.md and the measurement page. Every sign is very
    conservative (c mostly 2.5 to 3.5); the best d' is about 1.0 to 1.3. Rows scored against general
    hostility are marked, and the French corpus reads "not measured". Deterministic and
    dependency-free; the committed figures match the 2026-10-28 study exactly.

Fixed

  • CMMNS' window on comments no longer goes missing on pages that never stop changing. Every page
    change pushed the refresh pass back, so on a page changing more often than every 250 ms new
    comments were never wired. A pass now runs at most 1 s after the first waiting change (or after
    the stretched wait, when slow passes stretched it). Live on a YouTube watch page changing every
    150 ms, the old scheduler ran 2 passes in a minute and wired 0 of 60 and 20 of 62 comments; the new
    one ran a pass about every 1.05 s and wired all but one (project-docs/evidence/REFRESH-STARVATION.md,
    tests/guard-timing.test.js).
  • The hover Download control no longer hides behind YouTube's fullscreen player. The player entered
    the browser's top layer after the control, so the control was drawn underneath (seen live: the
    video was the topmost thing at the control's centre). The control is lifted back on top each time
    it shows and when the page enters fullscreen; a control whose host the page removed is redrawn in
    a new host instead of in the removed one (tests/hover-top-layer.test.js fails on the old code).
  • A YouTube reference no longer saves chapters from the video watched before. YouTube changes videos
    without reloading the page and can leave the old chapter list in it; each chapter links to its own
    video, so only this video's chapters are kept (tests/yt-chapters.test.js fails on the old code).
  • One-click saves pass their folder (site, study or library) with each save instead of keeping it in
    shared page state, so a save that waits on the network and another save started meanwhile can no
    longer swap folders (tests/library.browser.test.js fails on the old code).
  • Saved files no longer share a name across posts. Two posts by one account on one day with the same
    first words, or with no text at all (post), asked for the same file name, so the browser saved
    name (1).jpg, name (2).jpg and a note could list another post's picture. Each one-click save,
    study save and text note is now named <handle>_<date>_<words>[-n]_<post id>.<ext>: the id from the
    post's address (X, Mastodon, Reddit, Bluesky, YouTube, Instagram, TikTok), else an 8-character hash
    of the post, media or page address. A post's note is named the same way and lists the exact names
    its files were saved under. Saving the same post again still gets (1) names, and its note lists
    those. Library captures keep the names they had. The daylight.sidecar/1 format is unchanged: no new
    field, and older folders read as before (tests/sidecar-collision.test.js fails on the old code).

Documentation

  • Leftover "CMMNS Commons" naming in the privacy policy, How it works and the transparency page now
    reads "CMMNS". The data protection draft's network list is dated after release 3.9.1, where it said
    3.8.0. project-docs/SIDECAR-FORMAT.md and project-docs/MEDIA-LIBRARY.md describe the new file
    names.
  • Drafts of a legal and responsible-use pack (acceptable use, responsible use, legal notice, requests
    from people named or shown, data protection, platform capture, creators and IP) and a checklist for
    the lawyer who reviews them (project-docs/LEGAL-REVIEW-CHECKLIST.md). Each is marked as a draft
    awaiting that review and is not in force. The privacy policy links them under "More detail" with the
    same label and is otherwise unchanged.