Skip to content

Releases: js2264/chromosomix

v0.20.0

Choose a tag to compare

@github-actions github-actions released this 05 Nov 07:17

What's Changed

  • Notebooks: ask again when a download's server fails (§18.10) by @js2264 in #154
  • Documentation site (§32), its pictures, and the fixes writing it found by @js2264 in #157
  • Docs: live examples, the app itself in place of its picture (§32.4) by @js2264 in #158
  • Docs: a live view takes the page's width, the contents list stepping aside by @js2264 in #159
  • Timings: a deep human map, start to finish (§18.10) by @js2264 in #161
  • Label scaled metaplot axes by feature position, not column by @js2264 in #155
  • Add cell-of-its-own rectangle to Contact maps card by @js2264 in #156
  • Docs in PR previews; docs look like the app and a third shorter; embeds fill and resize their frame by @js2264 in #160
  • Move viewer timings to a separate notebook by @js2264 in #162

Full Changelog: v0.19.0...v0.20.0

v0.19.0

Choose a tag to compare

@github-actions github-actions released this 01 Nov 20:03

What's Changed

  • Add example sessions to landing page by @js2264 in #144
  • Measure analyses in the app as well as on the command line by @js2264 in #145
  • CI: hold the subsample and every normalisation to cooler and cooltools, run on the day by @js2264 in #147
  • Signal by group: one value per entry by default by @js2264 in #148
  • Compartments: divide by cooltools' own expectation by @js2264 in #146
  • Add TAD annotation (Juicer's Arrowhead, ported) and the map report by @js2264 in #149
  • Describe the site to AI agents in full: llms.txt and five reference pages (§11, §23) by @js2264 in #150
  • Laptop notebook: run each analysis in the app in Chromium, Firefox and WebKit (§18.10) by @js2264 in #152
  • Remove telemetry entirely (§1, §11, §13, §18.10; §31 removed) by @js2264 in #153
  • Service worker: answer only the app's own address, so llms.txt and a 404 open as themselves (§11) by @js2264 in #151

Full Changelog: v0.18.0...v0.19.0

v0.18.0

Choose a tag to compare

@github-actions github-actions released this 30 Oct 09:03

What's Changed

  • Domain pileup over expected: rescaled local window by @js2264 in #141
  • Speed up base composition and P(s) calculation by 8× and 2.5× by @js2264 in #142
  • Base composition on indexed FASTA; TSV/bedGraph in pieces by @js2264 in #143

Full Changelog: v0.17.0...v0.18.0

v0.17.0

Choose a tag to compare

@github-actions github-actions released this 26 Oct 18:23

What's Changed

  • Validate cis and trans, the saddle, FRiP and signal by group; signal by group reads exactly (§18.8, §18.10) by @js2264 in #125
  • Add parity notebook comparing app and CLI outputs by @js2264 in #126
  • Add laptop.qmd: benchmark 23 analyses on your own machine by @js2264 in #127
  • Peak calling 9.6× faster genome-wide in under a quarter of the memory, results bit-identical by @js2264 in #128
  • Test .hic reader against Juicer Tools demo file with hic-straw by @js2264 in #129
  • Notebooks: references get a process a core, renders log the memory, timings weigh every process by @js2264 in #130
  • pastis-NB fit runs on multiple threads with reproducible results by @js2264 in #131
  • Add Notebooks tab to About dialog with task reference citations by @js2264 in #132
  • Add anonymous telemetry and allow-access button for remembered files by @js2264 in #135
  • Telemetry: honour Global Privacy Control, not Do Not Track; shorter About blurb by @js2264 in #136
  • The lower triangle in a reading of its own (§5.10) by @js2264 in #137
  • Add Fig. 1a and Fig. 2 to the paper's store, and § 24 to the validation notebook by @js2264 in #138
  • Pairs stats: pairtools stats on import, the Pairs statistics card, and Pairs QC with restriction fragments (§3.7, §18.8) by @js2264 in #134
  • Paper store: GM12878's second CTCF track is UW's (ENCFF644EEX) by @js2264 in #139
  • Jupyter and R widgets: the viewer in a notebook cell, R Markdown, Quarto and Shiny (§21.5) by @js2264 in #133
  • Validation § 24: name its peaks' columns itself, not with § 5's key by @js2264 in #140

Full Changelog: v0.16.0...v0.17.0

v0.16.0

Choose a tag to compare

@github-actions github-actions released this 20 Oct 22:24

What's Changed

  • The page says when a picture is finished, and at what resolution (§12, §21.4) by @js2264 in #115
  • Publish to npm by trusted publishing, without a token by @js2264 in #116
  • A file let go on any cell of the deck lands there; store rows drag like Files rows; a + before back opens Load files (§5.5, §16.9, §24.2) by @js2264 in #117
  • A 3D model leaves as a PDB for PyMOL, and a connected folder offers every file the app opens (§20.5, §25.2, §25.3, §25.7) by @js2264 in #119
  • A 1D task is one Run, deck or not, and the star is nothing to it (§18.1, §18.4) by @js2264 in #118
  • Record every number validation figures draw in validation.txt by @js2264 in #120
  • Loop calling, a 2D task just before Pattern detection (§18.8, §18.9) by @js2264 in #121
  • Switch to GPL-3.0-or-later and add licence attribution in About by @js2264 in #124
  • Tutorial on K562 map with sequence copy and manual marks by @js2264 in #123

Full Changelog: v0.15.0...v0.16.0

v0.15.0

Choose a tag to compare

@github-actions github-actions released this 18 Oct 06:42
f8da5d8

What's Changed

  • Unify interface grammar across all reader-facing surfaces by @js2264 in #69
  • pastis-NB, four analysis cards, and a deck cell per model (§18.8, §25.3, §25.5, §25.6) by @js2264 in #68
  • Read a bed3d whose solver left bins unplaced as the structure it is (§25.2) by @js2264 in #70
  • Metagenome sequences card and unified level control by @js2264 in #71
  • The Map card follows the star, and no map or structure need be starred (§16.5, §25.3) by @js2264 in #73
  • Add bacterial chromosome analysis: origins, arms, and group signal by @js2264 in #72
  • Move synthetic tracks to Tracks card, add points style by @js2264 in #74
  • Inter-arm alignment's table gives the enrichment, as a fold by @js2264 in #76
  • Add peak calling task with yapc and IDR implementations by @js2264 in #75
  • Open every map of a multi-file drop in its cell (§16.11) by @js2264 in #77
  • Overlay map selection and validation notebook by @js2264 in #79
  • Draw maps to a chosen number of contacts (§6.5) by @js2264 in #78
  • Draw the inter-arm decay alone: inter-arm in the score picker on a circle (§7.5, §8.15) by @js2264 in #80
  • P(s)'s four orientations: from the import, from a dropped .pairs, and unbroken (§18.8) by @js2264 in #81
  • Reference editing: dev feature with stage, method, and exports by @js2264 in #82
  • Chromosome ideograms and RNA-seq splice junctions as 1D tracks (§8.22, §8.23) by @js2264 in #83
  • Open a large .hic without waiting on its contact count (§3.6, §6.5, §12) by @js2264 in #84
  • Draw a chromosome clicked during the contact count in seconds (§6.5, §8.15, §9, §12) by @js2264 in #85
  • Vector PDF export and 3D model support improvements by @js2264 in #86
  • Inter-arm alignment: one row per sequence, reach to alignment edge by @js2264 in #87
  • Reading frames, BAM base modifications, called sets as lanes, 3D by BED names, a live corner dot plot by @js2264 in #88
  • Validation notebook: use public data with accessions by @js2264 in #89
  • Cache reader answers to avoid re-reading visited maps by @js2264 in #90
  • Decide a circle on two separations; a tab's note goes with its file (§8.15, §5.6) by @js2264 in #91
  • Fold the columns from the digit row, step on the angle brackets, lock the map onto its diagonal, say how to zoom in, and scale the 1D browser's tracks together (§5.5, §5.8, §7.2, §8.9, §26.6) by @js2264 in #92
  • Read a whole chromosome one bin to a pixel up to 16,384 bins a side (§7.2, §9, §16.7) by @js2264 in #93
  • Add custom colour picker for all colour inputs by @js2264 in #95
  • Light structures in PDF exports with shaded beads and banded tubes by @js2264 in #94
  • Add pan and back/forward navigation controls by @js2264 in #96
  • Hide and show the reader's marks, one or all, on every map, track and model (§8.1, §25.5) by @js2264 in #98
  • Search a genome read by range for a motif, chromosome by chromosome, and open a --sequence by its index (§8.8, §8.21, §18.8, §23.1) by @js2264 in #99
  • Draw a called set as one row, a 1D track or on the map as its display says; boundaries as points or short diagonal markers, domains as corners (§8.1, §18.5, §18.7, §18.8) by @js2264 in #100
  • Track colouring: white-to-colour scale and editable key range by @js2264 in #101
  • Rename the app chromosomix, rebuild the lab's data stores around the paper, bring workspaces back as saved, and add Figs. 1 and 2 by @js2264 in #103
  • Fix figure window input and area fill behavior by @js2264 in #104
  • Serve the site from chromosomix.app, and keep every archived build working there by @js2264 in #106
  • Code review: CI oracles, security, CLI, capped readers, render cancellation, PDF paths, and the Session and MatrixPanel splits by @js2264 in #105
  • APA as coolpuppy computes it, with O/E and a five-bin nearest loop by default; track correlation over exact windows, a missing one left out of every pair; chromosomix boost by @js2264 in #108
  • Validation and timing notebooks, published at /validation/ and /timings/ with every release by @js2264 in #109
  • Tab navigation, derived resolution tasks, and figure rendering by @js2264 in #110
  • Faster DI, compartments, coverage and metaplot: Lanczos, rows from the index, held bigWig reads by @js2264 in #111
  • Add name column toggle to map's Tracks card by @js2264 in #113
  • Add guided tutorial for first-time users by @js2264 in #112
  • The npm package, ready to publish; the README reworked; the site written for search engines and language models by @js2264 in #114

Full Changelog: v0.13.0...v0.15.0

v0.14.0

Choose a tag to compare

@js2264 js2264 released this 17 Oct 18:57
f453599

What's Changed

  • Unify interface grammar across all reader-facing surfaces by @js2264 in #69
  • pastis-NB, four analysis cards, and a deck cell per model (§18.8, §25.3, §25.5, §25.6) by @js2264 in #68
  • Read a bed3d whose solver left bins unplaced as the structure it is (§25.2) by @js2264 in #70
  • Metagenome sequences card and unified level control by @js2264 in #71
  • The Map card follows the star, and no map or structure need be starred (§16.5, §25.3) by @js2264 in #73
  • Add bacterial chromosome analysis: origins, arms, and group signal by @js2264 in #72
  • Move synthetic tracks to Tracks card, add points style by @js2264 in #74
  • Inter-arm alignment's table gives the enrichment, as a fold by @js2264 in #76
  • Add peak calling task with yapc and IDR implementations by @js2264 in #75
  • Open every map of a multi-file drop in its cell (§16.11) by @js2264 in #77
  • Overlay map selection and validation notebook by @js2264 in #79
  • Draw maps to a chosen number of contacts (§6.5) by @js2264 in #78
  • Draw the inter-arm decay alone: inter-arm in the score picker on a circle (§7.5, §8.15) by @js2264 in #80
  • P(s)'s four orientations: from the import, from a dropped .pairs, and unbroken (§18.8) by @js2264 in #81
  • Reference editing: dev feature with stage, method, and exports by @js2264 in #82
  • Chromosome ideograms and RNA-seq splice junctions as 1D tracks (§8.22, §8.23) by @js2264 in #83
  • Open a large .hic without waiting on its contact count (§3.6, §6.5, §12) by @js2264 in #84
  • Draw a chromosome clicked during the contact count in seconds (§6.5, §8.15, §9, §12) by @js2264 in #85
  • Vector PDF export and 3D model support improvements by @js2264 in #86
  • Inter-arm alignment: one row per sequence, reach to alignment edge by @js2264 in #87
  • Reading frames, BAM base modifications, called sets as lanes, 3D by BED names, a live corner dot plot by @js2264 in #88
  • Validation notebook: use public data with accessions by @js2264 in #89
  • Cache reader answers to avoid re-reading visited maps by @js2264 in #90
  • Decide a circle on two separations; a tab's note goes with its file (§8.15, §5.6) by @js2264 in #91
  • Fold the columns from the digit row, step on the angle brackets, lock the map onto its diagonal, say how to zoom in, and scale the 1D browser's tracks together (§5.5, §5.8, §7.2, §8.9, §26.6) by @js2264 in #92
  • Read a whole chromosome one bin to a pixel up to 16,384 bins a side (§7.2, §9, §16.7) by @js2264 in #93
  • Add custom colour picker for all colour inputs by @js2264 in #95
  • Light structures in PDF exports with shaded beads and banded tubes by @js2264 in #94
  • Add pan and back/forward navigation controls by @js2264 in #96
  • Hide and show the reader's marks, one or all, on every map, track and model (§8.1, §25.5) by @js2264 in #98
  • Search a genome read by range for a motif, chromosome by chromosome, and open a --sequence by its index (§8.8, §8.21, §18.8, §23.1) by @js2264 in #99
  • Draw a called set as one row, a 1D track or on the map as its display says; boundaries as points or short diagonal markers, domains as corners (§8.1, §18.5, §18.7, §18.8) by @js2264 in #100
  • Track colouring: white-to-colour scale and editable key range by @js2264 in #101
  • Rename the app chromosomix, rebuild the lab's data stores around the paper, bring workspaces back as saved, and add Figs. 1 and 2 by @js2264 in #103
  • Fix figure window input and area fill behavior by @js2264 in #104
  • Serve the site from chromosomix.app, and keep every archived build working there by @js2264 in #106
  • Code review: CI oracles, security, CLI, capped readers, render cancellation, PDF paths, and the Session and MatrixPanel splits by @js2264 in #105
  • APA as coolpuppy computes it, with O/E and a five-bin nearest loop by default; track correlation over exact windows, a missing one left out of every pair; chromosomix boost by @js2264 in #108
  • Validation and timing notebooks, published at /validation/ and /timings/ with every release by @js2264 in #109
  • Tab navigation, derived resolution tasks, and figure rendering by @js2264 in #110
  • Faster DI, compartments, coverage and metaplot: Lanczos, rows from the index, held bigWig reads by @js2264 in #111
  • Add name column toggle to map's Tracks card by @js2264 in #113
  • Add guided tutorial for first-time users by @js2264 in #112
  • The npm package, ready to publish; the README reworked; the site written for search engines and language models by @js2264 in #114

Full Changelog: v0.13.0...v0.14.0

v0.13.0

Choose a tag to compare

@github-actions github-actions released this 15 Oct 20:37

What's Changed

  • Domain pileup APA and multi-scale insulation with heatmap rendering by @js2264 in #42
  • Add CLI command export and card folding UI features by @js2264 in #43
  • Publish a preview of every pull request on the Pages site by @js2264 in #45
  • Remote sources beyond S3: hubs, portal search, share links, Google Drive and Dropbox by @js2264 in #44
  • Publish the generated hubs through the site, because the repository is private by @js2264 in #48
  • Five new analysis tasks, and two that could not answer a question they were for by @js2264 in #46
  • Public stores: IGV's genomes, all of 4DN, all of ENCODE, and sessions with the tracks on by @js2264 in #50
  • Three near-diagonal readings, a 4C coverage divisor, a ribbon lane — and circular sequences by @js2264 in #51
  • Segments say which strand, which way and which column, on every interval lane by @js2264 in #52
  • Divide a map of several ticked genomes by genome, not by contig by @js2264 in #49
  • Rename hubs to stores and redesign store interface by @js2264 in #53
  • Open files from one door: Drive and Dropbox sign-ins, index-free BAMs, Drive by name and by range, and stores that list only what draws by @js2264 in #54
  • Enable all analysis tasks to run from CLI by supporting tracks and peer maps by @js2264 in #55
  • Add legend toggle, deck export, and compartments resolution warnings by @js2264 in #56
  • Give the four stores their marks, trim the picker and the Connect dialog, and stop a first visit eating a shared link by @js2264 in #57
  • Add DNA Zoo front end and bucket folder naming support by @js2264 in #59
  • Add barChart and bigBarChart support (§8.17) by @js2264 in #60
  • Add folder mounting to landing page with new connect doors by @js2264 in #58
  • Let a species with no README.json keep its files, as --readme optional says by @js2264 in #61
  • Support bacterial GFF annotations with no hierarchy by @js2264 in #64
  • Redefine HiC score as mean of bin's line across near-diagonal by @js2264 in #63
  • Read five UCSC formats, and open a track hub as a store (§8.18–§8.21, §24.10) by @js2264 in #62
  • Redesign landing page, unify button styles, and improve visual hierarchy by @js2264 in #65
  • A 3D structure beside the map, and every figure restyles (§25, §18.13) by @js2264 in #66
  • Resolutions the file does not store, zoom both ways from the bar, and an APA that pools its anchors by @js2264 in #67

Full Changelog: v0.12.0...v0.13.0

v0.12.0

Choose a tag to compare

@js2264 js2264 released this 10 Sep 08:56

Added

  • Find a sequence — a new analysis task. Choose a loaded reference, type a
    motif, and every place it occurs comes back as two lanes. The pattern is
    IUPAC — GGTCANNNTGACC, GATC, TTAGGG — spaces are ignored, and both
    strands are searched unless you say otherwise; a palindrome is reported once
    rather than twice, because searching both strands of GAATTC would otherwise
    count every site again and a reader totting up a lane would get double the
    answer with nothing on screen to explain it.

    Two lanes, not one, and that is the point. A set of sites answers two
    questions and no single picture answers both: where exactly is it, which is
    an interval per site — clickable, strand-carrying, exportable as BED, and an
    eligible input to overlap, the metaplot and APA like any other interval set
    (§18.3) — and how dense is it here, which is sites per bin and is the only
    one of the two that means anything at chromosome scale. The first is a picket
    fence when you zoom out; the second is a bar chart of ones when you zoom in.

    Two decisions inside it are worth stating because they are the ones that
    would be wrong by default. An N in the reference does not satisfy a
    pattern that names a base: an assembly gap is a base nobody knows, not a base
    that is any of four, and matching it would turn every gap into a solid bar of
    fictional hits — most of what you would see on a draft genome. And an empty
    bin in the density lane is a zero, never a gap: NaN in a per-bin vector
    draws as a hole, and "there is no site here" is the answer, not missing data.

    It is a registered task rather than a card of its own, which is what earns it
    the parameter form, a recipe that travels in the session, provenance on both
    outputs, a place in the deck's all-maps overlay, and Recompute on a
    reload — none of which a bespoke card would have had, and all of which every
    other task already has. venue: 'annotations', because the residues are in
    that worker and nowhere else: a genome is parsed into it and deliberately
    never shipped out whole, so searching anywhere else would mean hauling
    megabytes across the boundary or stitching thirty windowed reads together at
    the seams. A run declines a pattern that is not IUPAC before it starts, with
    the character it could not read named.

    It works in a genome-browser session as well as over a map, which is the
    session a reader who came in through a FASTA is actually in. That took
    fixing something older: a session with no contact map reported its bin size
    as 0, a placeholder that was safe only while nothing producing a per-bin
    result could be run without a matrix. A task answering from the sequence
    broke that assumption, and a zero bin size gives every contig zero bins — a
    lane that is permanently empty rather than one that is visibly wrong,
    which is the failure nothing reports. A mapless session is now given a
    browsing grid chosen from the assembly's own length; it is a resolution to
    draw at and never a claim that anything was measured at it, so the tasks
    that genuinely need a matrix go on saying they are waiting for one.

  • A fold button on every panel header. Each of the five columns now carries a
    chevron beside its shortcut chip, pointing the way that column travels — left
    for the three on the left, right for the two on the right. The affordance
    existed when a panel was folded (the rotated spine tab, which points the
    other way because it offers to open) and was missing when it was open, so the
    only way to close a column was a button in the header bar that does not look
    like it belongs to it. It goes through the same fold every other entry point
    does, so the remembered state, the relayout and the settle all happen once.

  • Shift-click ticks a range in the Tracks card. Ticking the first ten of
    thirty tracks was ten clicks on an eleven-pixel box, which is the arithmetic
    that makes a batch action cost more than doing the ten by hand. Click one box,
    shift-click another, and every row between the two takes the state the clicked
    box is moving to — the gesture every file list has. The anchor is remembered as
    a row's id and resolved to a position at click time, because the rows are
    drag-reorderable and the card's list is not the store's: an index cached at the
    first click is a lie the moment anything moves.

  • Z-score the tracks first, on the metaplot. A new option under Summary,
    off by default. Two tracks in different units — a CPM bigWig and a log2 ratio —
    plotted on one axis compare nothing; standardising each by its own mean and SD
    before the metric puts them on one scale. The statistics are taken per track
    and shared across that track's ranking groups, which is the part that is easy
    to get wrong: standardising each group separately would re-centre every one of
    them on zero and erase the ranking the reader split them by. A constant track
    is centred and not divided, so nothing comes back as ±∞, and gaps stay gaps.
    The heatmap shows the standardised values too, so the two halves of one figure
    cannot disagree, and the y axis is relabelled rather than left claiming a unit
    the numbers no longer have.

  • The viewer is published to npm, as a built directory rather than as
    something to import — so a page can embed a copy that depends on nobody's
    hosting, including ours:

    <iframe src="https://cdn.jsdelivr.net/npm/metacooler-viewer@0.11.0/dist/index.html?embed=1&config=…"></iframe>
    npm pack metacooler-viewer && tar xf metacooler-viewer-*.tgz && cp -r package/dist /var/www/viewer/

    A sixth invocation of the same bundler (vite.package.config.ts), and the one
    built for a path it does not know. Everything else this repository emits is
    served from somewhere decided in advance — the site root, /v/<commit>/, an
    asar, a disk — and bakes absolute asset URLs accordingly. A tarball is
    unpacked wherever somebody puts it, so its base is ./ and one published
    version serves all of them. The rule that comes with that is the mounter's:
    the URL has to end in a slash, or name index.html.

    Two things follow from a copy not knowing where it is, and both are enforced
    rather than assumed (IS_PORTABLE): it registers no service worker, since
    a copy mounted under a path it does not own must not claim a scope there — on
    a registry CDN that scope covers other people's packages — and it asks for
    no version manifest, since versions.json lives at a site root a vendored
    copy has no way to find. Before those guards, every vendored copy would have
    made two doomed requests: a 404 for a sw.js this build does not emit, and a
    cross-origin fetch for a manifest belonging to somebody else's server.

    files is dist/ and dist-cli/, which is the whole of what stands between
    a publish and 250 MB of demo data, and which makes npx metacooler-run work
    against a published version with no clone. prepack builds and
    check-package-build.mjs then refuses the result if it is wrong — a tarball
    packed over a site build would carry absolute asset URLs that 404 on every
    mount but the site root, and unlike everything else here a published version
    cannot be corrected.

    private: true is gone with its comment. The argument it made — an
    application, not a library — is still true and is not what publishing changes:
    what a registry is wanted for here is hosting nobody has to trust, not
    dependency resolution. Specified at SPECS §22.

  • A LICENSE file, with the MIT text the manifest has always declared. It
    was missing, which matters most at the moment a package is published: npm
    shows the licence of a package it can find one for, and a tarball that
    declares MIT without carrying it is asking to be taken on trust.

  • .pairs files open, at a resolution you choose. A 4DN pairs file — plain,
    gzipped or bgzipped, with a .px2 beside it or without one — is now an
    accepted input, and it is the first one the app imports rather than opens.
    Everything else in §3 is a matrix in a container; a pairs file is a list of
    contacts, and there is no grid in it. So the first open is refused, a dialog
    asks for a base resolution and a number of ladder rungs, and the file is read
    once. After that it is a map like any other: the same views, the same deck,
    the same turn, the same fourteen analysis tasks, none of which learns the
    container exists.

    The grid could not be guessed, and that is why there is a dialog rather
    than a default. A bin table inferred from the largest coordinate observed
    would differ between two subsamples of the same library — the one way of being
    wrong that survives every check downstream and makes two files silently
    incomparable.

    The ladder is nearly free, so it is offered rather than rationed: the base
    rung costs a pass over the records, and every rung above it is derived from
    that table, which is smaller than the record stream by the ratio of records to
    stored pixels — thirteen to one on demo/yeast/. The rungs are spelled the
    way an .mcool's resolutions are, which wires them into the resolution
    picker, the refine ladder and a deck descending to its lead's rung with no
    code in any of them.

    The shape of the data is what makes this the right design. On this
    repository's own demo files, yeast Micro-C is 179 M records and 13.7 M stored
    pixels at 1 kb; the mock community is 10.9 M records and 7.1 M pixels. The
    input is gigabytes and the output is tens of megabytes — so the file is paid
    for once, at open, and every view afterwards costs no I/O at all.
    demo/ecoli/SRR19500734.pairs.gz — 238 MB, 17.7 M records — imports in about
    forty seconds.

    What decides that time is the file's BGZF framing rather than this code,
    and it is worth knowing because it can be changed. A deflate...

Read more

v0.10.0

Choose a tag to compare

@github-actions github-actions released this 12 Aug 13:22

Added

  • Zooming sharpens the picture. A block larger than the screen has arrived
    coarsened since pooling landed — a whole yeast genome at 1 kb is drawn at
    3.0 kb cells, and the note under it said so honestly and told you the remedy
    was to open something smaller. That was true and it was not the interaction
    anybody has in their hands. A zoom that settles past what the block can
    show now reads that window again at the grid the picture could show: on the
    demo yeast cooler, 3.0 kb becomes 1.0 kb on one bounded read, and on a file
    that size the read comes out of the chunk cache without touching the network.

    It changes one clause of SPECS §2 and leaves three standing, which is the whole
    design. There is still no tile server, nothing is cut into tiles, and no
    picture is ever assembled out of parts: what is drawn is always exactly one
    bounded block. A gesture still costs nothing — the read goes out after the
    window stops moving, never during it, and neither the renderer nor the panel
    holds a reader to issue it with. It terminates, because there is no grid
    below the file's own bins: one step for a whole yeast genome, three for a human
    chromosome at 5 kb, and none at all for a MAG, whose block already is the
    finest data in the file — §2's own case is untouched. And it is bounded by
    the view
    : a zoom can never widen past the domain the view was entered on, so
    it cannot become the full pixel scan §7.3 gates.

    Zooming back out is served the same way and is usually free: two or three
    whole blocks are kept, so leaving what a fine one covers falls back to a coarse
    one still in hand. The colour range is measured from the widest of them and
    does not move while you zoom in — coarse cells are sums of bins, so a finer
    block holds smaller numbers, and re-measuring would re-brighten the map for
    somebody who changed nothing but the magnification.

    It refines under an O/E map, under two files blended on the diagonal, under a
    45° turn, and in every panel of a deck independently — the last of those
    because the navigation lock's currency was always base pairs, so five maps at
    five resolutions were already in step. The one picture that does not refine is
    the genome × genome overview, whose axes are sequences rather than positions:
    a cell there is a contig, and there is nothing finer to go to.

    The moment it happens is shown, not hidden. Crossing the edge of what the
    block can show freezes the map — no pan, no zoom — and puts a spinner in the
    corner of the plot until the new grid lands, so the picture is exactly where
    you left it when it sharpens and the change is one you can watch happen.
    Escape calls the read off and gives the map back, so no file can lock it.
    Panning to the edge of the block reads the next stretch at the same
    resolution
    : panning is a change of place, not of scale, and getting that
    wrong hands a reader back a four-times coarser map for dragging left.

    The key in the corner now names the cell size, on its own line, and in
    exported figures too. It is the one thing about a refining map that the pixels
    cannot tell you.

    Off by default, because a zoom that reads is a zoom that can wait and §2's
    promise has always been that it does not. The switch is the first of the three
    on the map's picture row — three bars, coarse at the top, fine at the bottom
    — beside the 45° turn and the O/E baseline, and Read finer data as you zoom
    in the command palette does the same thing. Off is exactly the old behaviour.
    The note under a pooled map now ends
    · zoom in to sharpen or · open a smaller block to sharpen depending on
    which. MULTIRES-ZOOM.md is the study this came out of, with a closing section
    on where the built thing differs from the plan.

  • The resolution ladder drives the zoom (§3.4, §9). "Wire the entries to the
    zoom control" has been in the spec since the first draft and in README's own
    list of gaps as inferred but not driven. A zoom now reads from the
    coarsest rung of an .mcool or .mmetacool whose bins still fill a cell — and
    never one finer than the level you opened, which is a choice the zoom does not
    get to overrule:
    fewer stored pixels for the same picture, and — the reason that matters — a
    balanced map that was balanced at that resolution rather than summed, which
    is what zoomify --balance produces and what the reader has been calling "the
    real answer" in a comment since pooling landed.

    The rungs a zoom touches are opened beside the one that is open, over the same
    mounted bytes, bounded at three and released with the session; a navigation
    still closes the map, because a navigation means the last place is finished
    with. Bin indices are not comparable between levels — five chromosomes of the
    demo .mcool have one fewer 2 kb bin than the division says — so windows cross
    through base pairs, per contig, with a whole sequence converting to its whole
    self at either end. None of it is required: a plain .metacool has no ladder
    and the zoom reads the open map, which is §3.4's own rule for what a ladder is
    allowed to be.

  • A 4C profile can be divided by P(s). A slice of a Hi-C map is mostly P(s):
    contact falls by orders of magnitude with distance, so a raw 4C profile's shape
    is dominated by how far away each bin is rather than by whether anything is
    happening there — a clean triangle with the anchor at its apex, on which the
    loops somebody ran the task to find are bumps visible near the anchor and
    invisible a megabase out. The task now offers a Normalisation choice: none,
    which stays the default because the raw profile is what a bigWig of a 4C
    experiment holds, or by P(s), which reports observed over expected at each
    separation.

    The decay is taken from the anchor's own sequence, accumulated from the
    square the task is already reading, rather than from the P(s) task's
    genome-wide curve: a small chromosome divided by the average behaviour of the
    large ones comes out with a slope that is a property of the set rather than of
    the locus. Bins in trans are divided by the mean of what the anchor reaches off
    its own sequence, since "separation" means nothing between chromosomes.

  • Synthetic tracks survive a change of resolution. Switching an .mcool from
    1 kb to 5 kb is not opening a different file — it is the same data at a
    different bin size — and everything a reader can see already treated it that
    way: the view carries across, and so does every dropped track. The one thing
    that did not was the output of an analysis. Run compartments genome-wide, wait
    the four minutes, change level to look at the same region more closely, and the
    Synthetic tracks card was empty: numbers the tab was still holding, dropped for
    no reason other than which object held them.

    They now come across live, as the rows the reader left — the colour, the name,
    the edge, the autoscale group, whether it was hidden — with the values still
    behind them and the exports still on them. Unrescaled, which is the honest
    thing and not a shortcut: a profile is one number per bin, its bins were 1 kb,
    and the row goes on saying so in its provenance line. Re-binning it to 5 kb
    would be inventing an insulation score no window ever produced. What makes that
    work is that a computed vector has always carried its own grid — the same
    reason a bigWig at one resolution draws correctly over a map at another.

    Called interval sets — insulation boundaries, CID domains, A/B compartments —
    come across the same way, with their features and their Save as BED. A row
    that was already waiting for a Recompute stays waiting for one rather than
    being promoted to a lane with nothing in it. And this is for a change of
    resolution only: opening a different file still leaves the analysis behind,
    because a computed row is an answer about the file being closed.

  • The Chromosomes table gets the browser's bar, and you can drag out a
    window.
    The table is where a reader of a cooler lands, and the only way out
    of it into a window was to type one — a row click opens a whole chromosome,
    and a quarter of a gigabase is a large read taken to reach a small one. The box
    that narrowed it asked for chrIV:1200000-1210000, which is a coordinate
    somebody is usually looking for rather than one they already have.

    So the header is now the 1D browser's own: the same locus box, with the same
    gene search behind it — by name or by accession, from whichever annotation the
    star picks — the same go, the same switch through to the tracks, and the
    same ruler. Dragging across a chromosome's band opens that window, which is
    the gesture the picture was missing; a press that does not travel opens the
    whole sequence under it, and * opens the whole genome as it does in the
    browser.

    The strip answers a pointer the way the browser's own does: a hairline
    follows it and the cursor is a caret. A band is fifty pixels of shading
    with a name in the middle, so without a line there is no way to see where in it
    a drag would start — on a chromosome an inch wide, a megabase of ambiguity —
    and the caret says the gesture names a stretch rather than moving an edge. The
    head above it is the bar and nothing else: the title and its sentence of prose
    are what a view needs when the reader has to be told where they are, and the
    breadcrumb, the ruler and the summary line all say it better.

    A band being dragged out looks exactly like one in the browser, from the
    same implementation: the wash, both edges as hairlines, and three chips — where
    the drag started, where the pointer is, and how much lies between. Those
    numbers are ...

Read more