Skip to content

Releases: ultra-trace-systems/mascope

v1.11.0

Choose a tag to compare

@kausiala kausiala released this 09 Oct 13:55
9b5dfae

Software that controls the instrument can now tell Mascope what each file is:
the File Agent sends an acquisition record with every upload, the server binds
the file by the chemistry the record names, and the agent is also available as
a Python library. Orbitrap spectra now draw the signal the instrument measured,
and calibration no longer depends on which server does it. Peak assignment
still ships off.

Added

  • Acquisition records. A program that controls the instrument can leave a
    small JSON file beside each data file, <file>.mascope.json, saying which
    run and step the file belongs to and under which chemistry it was measured.
    The File Agent sends it with the upload, together with the file's SHA-256.
    The server keeps it, binds the file to the ionization mode whose token the
    record names - before it reads the file's name - and lists the record's
    identifiers and the hash in the batch spreadsheet. A record never costs a
    file its upload.
  • The File Agent is also a Python library, mascope-file-agent on PyPI,
    for a program that already runs on the instrument computer and wants the
    uploads in its own process. The Windows program, its installer and its
    guided setup are as they were.
  • mascope_sdk.acquisition defines the record's schema,
    mascope-acquisition/1, with parse(), dump() and read_sidecar().
  • A file with no ionization mode token in its name can be bound by its
    acquisition method
    , instead of waiting for somebody to choose a chemistry.
    Off by default: set method_binding = "route" in the server's [backend]
    config. It can only affect a file that would park today, since a file-name
    token still wins. mascope prod db script run report_method_binding_disagreements
    shows what it would do before you switch it on; it writes nothing.
  • Every sample records how its chemistry was decided - by its acquisition
    record, a file-name token, its acquisition method or a person - and Raw
    files says which while the file is processed.
  • Exports can be traced back to what produced them. The Batch data
    spreadsheet carries the ids of the records behind each row, each sample's
    acquisition context and m/z calibration, and a new Provenance sheet naming
    the deployment and the versions that wrote the file. Each deployment has an
    id of its own; GET /api/provenance returns it, and the SDK's loaders carry
    it on df.attrs["provenance"].
  • Scan streams. Every experiment of an Orbitrap acquisition method is now
    recorded for each file. A new setting, composite_scan_streams, detects the
    peaks of a method that measures a sample as several scan ranges per
    experiment and stitches them into one spectrum per polarity. It ships off
    and is not yet for production
    ; it will be announced again when it is.

Changed

  • Orbitrap spectra show the signal the instrument measured, where they
    drew one Gaussian per detected peak. A shoulder, an asymmetry or an
    unresolved neighbour is now visible. Peaks look more angular, because a raw
    file keeps about three points across a peak's width, and the points are
    drawn where they can be told apart. The SDK's get_spectrum and
    get_spectra return the same measured profile.
  • A method binding follows the mode its method's files actually use, once
    the last three files of a method agree on it. Nothing already processed is
    re-bound.
  • Error monitoring groups a warning by where it was logged, not by what it
    says, so one problem is one issue rather than one per file.
  • A file's scan streams follow the experiments of its acquisition method
    in mascope file scans and in a file's processing detail. Nothing is
    processed differently.
  • The File Agent names the files it leaves behind when it is stopped, and
    a third Ctrl+C stops it without waiting for the uploads under way.
  • The demo dataset is bundle v1.3.1
    (10.5281/zenodo.23187550).

Fixed

Uploads:

  • Signing in no longer fails the uploads you still have queued. A second
    tab, another device or an expired session used to make the converter refuse
    every file still waiting.
  • A file the converter fails on leaves no sample directory behind, so
    uploading it again works.

Calibration:

  • A file's m/z calibration no longer depends on the server that calibrates
    it.
    Where two calibrants share a peak, as a labelled reagent does with its
    unlabelled compound, which one was used followed a randomly generated id.
  • Calibrants that disagree are no longer chosen between by rounding; the
    one closest to its theoretical mass is kept.
  • A calibration point's match score is its own. It was computed against
    another point's reference whenever an earlier calibrant had found no peak.

Orbitrap profiles:

  • The averaged profile sits on its peaks where the calibration moves
    between scans, on long acquisitions whose calibration drifts, and at the
    ends of each scan's range.
  • A cached profile no longer outlives the reader that averaged it.
  • An Orbitrap file kept without its raw file no longer shows its spectra
    one calibration off its own peaks.

Peaks:

  • A file that switches polarity no longer loses all of its peaks to two
    that share an m/z.
  • A peak's timeseries can no longer be written to the peak next to it.
  • Selecting a target ion whose main isotope has no detected peak draws its
    isotope plots again.

Live updates:

  • A large composition search no longer cuts the server's live updates or
    loses its own results.
    It lists its 500 best candidates; the count beside
    the peak still says how many it found.
  • Copying a large sample batch no longer floods the live updates, and
    re-processing or calibrating many files reports counts and the first ten
    failures.

File Agent:

  • It no longer stops when a new file is gone before its upload. A file
    deleted or renamed in those seconds used to end the agent until somebody
    restarted it.
  • Ctrl+C is handled at every point, where once in a great while it ended
    the agent with a traceback.
  • The installer carries the licences of the open-source software in the
    program.
  • The setup says where "Pair an agent" is: the Home menu's Settings tab.

SDK:

  • A time-ranged peak read says when it is short of peaks, and how to
    complete it.
    The frames list their warnings in df.attrs["warnings"],
    samples.compute_peak_timeseries() computes what is missing, and the
    client's concat() combines loads without dropping warnings or provenance,
    which pd.concat does.

And the sample browser's assignment status column explains itself in help
mode again.

Security

  • Creating an ionization mechanism refuses one too long to store while
    the request is validated, with a 422 naming the limit of 256 characters. A
    longer one used to be parsed whole first, which an editor could use to hold
    a backend process's memory.
  • Dependencies: urllib3 2.8.0, in the server image and the File Agent's
    installer, for three advisories; tornado 6.5.10, a development dependency
    only, for three more.

Upgrade notes

  • Four database migrations, applied automatically at start, on a single
    linear head. They add columns and a table and recreate a view; none
    rewrites existing rows.
  • Upgrade the CLI before the stack, as usual: the CLI and the runtime
    changed this release.
  • Re-run mascope prod db script run backfill_method_bindings after
    upgrading, so the method bindings learned so far follow the mode their
    newest files use. It merges, and running it twice changes nothing.
  • Error monitoring opens a burst of new issues after the upgrade: every
    warning and error logged without an exception is regrouped once. The old
    issues stop receiving events and can be resolved.
  • A reverse proxy of your own in front of Mascope has to let a request
    header line of 32 KB through on the upload path (docs/hosting.md).
    Where it does not, files still arrive, without their acquisition records.
    The bundled nginx is already set.
  • Orbitrap profiles are averaged again on the first view of each sample
    after the upgrade. The caches left behind are never read again; reclaim the
    space with
    python -m mascope_backend.db.admin.filestore delete-stale-sum-signal.
    Do not use delete-sum-signal for this: it also deletes a TOF file's full
    sum signal.
  • What moves in the numbers. Peak positions do not. A small number of
    peak heights change by a few percent on files whose calibration moves
    between scans. A sample on a calibration collection whose calibrants share
    a peak moves by up to 0.5 ppm when it is calibrated again; a sample already
    calibrated keeps its calibration until then.
  • The new settings ship off. method_binding stays "shadow" and
    composite_scan_streams stays false, which is where it should stay on a
    production server. Peak assignment remains peak_assignment = false.
  • File Agents already installed keep uploading as before. An agent sends
    acquisition records once it is updated to this release, and only to a
    server that announces it keeps them.
  • REST and SDK callers:
    • get_spectrum and get_spectra return the measured profile, not a
      reconstruction from the peak list.
    • A time-ranged peak load whose every peak was left out is an empty frame
      carrying a warning, where it was None. The peaks response lists its
      warnings in a warnings field beside data.
    • A composition search's results are fetched from
      GET /api/cheminfo/mz/match/result/{process_id} rather than carried in
      its notification, and hold at most 500 candidates.
    • The spreadsheet and CSV exports only gain sheets and columns; the ones
      earlier exports had keep their names and their places.
    • mascope-sdk depends on pydantic from this version on.
  • On PyPI with this release: mascope-sdk...
Read more

v1.10.1

Choose a tag to compare

@kausiala kausiala released this 01 Oct 11:04
f5e80bf

A fix release for 1.10.0: creating a target compound works again, and the raw
reader moves to opentfraw 2.0.0 with a corrected averaged profile, so Thermo
peak intensities rise by about 3% and no longer vary with mass.

Fixed

  • Creating a target compound works again. Since 1.10.0, adding a compound -
    directly, by saving a target collection that adds one, or by editing a
    compound's formula - failed with an error.
  • A shipped ionization mechanism such as [M+Br2]- is created on a server
    holding large halogenated compounds.
    Before, it failed and was rolled back
    at every start.
  • A failed target compound create or delete no longer leaks a database
    connection.
  • Creating several target compounds reports each one under its own entry
    in the response's message log.
  • Three single-scan Thermo files that only the Thermo library could open are
    readable again.
  • Error monitoring is quieter, and more useful where it does report. A
    batch that is already repairing itself after an upgrade no longer opens a
    warning, and neither does a server that has no instruments yet. An error
    event now carries the log line that says what failed. For a file the
    converter could not process, that line also gives the file's size and when
    it was last written.

Changed

  • The raw reader is on opentfraw 2.0.0, and the averaged profile reads the
    points the instrument measured.
    The old averaging read each peak between
    its stored samples. That lost a few percent of the apex, and the loss varied
    across the mass range by up to 2.4 percentage points, which distorted the
    shape of a spectrum as well as its scale. That variation is now 0.08 points.
    Thermo peak intensities rise by about 3%. Peak m/z values are unchanged,
    and so are isotope ratios.
  • The demo dataset is bundle v1.3.0
    (10.5281/zenodo.23077479), rebuilt
    with the new reader. mascope demo uses it by default.

Added

  • Scan statistics report PacketCount, SegmentNumber and ScanEventNumber
    under the OpenTFRaw reader.

Performance

  • Scan selection no longer decodes any peaks: 88.9 ms -> 19.0 ms on a
    1,486-scan file.

Security

  • The backend image updates PyJWT to 2.15.1, which closes ten advisories, one
    of them rated critical. Mascope loads no JWKs or public keys, so the
    key-confusion advisories had nothing to attack. The remaining fixes apply to
    every session token decode. Existing sessions stay valid.

Upgrade notes

  • No database migrations.
  • Upgrade the CLI before the stack, as usual. mascope-cli and
    mascope-runtime 2026.10.1 publish with this release.
  • Thermo intensities step up by about 3% at the upgrade. Files converted
    after the upgrade read higher than files converted before it. Files already
    in Mascope keep the intensities they were converted with. A time series or
    comparison that spans the upgrade will show the step. m/z values are not
    affected.

Installers

The two File Agent installers are identical - Mascope-File-Agent-Setup.exe is
the fixed-name copy the in-app download button points at,
Mascope-File-Agent-Setup-v1.10.1.exe the versioned copy for your records.
Download either.

v1.10.0

Choose a tag to compare

@kausiala kausiala released this 30 Sep 07:48
9b54935

Ionization settings learn what each instrument actually runs: a mode can now
belong to one instrument, Mascope ships the chemistries and mechanisms it
knows, and it records which chemistry each acquisition method uses. Peak
assignment gained a great deal in this release and still ships off, so
nothing about it changes for a deployment that has not opted in.

Added

  • An ionization mode can belong to one instrument. Left as "Every
    instrument" it behaves exactly as before, which is every existing mode. Set
    to an instrument, its filename token is only matched against that
    instrument's files - so two instruments can use the same token and mean a
    different chemistry by it.
  • Mascope ships the chemistries and ionization mechanisms it knows, seeded
    at every start, so a new deployment is not a blank page.
  • Mascope records which chemistry each acquisition method runs. Nothing is
    routed on it yet; this release only builds the record.
  • The composition search finds a compound at any of its isotopologue lines,
    not only its monoisotopic one.
  • The peak browser can group rows by neutral formula and shows what a
    reference list calls each one.
  • A sample's own noise estimate and measured mass accuracy are on the API.
  • Curated atmospheric CIMS reference lists ship with Mascope.
  • The scan-stream census can be filled in for files converted before it
    existed.

Changed

  • Ionization mechanisms are written in the standard adduct notation -
    [M+H]+, [M-H]-, [M+Br]-. The older spelling is still accepted when you
    type it; what is stored and shown is the standard one.
  • Peak assignment still ships off. The assignment work in this release
    stays dark until a deployment turns it on.

Fixed

  • A sample's properties survive a write that does not finish.
  • The match tab opens on the ion's monoisotopic line, and isotope colours
    survive a reload.
  • Edit Ionization Settings sits still: its tabs no longer jump, it refuses a
    mechanism that already exists, the modes table scrolls once, and the
    instrument field no longer draws its label over its placeholder.
  • An expanded composition-search result can be scrolled, and the search answers
    for a wide element box under distant adducts.

Upgrade notes

  • Six database migrations, applied automatically at start, on a single
    linear head. Five only change structure; one rewrites every stored ionization
    mechanism into the standard notation (and writes it back if you downgrade).
    The window is that rewrite plus a few index builds.
  • REST and SDK callers: mechanisms are returned in the standard notation. A
    caller that matches on the old spelling must read the new one. Input is
    unaffected - the older spelling is still accepted.
  • Upgrade the CLI before the stack, as usual: the CLI changed this release.
  • Peak assignment remains peak_assignment = false. A deployment that wants it
    must opt in deliberately.

Installers

The two File Agent installers are identical - Mascope-File-Agent-Setup.exe is
the fixed-name copy the in-app download button points at,
Mascope-File-Agent-Setup-v1.10.0.exe the versioned copy for your records.
Download either.

v1.9.0

Choose a tag to compare

@kausiala kausiala released this 24 Sep 07:30
f3bf755

A file Mascope cannot route no longer fails: it waits, says so, and tells
someone. This release makes ingestion explain itself - every raw file now
records how far processing got and why it stopped, files that need a person
are kept as notifications until read, and a file whose name names no
ionization mode can be given one by hand instead of being lost. The File
Agent follows each upload through to its outcome, and neither a file's name
nor its instrument has to be spelled a particular way any more.

Added

  • A file that binds to no ionization mode waits for a chemistry instead of
    failing.
    It is converted and stored as usual and shows in Raw files as
    Needs a chemistry with no samples. Select such files, right-click, and
    Choose chemistry to pick a mode per polarity; they are then processed as
    if their names had carried the tokens. The dialog also opens the ionization
    settings, for the common case that no configured mode is the right one, and
    comes back with your files still selected. POST /api/sample/files/bind
    does the same over the API.

  • Files that need someone are kept until read. When auto-processing leaves
    a file failed, needing a chemistry, or unmatched because calibration failed,
    the people answerable for the instrument are told even if nobody is signed
    in - the uploader (for a paired File Agent, the device's sponsor) and the
    owners of the instrument's acquisition workspace. They collect under Needs
    attention
    in the notifications pane, one entry per instrument and kind.
    GET /api/notifications and POST /api/notifications/read expose them.

  • Raw files shows how far processing got. Every sample file records a
    status - converted, bound, calibrated, then done, needs a chemistry,
    calibration failed or failed - with a hover detail saying why. The table has
    a Status column and a server-side filter, and GET /api/sample/files takes
    processing_status. A run that records nothing for a day has stopped, and
    Raw files says so.

  • Raw files lists every instrument at once. The instrument selector has an
    All instruments entry, and a new sortable Instrument column leads the
    table, so files from several instruments can be read side by side.

  • The File Agent says what became of each file it uploaded. After each
    upload it asks the server at a widening interval, for up to three hours, and
    logs each stage it sees: converted, bound, calibrated, then processed, needs
    a chemistry, calibration failed or processing failed, with the server's
    reason. Failures are logged as warnings or errors.

  • The browser uploads files that name no ionization mode, with one note
    that they may need their chemistry chosen in Raw files.

  • Each Orbitrap file's scan streams are recorded in its .props, grouped
    by scan filter and FT resolution, with counts, time spans and per-stream
    acquisition parameters. Nothing is processed per stream yet.

  • mascope file scans <file.raw> lists a raw file's scan streams, their
    counts, blocks, time spans and strongest peaks, and says when a polarity
    holds more than one MS1 stream. --json prints the full census.

Changed

  • A browser upload can name the instrument it came from, including one
    Mascope has never seen. A file whose name starts with no known instrument
    used to be accepted only under a name saying its own class (orbi, tof,
    api); the dialog now reports the instrument you pick, so any name of 1-64
    letters, digits and hyphens is accepted. The files keep their own names, and
    a name with no files yet creates the instrument after you confirm.

  • Both raw-file readers report every per-scan statistics field, and each
    scan's whole trailer
    . OpenTFRaw, the default reader, now reports all
    twenty ScanStats fields and the instrument's full trailer under its own
    labels rather than five of Mascope's own. See Upgrade notes.

Fixed

  • A NaN in an API response no longer fails the request as a bad one. One
    non-finite number used to fail the whole request with 400 and log below the
    level error monitoring sees. Such numbers are sent as null now, with a
    warning naming the route.

  • Reading an Orbitrap file's metadata no longer fails.
    GET /api/sample/files/{id}/metadata answered 400 for files read by the
    default reader, because missing charge states and precursor m/z became NaN.

  • Orbitrap sample files record their instrument method again. Since
    ingestion moved to the open-source reader every Orbitrap file was stored
    with an empty method file. Repair the files ingested meanwhile with the
    script in Upgrade notes.

  • Dual-polarity files keep the matches of both polarities. Calibrating one
    polarity removed the matches just computed for the other; every sample of a
    file is now calibrated before any is matched.

  • A file name must match exactly one ionization mode per polarity. A +-
    file whose name matched two positive modes and no negative one used to be
    processed twice as positive and never as negative.

  • Auto-processing no longer opens an error-monitoring issue per file.
    Since 1.8.0 every early-stopped run was logged as an error, so monitoring
    opened a separate alerting issue for each - over three hundred in three days
    on one production server, none of them faults. Blank files now finish like
    any other, and routine outcomes are logged at INFO.

  • Deleting an instrument config deletes only that config, rather than
    every config sharing its instrument and method file.

  • A File Agent upload that fails to process now tells someone. The error
    used to go only to the machine account nobody signs in as.

  • Averaging a profile without conversion parameters works again. The
    fallback for scans whose trailer carries no Conversion Parameters B and C
    called a NumPy function removed in NumPy 2, so it raised on every call.


Upgrade notes

Database migrations run automatically when the stack comes up (db_init).
Two, both cheap: four nullable columns plus two indexes on sample_file, and
two new empty tables. Nothing is backfilled, so the window is the two index
builds.

Reinstall the CLI before bringing the stack up. The checkout has just
swapped in this release's docker-compose.yaml, and the CLI is what matches
it. This release adds no compose secret, but the ordering still applies.

Run the method-file repair on each server after upgrading:

DRY_RUN=1 mascope prod db script run populate_orbitrap_method_file   # preview
mascope prod db script run populate_orbitrap_method_file

It reads the method name out of each raw file's header - header only, no
re-conversion - for Orbitrap files stored with an empty method file since the
reader changed.

GET /api/sample/files/{id}/metadata drops five keys. The labels of
Mascope's own that the default reader used to report - Ion Injection Time (ms), Charge State, Precursor m/z, Isolation Width (m/z) and Collision Energy - are replaced by the instrument's own trailer, under its labels and
in its order. A direct REST caller reading them must switch to the trailer's
keys, which carry a trailing colon (Ion Injection Time (ms):); an MS2 scan's
precursor and collision energy are in its filter (ScanType). The published
mascope-sdk does not expose this route, so SDK users are unaffected.

Files processed before this release carry no processing status and no
registration time. They read as "not recorded" rather than being backfilled,
so the Status column and its filter are empty for them until a file is
processed again.


The two File Agent installers are identical - Mascope-File-Agent-Setup.exe
is the fixed-name copy the in-app download button points at,
Mascope-File-Agent-Setup-v1.9.0.exe the versioned copy for your records.
Download either.

v1.8.1

Choose a tag to compare

@kausiala kausiala released this 16 Sep 14:29
bfdac0b
  • The badge now tells a poor calibration (amber) from an instrument whose axis drifted but calibrated well (teal). A failed calibration is red.
  • Pasting can create the new dataset or batch it needs.
  • The dataset in the top bar has the full dataset menu.
  • A newly created dataset or workspace opens reliably.

v1.8.0

Choose a tag to compare

@kausiala kausiala released this 15 Sep 12:01
849a210
  • Accounts and agents:
    • Optional TOTP two-factor authentication; no policy is enforced by default.
    • Agent device pairing with machine accounts, short-lived renewable device tokens, and per-device revocation.
  • File Agent: reports its instrument and release; uploads are filed under the reported instrument.
  • Processing fixes: upload and ingest reliability; Orbitrap peak splitting under lock-mass lag; MS2 from acquisitions that record it after the MS1 block. MS2 spectra can be grouped by collision energy on request.
  • Peak assignment and the batch ledger are included but off by default. Set peak_assignment = true under [meta] to opt in.
  • Other: an About tab with legal links and third-party notices, and the OpenAPI document published at /docs/openapi.json.

v1.7.3

Choose a tag to compare

@kausiala kausiala released this 01 Sep 15:18
9b9e54d
[v1.7.3]

v1.7.2

Choose a tag to compare

@kausiala kausiala released this 19 Aug 08:42
9fdf083
[v1.7.2]

v1.7.1

Choose a tag to compare

@kausiala kausiala released this 19 Aug 07:29
ae08a52
[v1.7.1]

v1.7.0

Choose a tag to compare

@kausiala kausiala released this 19 Aug 07:36
3b1fdd1
[v1.7.0]