Skip to content

v5.0.0

Choose a tag to compare

@pgarrett-scripps pgarrett-scripps released this 25 Sep 00:45
153b298

Breaking API cleanup for 5.0. docs/migration.md has the full old-to-new table.

Removed

  • plot_centroiding tolerance keywords mz_tolerance, mz_tolerance_type, im_tolerance, im_tolerance_type, min_peaks, max_peaks (deprecated in 4.1). Pass centroid=MergePeaksCentroider(...).
  • camelCase names with no alias: TimsData.indexToMz, mzToIndex, scanNumToOneOverK0, oneOverK0ToScanNum, scanNumToVoltage, voltageToScanNum, readScans; tdfpy.timsdata.oneOverK0ToCCSforMz, ccsToOneOverK0forMz, ccsToOneOverK0ToCCSforMz.
  • tdfpy.centroiding.batch_iterator, calculate_nmass, get_tdf_df and Peak, unused helpers outside the documented API.
  • DIAMs1Frame.dia_windows and PRMMs1Frame.prm_transitions, which were always empty. Use DIA.windows / PRM.transitions.

Added

  • tdfpy.errors: TdfpyError(ValueError) base, TdfpyKeyError (+KeyError), ReaderClosedError (+RuntimeError), AcquisitionTypeError. All exported from tdfpy.
  • Opening a DDA, DIA or PRM reader on another acquisition type raises AcquisitionTypeError ("not a DDA acquisition (found DIA); use tdfpy.get_acquisition_type()"). It used to open and return empty or wrong elements.
  • AcquisitionType (StrEnum), MsMsType, Polarity (a Literal), MetaValue, ook0_to_ccs and ccs_to_ook0 exported from tdfpy.
  • tdfpy.types with ToleranceUnit = Literal["da", "ppm"] and Polarity = Literal["positive", "negative"], the same names, values and order as tacular.types (tdfpy does not depend on tacular). Both exported from tdfpy; the m/z tolerance parameters are typed ToleranceUnit.
  • ms_level on every frame (1 for MS1, 2 for MS/MS) and on PasefFrameMsmsInfo, DiaWindow and PrmTransition (always 2), next to msms_type. The window classes carry msms_type as a class constant (DDA_MS2, DIA_MS2, PRM_MS2).
  • Precursor.precursor_mz: monoisotopic_mz when known, else largest_peak_mz. The raw fields stay.
  • Ms1FrameLookup.query_range(rt_range=) and query(rt=, rt_tolerance=30): MS1 frames by retention time.
  • tdfpy.iter_precursor_spectra(precursors): yields (precursor, precursor.merged_peaks()), decoding each PASEF frame once instead of once per precursor window.
  • scan_peaks(), raw_peaks() and centroid() on PasefFrameMsmsInfo.
  • Lookups: ids(), id in lookup, and get(id, default) on every lookup.
  • __all__ in every public module; Raises: sections in docstrings.
  • docs/migration.md ("Migrating to 5.0").
  • MCP: library errors (TdfpyError, missing paths, bad indices, unsupported formats) reach the client as tool errors with their message, e.g. "precursor queries require DDA, but this acquisition is DIA". The SDK used to replace them with a bare "Error executing tool ".
  • MCP: unknown tool arguments are rejected ("Extra inputs are not permitted"). They used to be dropped silently, so the 4.x query_frames(rt=...) returned unfiltered rows.
  • tests/test_mcp_vocabulary.py: drift guard for MCP schema names, shared enums, unknown-argument rejection, record keys and error text.

Changed

  • Renamed, no alias: .time -> .rt (Frame, PrmTarget); PrmTarget.one_over_k0 -> ook0; Frame.mz_calibration, tims_calibration, property_group -> *_id; Precursor.parent_frame -> parent_frame_id; PasefFrameMsmsInfo.precursor -> precursor_id; window_group -> window_group_id on DiaWindowGroup / DiaWindow; MetaData.one_over_k0_acq_range* -> ook0_acq_range*; TimsCalibration.scan_to_one_over_k0 / one_over_k0_to_scan -> scan_to_ook0 / ook0_to_scan; tdfpy.calibration.one_over_k0_to_ccs / ccs_to_one_over_k0 -> ook0_to_ccs / ccs_to_ook0; camelCase TimsData methods -> snake_case (index_to_mz, mz_to_index, scan_num_to_ook0, ook0_to_scan_num, scan_num_to_voltage, voltage_to_scan_num, read_scans).
  • Every spectral accessor is a method. The per-scan .peaks property of Frame, DiaWindow and PrmTransition is now scan_peaks(). Precursor.peaks and PasefFrameMsmsInfo.peaks (one mobility-collapsed (N, 2) spectrum, the most expensive accessor) are now merged_peaks(), and Precursor.pasef_peaks is pasef_merged_peaks().
  • Mobility ranges are (low, high), like isolation_mz_range, ook0_acq_range and query_range(ook0_range=). ook0_range, ccs_range and voltage_range on PasefFrameMsmsInfo, Precursor, DiaWindow and PrmTransition used to come out high-to-low because scan number and 1/K0 run in opposite directions; *_begin is now the low value and *_end the high one.
  • FrameMetadata (exported from tdfpy) renamed its fields to match Frame: time -> rt, mz_calibration -> mz_calibration_id, tims_calibration -> tims_calibration_id, property_group -> property_group_id.
  • TimsData(path, *, use_recalibrated_state=..., pressure_compensation_strategy=...): the options are keyword-only.
  • After a reader closes, reader.timsdata, metadata, calibration and pandas_tdf raise ReaderClosedError, as do Precursor.ook0 / ccs and the mobility ranges. timsdata used to hand back the closed handle and metadata kept reading the file.
  • AcquisitionType.UNKNOWN is "unknown" (was "Unknown").
  • TimsData.read_scans and read_frame_arrays raise TdfpyError for a non-integer bound or a range outside 0 <= scan_begin <= scan_end <= num_scans; an empty range (begin == end) is valid and reads nothing. read_scans used to pad out-of-range scans with empty arrays and read_frame_arrays clamped them.
  • Window accessors (scan_peaks, raw_peaks, centroid, merged_peaks) raise TdfpyError when a PASEF/DIA window's scan range does not fit its frame (a corrupt file). scan_peaks used to clamp it silently.
  • Every error is a TdfpyError. Lookup misses and missing MetaData / Calibration keys raise TdfpyKeyError; spectral access after close raises ReaderClosedError; SQLite errors in PandasTdf, convert_table_to_df and slice_d_folder raise TdfpyError (convert_table_to_df raised RuntimeError). PandasTdf.get_table_names opens the database read-only.
  • get_acquisition_type returns AcquisitionType; Frame.msms_type is MsMsType. Both still compare equal to the old str / int values.
  • Elements are frozen, slotted, keyword-only dataclasses. PrmTarget equality and hashing ignore transitions.
  • DiaWindowLookup[id] and PrmTransitionLookup[id] return tuples (were lists); DIA.window_groups returns a tuple (was a generator). query() / query_range() arguments are keyword-only, and DiaWindowLookup's window_group_index= is now window_group=.
  • MetaData and Calibration are read-only Mappings; the df field is replaced by table.
  • plot_centroiding: every argument after frame_id is keyword-only, including mobility_type.
  • MCP query_dia_windows: parameter window_group -> window_group_id.
  • MCP query_frames and the entity tools (query_precursors, query_dia_windows, query_prm_targets, query_prm_transitions): range parameter rt -> rt_range; mz -> precursor_mz_range on the precursor and PRM-target tools and isolation_mz_range on the DIA-window and PRM-transition tools.
  • ion_mobility_type -> mobility_type, no alias, on every function and method that takes it (get_raw_peaks, get_centroided_spectrum, convert, raw_peaks() / centroid() on every element, iter_window_spectra, the Centroider call, plot_centroiding), matching mzmlpy's MCP mobility_type. Values are unchanged ("ook0", "ccs", "voltage").
  • MCP Processing.ion_mobility_type -> mobility_type (values "ook0", "voltage") in preview_spectrum, export_spectrum and export_window_batch.
  • MCP query_frames rows use Frame attribute names instead of raw Frames SQL columns: Id -> frame_id, Time -> rt, Polarity "+"/"-" -> polarity "positive"/"negative", ScanMode -> scan_mode, MsMsType -> msms_type, TimsId -> tims_id, MaxIntensity -> base_peak_intensity, SummedIntensities -> total_ion_current, NumScans -> num_scans, NumPeaks -> num_peaks, MzCalibration -> mz_calibration_id, T1/T2 -> t1/t2, TimsCalibration -> tims_calibration_id, PropertyGroup -> property_group_id, AccumulationTime -> accumulation_time, RampTime -> ramp_time; each row adds ms_level and a frame selection. The page is {items, total, offset, next_offset} like the other query tools (was {table, columns, offset, rows, next_offset}). read_metadata_table still returns raw columns.
  • MCP frame metadata in convert_coordinates (frame) and in spectrum/export metadata (frames) reports polarity as "positive"/"negative" (was "+"/"-").
  • MCP server_info units key retention_time -> rt.
  • Reader vocabulary shared with mzmlpy, renamed with no alias: mz_range -> isolation_mz_range on PasefFrameMsmsInfo, DiaWindow, PrmTransition and Precursor; mz_begin / mz_end removed (use isolation_mz_range); PrmTarget.monoisotopic_mz -> precursor_mz; Frame.summed_intensities -> total_ion_current; Frame.max_intensity -> base_peak_intensity.
  • Polarity is Literal["positive", "negative"], not a StrEnum, and every polarity field is Polarity | None. A frame whose Polarity column is not + / - gives None and one UserWarning per file; a precursor whose PASEF frames disagree gives None with a warning. FrameMetadata.polarity stays the raw "+" / "-" string.
  • Lookup keywords (same as mzmlpy 0.10): a (low, high) tuple is *_range, a point is rt= / precursor_mz= / ook0= plus a tolerance. PrecursorLookup.query_range(mz_range=) and PrmTargetLookup.query_range(mz_range=) -> precursor_mz_range=; PrecursorLookup.query(mz=) and PrmTargetLookup.query(mz=) -> precursor_mz=.
  • Tolerance switches follow the tacular-omics naming rule, renamed with no alias: mz_tolerance_type -> mz_tolerance_unit and im_tolerance_type -> im_tolerance_unit on merge_peaks, get_mobility_collapsed_spectrum, MergePeaksCentroider, PrecursorLookup.query and PrmTargetLookup.query. Values are unchanged ("da" / "ppm" for m/z; "relative" / "absolute" for 1/K0).
  • tdfpy.constants.PROTON_MASS is the full CODATA 2018 value 1.007276466621 (was truncated to 1.007276466), the same as tacular.constants.PROTON_MASS.
  • The DDA, DIA and PRM readers load their tables with plain sqlite queries instead of pandas iterrows; SQLite errors there raise TdfpyError ("Failed to read TDF database ...").
  • Reading a scan range (read_frame_arrays, read_scans, every window accessor) decodes TOF indices for those scans only. Whole-frame layout checks still run; the TOF-bounds check now covers the scans read.
  • MergePeaksCentroider sorts peaks on the integer TOF index (a counting sort under numba) before converting to m/z, and merge_peaks (both kernels) puts its input in one canonical order: m/z, then descending 1/K0 (that is, TOF then scan for frame data), then intensity. Input already in that order, as the reader produces it, skips the sort after an O(n) check; anything else is lexsorted into it. The greedy seed order is a stable sort on descending intensity, so equal-intensity seeds are taken in that canonical order instead of an unstable quicksort order. Results no longer depend on the NumPy version or on input order, including when many points share one m/z; only points identical in m/z, 1/K0 and intensity keep their input order, and those are interchangeable. Measured against 4.x on 12 dense MS1 frames each: 1.43% of centroids differ on a 15-min diaPASEF run (11,919 of 834,389) and 1.16% on a 15-min DDA run (4,278 of 368,896); summed intensity per frame moves by under 0.02%. merged_peaks() output is unchanged.

Performance

Speedups only, indicative, not exclusive timing: single-core direct runs on a shared, loaded machine, 5.0 against commit b783db4 (readers still built with pandas iterrows, as in 4.1).

workload speedup
open DDA, 35-min plasma run (174,658 precursors) about 9x
open DDA, 15-min HeLa run about 8x
open DIA, 15-min run about 3.5x
centroid() on MS1 frames about 1.1x
merged_peaks() over 2000 precursors about 1.25x in a loop, about 3.3x with iter_precursor_spectra
centroid() over 1500 DIA windows about 1.9x in a loop; iter_window_spectra itself about 1.8x