MSF_plot_tools

Small command-line plotting tools for MSF GC/MS outputs.

This cleaned project keeps the strict TSV-based workflow and the raw-text converter:

• MSF_calplot_tsv.py
• MSF_hourplot.py
• MSF_ticplot_tsv.py
• convert_search_output_txt_to_tsv.py
• convert_nistlog_txt_to_tsv.py

The goal is to keep the supported formats narrow, fail loudly on malformed input, and make the common tasks easy for lab mates to run from the command line.

Install

Create and activate a virtual environment, then install the requirements:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Supported Formats

Strict search-results TSV

Used by MSF_calplot_tsv.py and MSF_hourplot.py.

It must be a headerless tab-separated file with exactly 5 columns in this order:

1. site such as AMB, CAL, or CYL
2. sample_name such as 00004_2025_01_01__06H19_AMB60_C6_C12_elu
3. integrated_signal
4. compound_name
5. cas_number

Example row:

AMB	00004_2025_01_01__06H19_AMB60_C6_C12_elu	66,374,622	Butane	106-97-8

Null-period TSV

Optional companion file for MSF_calplot_tsv.py.

It must be a headerless 2-column TSV with:

1. start_date
2. end_date

Dates must be YYYY-MM-DD, and ranges are inclusive.

Example row:

2025-01-12	2025-01-18

Strict TIC summary TSV

Used by MSF_ticplot_tsv.py.

It must be a headerless 2-column TSV with:

1. sample_name
2. benzene_ppbv_equivalent

Example row:

00563_2024_12_29__07H19_AMB60_C6_C12_elu	610.3

Raw search-output text files

Used by convert_search_output_txt_to_tsv.py.

The converter expects UTF-16 .txt files exported from the search-output reports and rewrites them into the strict 5-column TSV format above.

Raw NISTLOG text files

Used by convert_nistlog_txt_to_tsv.py.

The converter expects the line-oriented NISTLOG-*.txt files where each record is an Unknown: line followed by a Hit 1: line, and rewrites them into the same strict 5-column TSV format above.

Quick Start

Run a calendar plot for the example search-results TSV:

python MSF_calplot_tsv.py \
  examples/strict_search_results_demo.tsv \
  --site AMB \
  --value-mode hit_count \
  --null-periods examples/null_periods_demo.tsv \
  --save-dir out/calplots \
  --no-show

Run the hour plot for the same input:

python MSF_hourplot.py \
  examples/strict_search_results_demo.tsv \
  --site AMB \
  --hour-mode detections \
  --top-n 2 \
  --save-dir out/hourplots \
  --no-show

Run the TIC summary plot:

python MSF_ticplot_tsv.py \
  examples/tic_summary_demo.tsv \
  --title "Demo TIC benzene-equivalent summary" \
  --output out/tic_summary.png \
  --no-show

Convert raw text exports into the strict search-results TSV:

python convert_search_output_txt_to_tsv.py raw_exports -o data/search_results.tsv

Convert a NISTLOG text file into the strict search-results TSV:

python convert_nistlog_txt_to_tsv.py data/NISTLOG-2022-1-519pks.txt

Script Notes

MSF_calplot_tsv.py

• Plots one calendar heatmap per compound.
• Supports presence, hit_count, max_value, and sum_value.
• Supports null collection periods in a separate TSV.
• Uses a fixed null color outside the main colormap.
• Saves one PNG per compound when --save-dir is provided.

MSF_hourplot.py

• Plots one hourly bar chart per compound.
• Uses time of day on the horizontal axis.
• Labels titles as compound (CAS: number).
• Supports:
  • detections
  • sum_value
  • mean_value
• Supports normalization by unique samples per hour.

MSF_ticplot_tsv.py

• Plots one calendar heatmap from the 2-column TIC benzene-equivalent summary TSV.
• Does not require site, compound, or CAS columns.

convert_search_output_txt_to_tsv.py

• Reads all matching .txt files in a directory.
• Converts them to a strict 5-column TSV.
• Normalizes the single-underscore timestamp variant into the standard double-underscore sample format.
• Skips malformed blocks by default and reports them at the end.
• Use --fail-on-error if you want conversion to stop on the first malformed block.

convert_nistlog_txt_to_tsv.py

• Reads a line-oriented NISTLOG-*.txt file.
• Converts each Unknown: / Hit 1: pair into the strict 5-column TSV.
• Normalizes source tokens like CALIB, CYL07, and 2CYL into strict site codes such as CAL and CYL.

Examples

The examples/ folder contains small demo inputs:

• strict_search_results_demo.tsv
• null_periods_demo.tsv
• tic_summary_demo.tsv

These are meant for smoke testing and documentation, not for scientific use.

Tests

Run the smoke tests with:

python -m unittest discover -s tests -v

The tests use Matplotlib's non-interactive Agg backend and write outputs to temporary directories.

Notes For Lab Use

• The scripts intentionally validate input formats strictly.
• Use --site ALL to disable site filtering for the search-results tools.
• If you want to publish this repo publicly, review the license choice and replace it if your lab prefers something else.