v0.3.1

mfethuls - v0.3.1

PyPi package made available. Package features are the same as v0.3.0

v0.3.0

mfethuls v0.3.0

A friendlier start, and safer data.

New

• Welcome page: see what mfethuls does, then Get started (choose your data folder and registry in the app, with live checks and a folder-layout guide), Try the example (a full walkthrough on example DSC, TGA, FTIR and fluorescence data), or Use it from Python (opens the tutorial notebook).
• Getting-started checklist with a compact tracker in the sidebar; progress is remembered between sessions.
• ⚙ Settings: data folder and registry, Clean up, data loading and example data in one place; 🏠 returns to the welcome page.
• Plots: Style by (line style or marker shape) next to Colour by; axis limits open above the chart instead of over it; SVG exports no longer cut off legends.
• Tutorial included: mfethuls-tutorial opens a marimo notebook on its own copy of the example data; the example data now ships with the package (mfethuls.examples).
• Python helpers: mfethuls.settings.use_folders(...) and mfethuls.storage.catalog_path().

Safer data

• Files that don't read as the chosen instrument's data are flagged in Review & register and are never ingested silently.
• More robust file writes on Windows when a virus scanner or OneDrive briefly locks a file.

Changes

• Folders chosen in the app are saved in ~/.mfethuls/settings.json and take precedence over .env.
• The .env setup wizard is gone from launch.bat and launch.sh; setup happens in the app.
• The example data moved from examples/ to the package (mfethuls.examples), and the tutorial moved from notebooks/ to the package (mfethuls.tutorials).

v0.2.0

mfethuls v0.2.0

This release makes it much easier to get instrument data into mfethuls and keep it tidy. The app finds raw data you haven't registered yet, a new Clean up step keeps everything in line with your registry, and plotting is ready for publication figures. mfethuls is now licensed under Apache 2.0.

Highlights

Register new data from the app

• The sidebar shows how many raw data files or folders in your data folder aren't in the registry yet. Review & register proposes the registry rows for them, with the instrument, file name and run filled in where the data allows.
• Subfolders can be registered as one experiment (e.g. a titration or time series) or as one experiment per file (repeated runs).
• Rows are checked as you type, using the same rules as ingest, and saved straight into your CSV or XLSX registry. A backup of the previous version is kept.
• Filters for recent data (last 30 days by default), instrument folder and name keep long lists manageable.
• Files next to your data that the instrument can't read, such as a plot saved in the experiment folder, are listed and skipped. They don't block registering.

Clean up after registry changes

• The new Clean up section, and python -m mfethuls.reconcile on the command line, compare what was ingested with your registry:
  • Forget experiments you removed from the registry.
  • Refresh renamed or corrected ones.
  • Relink experiments whose raw file you renamed: they're recognised by identical file contents.
  • Remove stored files nothing uses any more.
• Check shows the plan, and Apply only does what you reviewed. Your raw data is never touched.
• Two safety rules: rows that are in the registry but invalid still protect their data, and a plan that would forget many experiments at once needs an extra confirmation.

Experiments can be renamed

• Change an experiment's name in the registry and ingest again: it keeps its internal ID and stored data.
• Each experiment now records a fingerprint (content hash) of its raw files, which is how renamed files are recognised and identical data is flagged.

Publication-ready plotting

• Two plotting backends:
  • Matplotlib (default): a fixed publication style and vector SVG/PDF output.
  • Plotly (WebGL): interactive, showing every data point. Choose it with backend="plotly" or mfethuls.set_plot_backend("plotly").
• Axis labels with units (Temperature (°C), Wavenumber (cm⁻¹)).
• Time-resolved series get an elapsed-time column (time_s) and are coloured by time with a colour bar.
• Better DSC plots: segments are coloured by type, and overlays are easier to read.
• In the app: set axis limits, and export vector SVG or interactive HTML.

Instruments and data

• Ocean Optics Flame fluorescence support, with its own plot.
• Multi-file experiments get a source_file column, so each curve in a series stays separate.
• Example data (DSC, TGA, FTIR, fluorescence) and a registry template (experiments_template.csv), for trying mfethuls without your own data.

Improvements

• Registry object: load a registry with Registry.load(path) and pass it to the functions that need it. It always reflects the current file, and Registry.problems lists skipped rows and why.
• Catalog: the DuckDB catalog is the single record of ingested datasets. Older catalogs are upgraded automatically.
• Experiment ID file (.mfethuls_manifest.json):
  • changes are made under a lock and saved safely (a crash mid-write can't corrupt it);
  • an unreadable file stops ingest instead of silently starting over.
• Configuration: all environment and .env settings are read in one place (mfethuls.settings).
• Starting the app: the app is now part of the package. Start it with mfethuls-app; launch.bat and launch.sh use it.
• Packaging: tests run in CI on Ubuntu and Windows, and the package metadata states the licence in the current standard form.

Breaking changes

• Ingest and load functions take a stores= argument (see mfethuls.stores) in place of storage_mode, cloud_provider, db_url and query_backend. The default uses local storage, so most notebook code needs no change.
• MFETHULS_MODE is no longer used. Remove it from your .env.
• The REST API, background worker, Postgres integration and S3/Azure storage are no longer included in this package. mfethuls now focuses on local, single-user analysis: notebooks, the command line and the app.
• Imports: use_test_env moved to mfethuls.settings; it is still available as mfethuls.use_test_env.
• storage.notebook: list_datasets() and get_dataset() read the local DuckDB catalog and no longer take a database URL.
• Starting the app: streamlit run apps/Home.py is replaced by mfethuls-app.
• Registry functions: the global registry functions are replaced by the Registry object.

Upgrading

1. Update .env from the new env_example, removing MFETHULS_MODE and any Postgres/S3 settings.
2. We recommend a clean re-ingest so every experiment gets a content fingerprint:
   • delete .mfethuls_manifest.json in your data folder, the DuckDB catalog and the Parquet output folder;
   • ingest again.

License

mfethuls is now licensed under the Apache License 2.0 (previously MIT). Releases up to v0.1.1 remain available under MIT. Contributions are welcome; first-time contributors are asked to sign a Contributor License Agreement (see CONTRIBUTING.md).

---

Three things to check before publishing:

• CLA mention: the License section says contributors sign a CLA. If CLA Assistant isn't set up yet, remove that sentence or set it up first.
• README: it still mentions mfethuls-server in a "For teams" section. If you want the release to stay low-key about the split, consider removing that line from the README too.
• Re-ingest advice: the Upgrading step assumes local use. If people already run this against the Droplet's data, do the re-ingest there separately.

v0.1.0 — Initial public release

First stable release of mfethuls — a registry-driven ETL framework for laboratory instrument data.

Parsers — DSC, TGA, FTIR, UV-Vis, NMR, SEC, DMA, Rheometer, SAXS, MS (13 instruments) with canonical schema normalisation.

Pipeline — registry CSV describes experiments, system matches raw files by filename, outputs Parquet + DuckDB catalog + Postgres metadata.

Service mode — FastAPI + background worker, Bearer token auth, OneDrive sync via rclone, per-experiment ingest with live progress.

Streamlit dashboard — sync → ingest → browse datasets → ad-hoc plots → SVG/HTML export.

Deployment — DigitalOcean + Tailscale + Docker Compose, full guide in docs/deployment.md.