feat: CLI improvements, FCS writing, and code cleanup#8
Merged
Conversation
- Standardize formatting in Cargo.toml files across multiple crates - Update repository URLs to reflect new structure - Enhance keywords and categories for better discoverability - Ensure consistent dependency declarations and script commands
Add export functionality to support multiple output formats: - Boolean CSV (0/1 values) for general use - Numeric CSV (2000/6000 values) for R compatibility - JSON metadata with comprehensive QC metrics Includes: - export_csv_boolean() and export_csv_numeric() functions - export_json_metadata() function - Export-specific error types - Unit tests for all export functions Dependencies added: - serde_json for JSON serialization - tempfile for test utilities
Add methods to PeacoQCResult for easier export: - export_csv_boolean() and export_csv_boolean_with_name() - export_csv_numeric() and export_csv_numeric_with_name() - export_json_metadata() These methods wrap the export functions and provide a more ergonomic API for users.
Add CLI flags to export QC results in multiple formats: - --export-csv: Export boolean CSV (0/1 values) - --export-csv-numeric: Export numeric CSV (2000/6000 values) - --export-json: Export JSON metadata - --csv-column-name: Custom column name for CSV exports Exports are automatically named when outputting to directories. Updated CLI README with export format documentation and examples.
Add comprehensive documentation for export formats: - Boolean CSV format description and use cases - Numeric CSV format (R-compatible) description - JSON metadata format with example structure - Custom column name examples - Integration notes for downstream tools (pandas, R, SQL) Includes code examples for each export format.
Update lock file with serde_json and tempfile dependencies added for export functionality.
Replace naive O(n×m) KDE implementation with FFT-based O(n log n) version for significant performance improvements. - Add realfft dependency for efficient FFT operations - Implement FFT-based convolution for KDE computation - Update benchmarks to reflect FFT implementation - Update performance analysis documentation Performance improvements: - 30-87x faster for typical use cases (1k-50k events) - Better scaling for larger datasets - No accuracy loss - all tests pass Benchmarks show ~48x speedup for default bin size (1k events), reducing KDE time from ~1.4ms to ~29µs per bin.
Add comprehensive FCS file writing capabilities to the previously read-only flow-fcs crate. New functions: - write_fcs_file: Write Fcs struct to disk - duplicate_fcs_file: Copy existing FCS file - edit_metadata_and_save: Modify metadata and persist changes - filter_events: Remove events based on boolean mask - concatenate_events: Combine multiple FCS files - add_column: Add new parameter column to existing data These utilities enable: - Creating modified FCS files (e.g., after QC filtering) - Adding computed columns (e.g., QC scores) - Combining multiple files - Metadata updates with proper keyword handling All functions maintain FCS format compliance and update relevant keywords (e.g., $TOT, $PAR) automatically.
Add comprehensive QC plot generation for visualizing PeacoQC results. Features: - create_qc_plots function for generating QC visualization plots - CLI integration with --plots flag and interactive prompts - Support for multiple cofactor values for comparison - Plot directory configuration - Integration with processing pipeline to store data for plotting CLI changes: - Add dialoguer dependency for interactive prompts - Add --plots, --plot_dir, --cofactor, --cofactors flags - Store FCS data and QC results for plot generation - Support multiple cofactor iterations for comparison This enables users to visualize QC results and compare different transformation parameters.
- Replace --remove-margins/--no-remove-margins with --keep-margins flag - Replace --remove-doublets/--no-remove-doublets with --keep-doublets flag - Default behavior: margins and doublets are removed (matches R recommendations) - Implement FCS file writing using flow-fcs write_fcs_file - Update documentation with examples for FCS output - Feature parity with R PeacoQC save_fcs=TRUE option
- Create DEV_NOTES.md consolidating technical implementation details - Update QUICK_START.md with cleaner examples - Update README.md with improved documentation - Add .gitignore entry for test artifacts
- Remove R reference implementation files (moved to separate location) - Remove test plot images - Clean up repository for production use
- Add regression tests for critical fixes - Add algorithm correctness tests - Add integration tests with known outputs - Add R compatibility tests - Add spline comparison tests - Add peak detection tests - Add test documentation in tests/README.md - Add debug utilities for QC development
- Improve QC algorithm implementations - Add plot generation functionality - Enhance error handling - Update dependencies - Improve code organization
- Add PeacoQC paper PDF for reference - Add cargo config for build settings
temporarily disabling this acton
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR includes several improvements to the PeacoQC CLI and codebase:
CLI Improvements
--remove-margins/--no-remove-marginswith--keep-marginsflag (default: margins removed)--remove-doublets/--no-remove-doubletswith--keep-doubletsflag (default: doublets removed)flow-fcscratesave_fcs=TRUEoptionDocumentation
DEV_NOTES.mdQUICK_START.mdwith cleaner examplesCode Quality
Cleanup
Commits
feat(cli): improve flag interface and implement FCS writingdocs: consolidate and clean up documentationchore: remove R reference files and test artifactstest: add comprehensive test suiterefactor: improve code quality and add featureschore: add reference materials and cargo config