Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR implements unit conversion functionality for ModelTimeSeries objects, along with extensive code reorganization and testing improvements. The changes modernize the codebase by consolidating DataFrame constructors, adding proper logging decorators, and enhancing the testing infrastructure.
Key changes include:
- Implementation of unit conversion capabilities for model time series data
- Reorganization of DataFrame construction logic into centralized modules
- Addition of comprehensive logging decorators throughout the codebase
- Migration from
toolkit_tests/to standardizedtests/directory structure
Reviewed Changes
Copilot reviewed 77 out of 163 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/ | New comprehensive test files covering model data, xarray conversions, and addition operations |
| src/metobs_toolkit/xrconversions.py | New module for converting MetObs objects to xarray format |
| src/metobs_toolkit/backend_collection/ | New utility modules for uniqueness operations, datetime aggregation, and DataFrame construction |
| metobs_toolkit/station.py | Enhanced Station class with model data management and addition operations |
| metobs_toolkit/sensordata.py | Updated SensorData class with logging decorators and addition functionality |
| toolkit_tests/ | Updated existing tests with parameter name changes and new test methods |
| pyproject.toml | Version bump and dependency updates including xarray requirement |
Comments suppressed due to low confidence (8)
vergauwenthomas
force-pushed
the
modeltimeseries_unit_conv
branch
from
251e944 to
6789976
Compare
Owner
Author
|
Closes #545 |
Merged
vergauwenthomas
added a commit
that referenced
this pull request
Sep 5, 2025
* Modeltimeseries unit conv (#545) * functionality for unit conversion in creation of modeltimeseries * import at top * update docstring of Modeltimeseries * rename obstype to modelobstype for modeltimeseries * black edits * rename obstype attr * typo * fixing bugs * Update the exaple to illustrate how units are converted * Update tests and fixed bugs * code review fixes * black edits * Update src/metobs_toolkit/modeltimeseries.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * minor version bump * Add to_parquet and to_csv methods for Dataset and Station classes (#556) * Initial plan * Implement to_parquet and to_csv methods for Dataset and Station classes Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> * Update documentation for to_parquet and to_csv methods Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> * Apply black formatting to tests and validate implementation Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> * fix the parquet implementation * update the introduction example * new subsection in api docs * update docstrings * fix formatting the csv test * black edits --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> Co-authored-by: Thomas Vergauwen <thomas.vergauwen@meteo.be> * Update README.md to include conda install and badge (#555) * Update README.md to include conda install and badge * flags before package name convention * update the docs to include conda install description --------- Co-authored-by: Thomas Vergauwen <thomas.vergauwen@meteo.be> * Implement CF-compliant netCDF serialization for xarray Datasets with nested attributes (#558) * Initial plan * Implement core netCDF serialization functionality with CF compliance Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> * Complete netCDF serialization implementation with documentation and examples Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> * version in seperate file to be accesible by other methods * fix version in CI * fix serializable cr datasets * update docstrings * add label conversion in the example * fix bugs and xr tests * fix version test * fix the to_netcdf methods * add to netcdf tests in the xr testing module * code review style fixes * black edits * add to netcdf mode in the xarray topic * to_netcdf in the introduction example * sync version with pyproject * bugfix * fix merge issue with version * add netcdf4 --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> Co-authored-by: Thomas Vergauwen <thomas.vergauwen@meteo.be> * Parquet reader (#557) * Adding tests and Parquet reader * Adding parquet test data * trigger tests * fixed mismatch in tz + tests + docstring * not intended for commit * replace csvfilereader by the find_suitable_reader function * update notebook * bugfix * black edits * fix failing test * fix tests --------- Co-authored-by: nea-ppatel <patel_pratiman@nea.gov.sg> Co-authored-by: Thomas Vergauwen <thomas.vergauwen@meteo.be> * Qc on dataset error handling (#560) * drop faulty files * raise warnings if target station does not hold target obstype when qc on dataset level * write test * rename func * black edits * use tmp module in tests * Fix NaTType error in frequency estimation for empty variable lists (#562) * Initial plan * Fix NaTType error in frequency estimation for empty timestamps Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> * fix issue by selecting a minimum of 1 non-na values * Delete tests/test_timestampmatcher.py * test with nans * black edits * use warning instead of logging --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> Co-authored-by: Thomas Vergauwen <thomas.vergauwen@meteo.be> * Standardize warning formatting by converting operational warnings to logging (#565) * Initial plan * Replace warnings.warn with logger.warning for operational warnings Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> * Update test to check for logging instead of warnings Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> * fix warnings bug * Check stations for obstype when GF is called on Dataset (#566) * add filter_to_stations_with_target_obstype to GF methods on Dataset * black edits * black edits * Implement human-readable __repr__ methods for all main classes (#568) * Initial plan * Implement human-readable __repr__ methods for all main classes Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> * improve the __repr__ returns --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: vergauwenthomas <82087298+vergauwenthomas@users.noreply.github.com> Co-authored-by: Thomas Vergauwen <thomas.vergauwen@meteo.be> * reduce xarray restriction to >=2022.3.0 * review * black edits * use tempdir for parquet files in test --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: Pratiman <31694629+pratiman-91@users.noreply.github.com> Co-authored-by: nea-ppatel <patel_pratiman@nea.gov.sg>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.