Skip to content

Bug fix: update infer_data_source with a tdiff tolerance, and update resample_dataframe - #39

Merged
emmleroy merged 11 commits into
masterfrom
emmleroy-v1.0.0
Aug 28, 2026
Merged

Bug fix: update infer_data_source with a tdiff tolerance, and update resample_dataframe#39
emmleroy merged 11 commits into
masterfrom
emmleroy-v1.0.0

Conversation

@emmleroy

@emmleroy emmleroy commented Jul 30, 2026

Copy link
Copy Markdown
Contributor
  1. Several MOD-UFP files from the database have a dominant sampling interval of 58s or 59s, rather than 60s. This updates infer_data_source() with a 2-second tolerance around the expected dominant_tdiff values.

  2. A couple old column names need to be added to COLUMN_DEFINITIONS: (opcn3_temp --> opc_temp, opcn3_rh --> opc_rh, operating_state --> dd_operating_state).

  3. Bug fix: neph_bin0/opc_bin0 ratio check should occur when neph_bin0>=10, not when opc_bin0>=10 (following fw-modulair v17?)

  4. Fix: some int64 columns need to be nullable (fw, dd_operating_state, dd_measurement_state) for older data that doesn't have these values; changed dtype to float64

  5. Speed-up flag-aware resampling by changing to a vectorized version

  6. Add a force_nonnumeric to resample_dataframe to force some columns to be non-numeric (like 'fw') regardless of dtype

  7. Bump pre-release version to rc4

@emmleroy
emmleroy requested a review from dhhagan July 30, 2026 17:01
@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 39.34426% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.64%. Comparing base (20f6aa3) to head (82a84e8).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
quantaq_cli/toolkit/resample.py 35.08% 37 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #39      +/-   ##
==========================================
- Coverage   91.82%   89.64%   -2.19%     
==========================================
  Files          24       24              
  Lines        1260     1294      +34     
==========================================
+ Hits         1157     1160       +3     
- Misses        103      134      +31     
Flag Coverage Δ
unittests 89.64% <39.34%> (-2.19%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
quantaq_cli/schema.py 93.05% <ø> (-1.39%) ⬇️
quantaq_cli/utilities.py 82.85% <100.00%> (+0.24%) ⬆️
quantaq_cli/variables.py 90.00% <ø> (ø)
quantaq_cli/toolkit/resample.py 64.60% <35.08%> (-22.90%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@emmleroy emmleroy changed the title Bug fix: update infer_data_source with a tdiff tolerance and add extra columns to be renamed Bug fix: update infer_data_source with a tdiff tolerance, and update resample_dataframe Aug 28, 2026
@emmleroy
emmleroy merged commit 8e0d697 into master Aug 28, 2026
7 of 9 checks passed
@emmleroy
emmleroy deleted the emmleroy-v1.0.0 branch August 28, 2026 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants