Skip to content

Feature: support double precision option#33

Merged
dparrini merged 6 commits intodparrini:masterfrom
drewsilcock:feature/support-double-precision-option
Sep 7, 2023
Merged

Feature: support double precision option#33
dparrini merged 6 commits intodparrini:masterfrom
drewsilcock:feature/support-double-precision-option

Conversation

@drewsilcock
Copy link
Contributor

@drewsilcock drewsilcock commented Jun 13, 2022

This adds a keyword argument to the Comtrade() constructor called use_double_precision (default: false) which causes the time and analog data arrays to use double-precision floating point precision instead of single precision.

This also adds:

  • Some very simple tests for the new option.
  • A modification to a previously failing UTF-8 test.
  • A GitHub Action to run the tests so that they run on all pushes and pull requests.

This arose from a need we have to have double-precision on the timestamps specifically - the time values are in nanosecond precision and single precision floating point only gets you 7-8 decimal points of precision.

As an example, if you take the 2013 sample file, the sampling rate is 1200 Hz and there are 40 data points which means the last data point has timestamp 39 / 1200 = 0.0325 but when loaded into the single-precision array it gets stored as 0.032499998807907104. The larger the index of the data point, the larger the discrepancy. We're dealing with files that have almost 400,000 data points so the precision errors are non-insignificant.

@dparrini
Copy link
Owner

dparrini commented Sep 7, 2023

Hi @drewsilcock , this is a nice feature. I just solved some old issues and this caused some conflicts with your PR.

@dparrini dparrini merged commit 6e035f6 into dparrini:master Sep 7, 2023
@drewsilcock drewsilcock deleted the feature/support-double-precision-option branch October 2, 2023 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants