v2.4.0

Added

• New recommended data writing functions have been added AtriumSDK.write_segment, AtriumSDK.write_segments, AtriumSDK.write_time_value_pairs. These functions replace write_data_easy and write_data (although those functions will remain to support older code). These functions provide a simpler interface and write_segment(s) is designed to work directly with continuous strips of data which are found in many waveform storage formats, archives and streaming message formats.
• A new buffer context manager AtriumSDK.write_buffer has been added for writing to AtriumDB, all writes with write_segment(s) and write_time_value_pairs within the buffers context will collect and combine the individual pieces of data and flush them to the dataset when they hit an efficient threshold, are manually flushed or exits the context so that the dataset can be built optimally without the need for external logic from the user.
• You can now pass in your own custom FileHandler subclass so that AtriumDB can continue to work on otherwise not-supported file systems.
• MacOS is still not supported by AtriumDB, but a helpful Error now alerts users attemping to run AtriumDB on a Mac.
• A new function called AtriumSDK.load_device which replaces passing in the block_info param in get_data, it caches the file location of waveform data in RAM which then dramatically improves the performance of small data queries. Highly recommended when taking on large computation projects. After calling AtriumSDK.load_device the data queried will not reflect any new data added after AtriumSDK.load_device was called until it is called again.
• A new param in AtriumSDK.get_data, return_nan_filled changes the return type to a continuous array of values from start_time_n to end_time_n with np.nan values filling in any gaps in data.
• The Iterator is now powered by the Nan Filled Gap C function, which simplifies and speeds up the sorting/filling of window signals dictionaries.
• Dramatically decreased the overhead of starting up a DatasetIterator for a large dataset by querying all relevant data at once and then organizing that information in RAM.

Fixed

• All OS based file functions have been moved back into the FileHandler class so that those functions can be abstracted to other storage types in the future.
• Gap based time format (time type 2) sorting and merging is now written with a much simpler and robust strategy.
• Block merging now correctly handles a change in time/value type for a given measure-device, issuing a warning when types are mixed instead of throwing an error.

Removed

• AtriumSDK.write_data_easy, AtriumSDK.write_data are no longer recommended for writing data to a dataset, see AtriumSDK.write_segment, AtriumSDK.write_segments, AtriumSDK.write_time_value_pairs in the Reference API and the Quick Start / Tutorial in the docs.

v2.3.0

Added

• CSV exports create human readable folders and can output time information as datestrings instead of epochs.
• Added option for Parquet exports.
• Block Merging: Small pieces of new data written to AtriumDB now have the option to be merged into existing blocks, maintaining higher efficiency get_data calls.
• Data Transfer De-identification Files can now be reused even if the source dataset has added patients since creation.
• Unused TSC files can be detected and removed.
• get_device_patient_data now truncates its output to agree with given start/end time parameters.
• MariaDB SQL Handler now uses custom Connection Manager to leave open a single connection for all SDK uses.
• TSC Exports now, where possible, reuse blocks from the source SDK instead of re-encoding them, dramatically increasing transfer speeds.

Fixed

• Intervals of continuous data will now be properly bounded to the requested start/end times.
• Intervals wholly within other intervals will no longer cause interval data to be deleted.
• User entered parameters that will end up in the back end database are now force cast to Python native types to avoid known issues writing numpy objects to SQL/YAML.
• Blocks now concatenate correctly when reading a large sequential group of blocks in a single file.
• Intervals queried by patient are now properly bounded.
• Encounters with NULL end times (encounter ongoing) will be replaced with time.time_ns().

Removed

• The iterator number of samples per window can no longer be a floating point value (will be rounded to nearest int).

v2.2.3

Fixed

• Fixed a bug where patient_ids weren't returned by insert_patient, when duplicate patients were entered through the AtriumSDK Object.
• Fixed a bug that prevented time data from using lz4 and zstd compression. This feature is useful for storing aperiodic signals.

v2.2.2

Fixed

• fix cli requests, qrcodet import bug by adding them to cli section of pyproject.toml
• fix incorrect url with blocks endpoint
• change the websocket connection to only happen once get_data is called in api mode not when the sdk object is created
• fix small bug where if validate_token was false they sdk would error out because there is no expiry time

v2.2.1

Fixed

• Fixed a bug that required wfdb to be installed to import any main atriumdb functions/classes. Now wfdb is only required when testing or exporting in wfdb format, as intended.

v2.2.0

Added

• Websocket Remote Data Requests: Upgraded API now utilizes WebSockets for faster data retrieval of bulky binary block data, improving the performance significantly for get data queries. REST API continues to be used for metadata requests.
• Persistent Remote Connection: A single WebSocket connection is maintained open for the duration of the AtriumSDK object's lifetime, optimizing data querying.
• Refresh Token Authentication: SDK objects in API mode now require a Refresh Token along with an authentication token, facilitating automatic refreshing of authentication tokens as needed.
• Patient History Management: Introduced get_patient_history_fields, insert_patient_history, and get_patient_history methods in AtriumSDK, allowing getting and setting of patient meta information which changes over time (e.g., height and weight).
• Updated get_patient_info: You can now specify a time where the function will get you the nearest height/weight to the time specified.
• Updated insert_patient: Now when inserting a patient with a height and weight it will also make patient_history entries for the height/weight entered at the current insertion time.
• Introducing Labels functionality: Now you can input time series labels to the sdk. This includes entering label names the define a label then applying that label to sections of data with a start and end time or as a point label. Deletion of labels is also supported. The label functionality is also supported in API mode except for insert and delete.
• Label Hierarchies: Label names can have parents, enabling the creation of a label hierarchy tree. This permits fetching all descendants of a label, enhancing data categorization. This feature is supported by new methods: get_all_label_name_descendents, get_label_name_children, and get_label_name_parent as well as an added parent param in insert_label_name.
• Expanded Dataset Definition: The dataset definition file format and object have been expanded to include the specification of label names.
• Dataset Definition Validator: Now maps all data regions to both a device and a patient whenever possible, preventing data leakage and speeding up patient-based queries.
• Dataset Iterator Enhancements: The DatasetIterator class now passes label information along with the window object yielded by the iterator. It has also introduced FilteredDatasetIterator and RandomAccessDatasetIterator subclasses for enhanced data iteration.
• Dataset Partitioning: Added partition_dataset function for creating stratified dataset partitions based on specified ratios, ensuring balanced label representation without information leakage.
• Data Exports and Transfers: Data transfer functionalities now leverage Dataset Definition files, supporting multiple formats (tsc, numpy, csv, wfdb) and including labels and patient histories in transfers.
• New CLI Commands: Added commands atriumdb refresh_token and atriumdb config for token refreshing and configuration display.
• Upgrated CLI Export: Updated atriumdb export to utilize Dataset Definition YAML files for defining data transfer/export operations.
• updated write data: The stored procedure in write_data now has a gap tolerance so when you write you can specify a gap tolerance for the intervals. This will merge the intervals with older intervals based on your gap tolerance.
• Scale factor application upgrade: The scale factor application when getting data was moved to the C code speeding up decompression time. This means the old libTSC.dll and libTSC.so files will no longer work so if you manually compiled them you will have to do it again. If installing from wheel file the new files are already included.

Fixed

• Addressed a bug in insert_labels function causing failures with NULL label_source in MariaDB, by adjusting SQL query construction.
• Centralized some identical SQL functions/queries in SQLHandler, reducing redundancy between MariaDBHandler and SqliteHandler.
• Optimized time unit conversion methods by using a globally defined time units dictionary for all relevant conversions.
• Addressed bug with the measure/device cache being accessed before its creation.

Removed

• REST calls serviced remote get_data commands, replaced with Websockets.
• Removed atriumdb import command, since atriumdb export does the same thing with a different argument order.
• Removed unnecessary short private helper methods by integrating their logic directly into the calling methods.

v2.1.1

Changes:

• 5bf6c95 Merged PR 187: CLI Authentication Bugfix

This list of changes was auto generated.

v2.1.0

Changes:

• 9786b5e Merged PR 178: bump version number
• a97f07f bump version number
• 92ae2fa Merged PR 172: Give both device and patient information for all windows. And enforce windows not sharing two devices or two patients.
• 9340c2c Merged PR 171: Add unit testing for iterators
• cbbdf4d Merge remote-tracking branch 'origin/testing/iterator' into testing/iterator
• 98f5714 Add iterator test yamls to the definition validation test.
• 2680b84 Remove print.
• 8aa98b7 Comment out test file creation (only needs to be called once and should come with the repo)
• ccf78ef Add iterator unit tests and associated files.
• ffc543c allow freq_nhz as a key.

See More

• 5c39ae9 Merged PR 167: Allow empty dictionaries in the Definition file or object arguments.
• 8bfbe84 Add iterator test yamls to the definition validation test.
• ab47c09 Remove print.
• 3b66694 Comment out test file creation (only needs to be called once and should come with the repo)
• 53ee1e1 Add iterator unit tests and associated files.
• a41d434 allow freq_nhz as a key.
• 8df7b91 Add two types of definition testing with omitted and empty patient sources.
• 6a2f297 Allow empty dictionaries in the Definition file or object arguments.
• c15cfb6 Merged PR 166: pass copies of the times/values in the signal dictionary.
• 0ce013d pass copies of the times/values in the signal dictionary.
• 710ef04 Merged PR 165: Iterator Error for Missing Measures
• d74493d If requested measure not present in the measure table, raise Error.
• 1582017 Merged PR 138: AtriumDB Datasets
• 6c39a1c Merged PR 149: remove 'select *' queries so they dont break backwards compatibility
• 08b3875 remove 'select *' queries so they dont break backwards compatibility between the sdk and datasets created with an older schema. Also fix some broken tests
• a7ec95b Merged PR 132: Fix sqlite patient mapping bug.
• 851969c Fix sqlite patient mapping bug.
• bba99dd Merged PR 127: Add height and weight for patients
• 0a2149e Merge remote-tracking branch 'origin/height-weight' into height-weight
• 4850588 fix weight/height order
• c9b7779 fix sqlite get patients query
• ff7026b fix get all patients
• 0d0716e add sqlite triggers to sqlite handler
• 13463ba update patient related functions to include height and weight
• c7fa7b7 add height and weight to sqlite tables
• 54ad41c add sqlite triggers and replace cascade delete trigger with on del cascade in foreign key def
• be8a888 add patient history table, update current census, patient and log hl7 tables for height and weight
• bc8978d fix weight/height order
• ba15e7f Merged PR 114: Add interval index options to write_data
• 04b4d3b fix sqlite get patients query
• 2822495 fix get all patients
• 1dd4a1a add sqlite triggers to sqlite handler
• efcdba0 update patient related functions to include height and weight
• 6e0809c add height and weight to sqlite tables
• c79f143 add sqlite triggers and replace cascade delete trigger with on del cascade in foreign key def
• acbcc2a add patient history table, update current census, patient and log hl7 tables for height and weight
• 8a82bff Put default interval_mode in source code instead of config.toml to simplify defaults.
• 8061d6c Fix docstring formatting.
• ad87808 Simplify imports
• f87f5ab Merge branch 'main' into feature/disable-write-intervals
• f0994cc Remove interval_index_mode from write_data_easy
• 492e245 default mode merge.
• 2735bf5 Merged PR 113: Optimized CWF Functionality
• f218ffb Allow the default interval index mode to be set in the config.toml
• 4f34aeb Add interval_index param docstring.
• d7c516a Add interval_index param docstring.
• 995f5f5 Fixed Types.
• 390190d Merged PR 115: Add context for measure_ids that aren't found.
• 134536a Fixed Types.
• 20be3dd Complete the WindowConfig.init docstring.
• be74d7d Merged PR 116: release-pipeline
• 6979b99 Update release.yaml for Azure Pipelines
• 0f9de82 Update release.yaml for Azure Pipelines
• 10c8e3a Update release.yaml for Azure Pipelines
• c3900d8 Add context for measure_ids that aren't found.
• cdd2b5e Include 3 possible modes for interval_index writing.
• 18c7fe4 propogate interval_index option to write_data_easy.
• 1f1e816 Allow disabling of interval_index.
• 65fb6b0 Allow integer freq
• a2c1595 Add union intervals code
• cb9f4bc fix search sorted
• 68dc706 Use Numpy optimization.
• 2b1acfe Use Union for Backwards Compatibility.
• 6463e2f Use "measure names" instead of tuples as the signal key.
• 659d791 Merge remote-tracking branch 'origin/main' into feature/cwf
• f2ed04c Allow end_times to be set in CWF
• 989a853 Add WindowConfig to package init.
• 194ac19 Add Patient Mapping for Windows.
• 95d840b Comment test
• 37df5b5 Fix cwf and add test with reasonable checks.
• ee2b5ef Begin logic for get_windows
• 1eb542c Create a test file
• 1428cb8 Import windowing
• bcee327 Add Base Windowing Files.

This list of changes was auto generated.

v2.0.0

Changes:

• be74d7d Merged PR 116: release-pipeline
• 6979b99 Update release.yaml for Azure Pipelines
• 0f9de82 Update release.yaml for Azure Pipelines
• 10c8e3a Update release.yaml for Azure Pipelines
• 644030a Update release.yaml for Azure Pipelines
• 52313f2 Update release.yaml for Azure Pipelines
• 347c430 Update release.yaml for Azure Pipelines
• ca29563 Updated pyproject.toml
• 7294b0e Update release.yaml for Azure Pipelines
• ebcb562 Update github-sync.yaml for Azure Pipelines

See More

• 7caf0dc Update github-sync.yaml for Azure Pipelines
• 6d58b1f Update github-sync.yaml for Azure Pipelines
• acfdad5 Update github-sync.yaml for Azure Pipelines
• 55be476 Update github-sync.yaml for Azure Pipelines
• edf75c4 Update github-sync.yaml for Azure Pipelines
• 9595ae5 Update github-sync.yaml for Azure Pipelines
• cb3c9c8 Update github-sync.yaml for Azure Pipelines
• cc156b8 Update github-sync.yaml for Azure Pipelines
• 5b9197f Update github-sync.yaml for Azure Pipelines
• a06a234 Update github-sync.yaml for Azure Pipelines
• 8998bff Update github-sync.yaml for Azure Pipelines
• 8d9edcc Update github-sync.yaml for Azure Pipelines
• 5078eb5 Update github-sync.yaml for Azure Pipelines
• 6721e9e Update github-sync.yaml for Azure Pipelines
• 1006b63 Update github-sync.yaml for Azure Pipelines
• 05f38d3 Update github-sync.yaml for Azure Pipelines
• 3807afa Updated pyproject.toml
• faa35b1 Update release.yaml for Azure Pipelines
• 5353a85 Update github-sync.yaml for Azure Pipelines
• 47b6f27 Update release.yaml for Azure Pipelines
• 8ed57d6 Update github-sync.yaml for Azure Pipelines
• fe1ac75 Update github-sync.yaml for Azure Pipelines
• 763e734 Update github-sync.yaml for Azure Pipelines
• 49abaca Update github-sync.yaml for Azure Pipelines
• 4650931 Update github-sync.yaml for Azure Pipelines
• 882e180 Update github-sync.yaml for Azure Pipelines
• e5113e8 Update github-sync.yaml for Azure Pipelines
• 3b31008 Update github-sync.yaml for Azure Pipelines
• f30a197 Update github-sync.yaml for Azure Pipelines
• e54ccb4 Update github-sync.yaml for Azure Pipelines
• 6744fe3 Update github-sync.yaml for Azure Pipelines
• a929a15 Added github-sync.yaml
• 2cc0555 Update release.yaml for Azure Pipelines
• c334dad Update release.yaml for Azure Pipelines
• a20fac0 Update release.yaml for Azure Pipelines
• 4504e36 Update release.yaml for Azure Pipelines
• 6532628 Update release.yaml for Azure Pipelines
• 392bf4a Update release.yaml for Azure Pipelines
• 9322b90 Update release.yaml for Azure Pipelines
• 213a728 Update release.yaml for Azure Pipelines
• 3138c47 Update release.yaml for Azure Pipelines
• 54f536f Update release.yaml for Azure Pipelines
• d5a5c77 Updated pyproject.toml
• d0c537b Updated pyproject.toml
• dd3ac8d Updated pyproject.toml
• b30a551 Updated pyproject.toml
• 8cd31c0 Updated pyproject.toml
• ed0e01e Updated pyproject.toml
• a7286e0 Update release.yaml for Azure Pipelines
• 5a1c772 Updated pyproject.toml
• 8ca8d6e Update release.yaml for Azure Pipelines
• f0f1d9f Update release.yaml for Azure Pipelines
• 6e325d4 Update release.yaml for Azure Pipelines
• 2bbdd93 Update release.yaml for Azure Pipelines
• 3d689a3 Update release.yaml for Azure Pipelines
• bd23659 Update release.yaml for Azure Pipelines
• 554cf8a Update release.yaml for Azure Pipelines
• cd153bc Update release.yaml for Azure Pipelines
• cf021e8 Update release.yaml for Azure Pipelines
• db33f70 Update release.yaml for Azure Pipelines
• ccdf5b1 Update release.yaml for Azure Pipelines
• feb175c Update release.yaml for Azure Pipelines
• acb5fd7 Update release.yaml for Azure Pipelines
• 1d4ff52 Update release.yaml for Azure Pipelines
• 6172068 Update release.yaml for Azure Pipelines
• 9af361f Update release.yaml for Azure Pipelines
• e23ec87 Update release.yaml for Azure Pipelines
• 7e2df3b Set up CI with Azure Pipelines
• d3c7551 initial_commit
• 524f4ae Merged PR 109: updated readme and other things for open source release
• 89966d2 fix chmod and directory bugs
• 04e14e0 fix chmod and directory bugs
• 737bbb8 add atriumdb tag
• b46ad6f add readme for sdk on pypi
• 494df4f updated toml file with pypi info
• 7ef4205 make separate sdk readme for pypi. update docs,
• ed5a17f update readme and how C is compiled by making c paths relative
• 6f027f4 Merged PR 105: Cast all measure, device and patient ids received from the API to int.
• 3f7aee1 Merged PR 106: Pass forward the sorting parameter to get_data_api.
• b618df4 Merged PR 104: Add perpetual invite link for Slack.
• f191296 Pass forward the sorting parameter to get_data_api.
• 8d5af33 Cast all measure, device and patient ids received from the API to int.
• 93e9e82 Add perpetual invite link for Slack.
• e385b77 Merged PR 103: added part to docs about end-time not being inclusive to get data
• eba6faf added part about end-time not being inclusive to get data
• 14b5613 Merged PR 101: Doc fixes
• eb2d04d Change to correct Copyright.
• deaf535 Merged PR 100: Change default overwrite behavior to not error on duplicate data.
• 625de76 Merged PR 99: Fix Gaps on the Block Bug
• 4ae8c4c Change header type.
• 1bda641 Change header type.
• aea835f Change Capitalization of AtriumDB.
• 4a78f05 Remove broken search, and relegate Index to subheader of API ref.
• 81845cf Fix formatting on autodoc.
• ec04afe Change default overwrite behavior to not error on duplicate data.
• 8a7682a Merge remote-tracking branch 'origin/main' into bug/small-block
• de3d065 Merged PR 98: update c-build container and instructions for new repo structure
• 7aa2bc1 Merged PR 97: Comprehensive Test and Small Block Bug
• 9ea2e52 Clean up test.
• 8882c9d Write experiment to better understand bug.
• 8ef4d22 Add the ability to specify no_pool in create dataset.
• 55bfc3e Change the gap to timestamp C algorithm to handle gaps at index 0.
• bcaaed7 update c-build container and instructions for new repo structure
• eaf7c4a Alter Test
• 879bdf1 Add visualization
• 1c210a9 Add more verbose errors.
• b36af35 Setup a test that displays the time type 2 small block bug.
• c7180e4 Add for 1 signa...