Skip to content

Add use_stored_data method#34

Merged
DanPorter merged 3 commits intomasterfrom
33-hdfmapeval-loading-data-from-previous-file-not-current-one
Oct 14, 2025
Merged

Add use_stored_data method#34
DanPorter merged 3 commits intomasterfrom
33-hdfmapeval-loading-data-from-previous-file-not-current-one

Conversation

@DanPorter
Copy link
Copy Markdown
Collaborator

Option to turn on and off local data loading in hdfmap.

eval_functions.py

  • Add option use_stored_data to prepare_expression_load_data()
  • populate data_namspace in generate_namespace(), don't populate defaults.
  • load data from file based on identifiers. Remove identifiers if they are in data_namespace and use_stored_data is true.
  • propagate option to eval and format_hdf

hdfmap_class.py

  • propagate option to eval and format_hdf
  • add _use_local_data parameter
  • add use_local_data function

test_many_files.py

  • add test to check that this works in both scenarios.

tests

  • all tests pass.
  • updated some tests so they are note comparing absolute file paths.

Option to turn on and off local data in hdfmap.

Fix for #33

eval_functions.py
 - Add option use_stored_data to prepare_expression_load_data()
 - populate data_namspace in generate_namespace(), don't populate defaults.
 - load data from file based on identifiers. Remove identifiers if they are in data_namespace and use_stored_data is true.
 - propagate option to eval and format_hdf

hdfmap_class.py
 - propagate option to eval and format_hdf
 - add _use_local_data parameter
 - add use_local_data function

test_many_files.py
 - add test to check that this works in both scenarios.

tests
 - tests pass.
 - 2 problems, unrelated I think (windows PC):
  - test_hdf_compare. Expected 26879, got 26967
  - test_create_metadata_list. Expected 5280, got 5342
fixed tests so they don't compare absolute filepaths, which can change.
fixed tests so they don't compare absolute filepaths, which can change.
@DanPorter DanPorter linked an issue Oct 14, 2025 that may be closed by this pull request
@DanPorter DanPorter merged commit 6b3ef01 into master Oct 14, 2025
10 checks passed
@DanPorter DanPorter deleted the 33-hdfmapeval-loading-data-from-previous-file-not-current-one branch October 14, 2025 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

hdfmap.eval loading data from previous file, not current one

1 participant