-
Notifications
You must be signed in to change notification settings - Fork 1
Add load_field and load_group to allow loading custom entries using GenericNeXusWorkflow #278
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
681179c
50ad6a0
43df02f
ca661a1
b95ef2f
8fbb233
f5a448e
49b4b4c
e18cd6b
c43c1b0
0f3fd11
c6e1bf8
f19dd47
f4b1611
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||
|---|---|---|---|---|
|
|
@@ -4,11 +4,12 @@ | |||
| from pathlib import Path | ||||
|
|
||||
| import pytest | ||||
| import sciline as sl | ||||
| import scipp as sc | ||||
| import scippnexus as snx | ||||
| from scipp.testing import assert_identical | ||||
|
|
||||
| from ess.reduce.nexus import compute_component_position, workflow | ||||
| from ess.reduce.nexus import compute_component_position, load_from_path, workflow | ||||
| from ess.reduce.nexus.types import ( | ||||
| BackgroundRun, | ||||
| Beamline, | ||||
|
|
@@ -21,6 +22,8 @@ | |||
| Measurement, | ||||
| MonitorType, | ||||
| NeXusComponentLocationSpec, | ||||
| NeXusFileSpec, | ||||
| NeXusLocationSpec, | ||||
| NeXusName, | ||||
| NeXusTransformation, | ||||
| PreopenNeXusFile, | ||||
|
|
@@ -577,7 +580,7 @@ def test_load_histogram_monitor_workflow(dream_coda_test_file: Path) -> None: | |||
|
|
||||
|
|
||||
| def test_load_detector_workflow(loki_tutorial_sample_run_60250: Path) -> None: | ||||
| wf = LoadDetectorWorkflow(run_types=[SampleRun], monitor_types=[]) | ||||
| wf = LoadDetectorWorkflow(run_types=[SampleRun]) | ||||
| wf[Filename[SampleRun]] = loki_tutorial_sample_run_60250 | ||||
| wf[NeXusName[snx.NXdetector]] = 'larmor_detector' | ||||
| da = wf.compute(RawDetector[SampleRun]) | ||||
|
|
@@ -587,7 +590,7 @@ def test_load_detector_workflow(loki_tutorial_sample_run_60250: Path) -> None: | |||
|
|
||||
|
|
||||
| def test_load_histogram_detector_workflow(tbl_commissioning_orca_file: Path) -> None: | ||||
| wf = LoadDetectorWorkflow(run_types=[SampleRun], monitor_types=[]) | ||||
| wf = LoadDetectorWorkflow(run_types=[SampleRun]) | ||||
| wf[Filename[SampleRun]] = tbl_commissioning_orca_file | ||||
| wf[NeXusName[snx.NXdetector]] = 'orca_detector' | ||||
| da = wf.compute(RawDetector[SampleRun]) | ||||
|
|
@@ -600,7 +603,7 @@ def test_load_histogram_detector_workflow(tbl_commissioning_orca_file: Path) -> | |||
| def test_load_empty_histogram_detector_workflow( | ||||
| tbl_commissioning_orca_file: Path, | ||||
| ) -> None: | ||||
| wf = LoadDetectorWorkflow(run_types=[SampleRun], monitor_types=[]) | ||||
| wf = LoadDetectorWorkflow(run_types=[SampleRun]) | ||||
| wf[Filename[SampleRun]] = tbl_commissioning_orca_file | ||||
| wf[NeXusName[snx.NXdetector]] = 'orca_detector' | ||||
| da = wf.compute(EmptyDetector[SampleRun]) | ||||
|
|
@@ -776,3 +779,48 @@ def assert_not_contains_type_arg(node: object, excluded: set[type]) -> None: | |||
| assert not any( | ||||
| arg in excluded for arg in getattr(node, "__args__", ()) | ||||
| ), f"Node {node} contains one of {excluded!r}" | ||||
|
|
||||
|
|
||||
| def test_generic_nexus_workflow_load_custom_field_user_affiliation( | ||||
| loki_tutorial_sample_run_60250: Path, | ||||
| ) -> None: | ||||
| class UserAffiliation(sl.Scope[RunType, str], str): | ||||
| """User affiliation.""" | ||||
|
|
||||
| def load_user_affiliation( | ||||
| file: NeXusFileSpec[RunType], path: NeXusName[UserAffiliation[RunType]] | ||||
| ) -> UserAffiliation[RunType]: | ||||
| return UserAffiliation[RunType]( | ||||
| load_from_path(NeXusLocationSpec(filename=file.value, component_name=path)) | ||||
| ) | ||||
|
|
||||
| wf = GenericNeXusWorkflow(run_types=[SampleRun], monitor_types=[]) | ||||
| wf.insert(load_user_affiliation) | ||||
| wf[Filename[SampleRun]] = loki_tutorial_sample_run_60250 | ||||
| # Path is relative to the top-level '/entry' | ||||
| wf[NeXusName[UserAffiliation[SampleRun]]] = 'user_0/affiliation' | ||||
| affiliation = wf.compute(UserAffiliation[SampleRun]) | ||||
| assert affiliation == 'ESS' | ||||
|
|
||||
|
|
||||
| def test_generic_nexus_workflow_load_custom_group_user( | ||||
| loki_tutorial_sample_run_60250: Path, | ||||
| ) -> None: | ||||
| class UserInfo(sl.Scope[RunType, str], str): | ||||
| """User info.""" | ||||
|
|
||||
| def load_user_info( | ||||
| file: NeXusFileSpec[RunType], path: NeXusName[UserInfo[RunType]] | ||||
| ) -> UserInfo[RunType]: | ||||
| return UserInfo[RunType]( | ||||
| load_from_path(NeXusLocationSpec(filename=file.value, component_name=path)) | ||||
| ) | ||||
|
|
||||
| wf = GenericNeXusWorkflow(run_types=[SampleRun], monitor_types=[]) | ||||
| wf.insert(load_user_info) | ||||
| wf[Filename[SampleRun]] = loki_tutorial_sample_run_60250 | ||||
| # Path is relative to the top-level '/entry' | ||||
| wf[NeXusName[UserInfo]] = 'user_0' | ||||
| user_info = wf.compute(UserInfo[SampleRun]) | ||||
| assert user_info['affiliation'] == 'ESS' | ||||
| assert user_info['name'] == 'John Doe' | ||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should ideally be handled similarly to
and load one or more https://github.com/scipp/scippneutron/blob/3c49525dd89af68d375119c6f9072008f337dc6c/src/scippneutron/metadata/_model.py#L168 Does your code provide the basis for doing that?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So are you suggesting that instead of having class UserAffiliation(sl.Scope[RunType, str], str):
"""User affiliation."""
def load_user_affiliation(
file: NeXusFileSpec[RunType],
path: NeXusName[UserAffiliation[RunType]],
) -> UserAffiliation[RunType]:
return UserAffiliation[RunType](load_field(file, path))I would need to create a pydantic model for the I guess I could but it feels a bit overkill? In addition, I already felt it was annoying to have to make changes in
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You don't need to make a model for I think that the generic workflow should be able to load all users and provide a way to select one. (by index, path, or name, probably) Especially because this will be relevant to all workflows in some way. And please don't misunderstand me, I think it is good to have tools for loading anything from the file in a simple way. But people related info is used in a lot of places. So I think we should have a common, robust, and flexible solution.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So, did I understand correctly that your objection was not adding I just picked that field randomly, to illustrate that we can load something custom from the file. Is it better if I pick something else to load?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Correct. I am fine with the example and test case. My comment was only about checking whether this implementation is useful for implementing a loader for |
||||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure why we needed to have the monitor_types here, when we are loading detectors?
So I removed it.