Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend do_nd with option to perform "Parallel sweeps" #4325

Merged
merged 94 commits into from Sep 30, 2022
Merged
Show file tree
Hide file tree
Changes from 82 commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
c74c8a5
Add MultiSweep wip
jenshnielsen Jun 27, 2022
dfe45f3
Wip adapt for parallel sweeps
jenshnielsen Jun 27, 2022
0b7a4f3
make names unique
jenshnielsen Jun 28, 2022
906b76f
fix unpacking ofr parameters
jenshnielsen Jun 29, 2022
92144c6
Add additional setpoints to parameter groups
jenshnielsen Jun 29, 2022
52dc56d
dond fix typecheck
jenshnielsen Jul 1, 2022
993e41e
test dond cleanup imports
jenshnielsen Jul 5, 2022
658986b
Start refactoring measure/sweep logic into class
jenshnielsen Jul 8, 2022
318bcb9
wip loop over sweeper
jenshnielsen Sep 16, 2022
ebf2f44
only set when required
jenshnielsen Sep 16, 2022
edd52e1
wip fixes
jenshnielsen Sep 19, 2022
4325b3f
fix type checking
jenshnielsen Sep 19, 2022
51c12d4
functional way of mapping setpoints to meas parameters
jenshnielsen Sep 19, 2022
58ec711
add basic test of multisweep with complex dependencies
jenshnielsen Sep 19, 2022
ad30fe5
split dond tests into different files
jenshnielsen Sep 19, 2022
bc10f4b
tighten type to avoid an assert
jenshnielsen Sep 19, 2022
d253e05
remove unused results from dataclass
jenshnielsen Sep 19, 2022
e3595fa
deprecate unused method
jenshnielsen Sep 19, 2022
1507c27
use public group member
jenshnielsen Sep 19, 2022
757e9df
use tuples for parameters
jenshnielsen Sep 19, 2022
5e79e25
validate mapping before use
jenshnielsen Sep 19, 2022
29583d7
remove unused property
jenshnielsen Sep 19, 2022
38be051
Allow MultiSweep passed to dond
jenshnielsen Sep 20, 2022
c6c3eb2
Add basic validation and test of MultiSweep
jenshnielsen Sep 20, 2022
b8e86dc
Adoid regenerating setpoints
jenshnielsen Sep 20, 2022
02daff4
test output of _Sweeper
jenshnielsen Sep 20, 2022
0b18dbb
add tests for errors on wrong mapping
jenshnielsen Sep 20, 2022
75df71b
move multisweep
jenshnielsen Sep 21, 2022
636a24d
fix typo in error message
jenshnielsen Sep 21, 2022
ca9c73f
Check against the correct error
jenshnielsen Sep 21, 2022
49f6a73
ignore overstrict type error
jenshnielsen Sep 21, 2022
322106c
add basic test with parameterwithsetpoints and multisweep
jenshnielsen Sep 21, 2022
7d6baef
dond pass measurement name to dataset
jenshnielsen Sep 21, 2022
a9082d3
Add test of multiple meas names
jenshnielsen Sep 21, 2022
f4a343f
Update shapes type to not include None
jenshnielsen Sep 21, 2022
b70953b
dont set unrelated shapes in dataset
jenshnielsen Sep 21, 2022
4421b8b
Add docstring to MultiSweep
jenshnielsen Sep 21, 2022
694a204
set len of sweep so progess can be updated by tqdm
jenshnielsen Sep 21, 2022
94b2cad
ensure that mapping can also contain non tuples
jenshnielsen Sep 21, 2022
937acb5
Add multisweep examples (without text)
jenshnielsen Sep 21, 2022
831bc45
correctly filter multparameter
jenshnielsen Sep 22, 2022
11492aa
Remove shape validation
jenshnielsen Sep 22, 2022
a82f34d
Default to measure multisweep as a function of all parameters
jenshnielsen Sep 22, 2022
083d67d
Update docs to reflect multisweep
jenshnielsen Sep 22, 2022
0583400
run darker
jenshnielsen Sep 22, 2022
308ab9f
Update qcodes/dataset/threading.py
jenshnielsen Sep 23, 2022
5718522
Update qcodes/dataset/dond/do_nd_utils.py
jenshnielsen Sep 23, 2022
9a8be26
rename variable for clarity
jenshnielsen Sep 23, 2022
817d276
Let MultiSweep take a *list
jenshnielsen Sep 23, 2022
e35a19b
Remove repr
jenshnielsen Sep 23, 2022
769779e
rename together class
jenshnielsen Sep 23, 2022
6ff18cc
rename dataset_mapping to dataset_dependencies
jenshnielsen Sep 23, 2022
c59a3a9
Add test that ArraySweep can handle ints
jenshnielsen Sep 23, 2022
52744d1
add changes for 4325
jenshnielsen Sep 23, 2022
e7c051a
Update qcodes/dataset/dond/do_nd.py
jenshnielsen Sep 23, 2022
5dacef7
Apply suggestions from code review
jenshnielsen Sep 24, 2022
b80d71c
rename private variable to reflect public one
jenshnielsen Sep 24, 2022
07ab0fc
rename for consistency
jenshnielsen Sep 24, 2022
5adc9be
add missing elipeces to typeS
jenshnielsen Sep 24, 2022
bd578c0
rename variables
jenshnielsen Sep 24, 2022
5ff9cae
cleanup sweepmeasgroup
jenshnielsen Sep 24, 2022
a41e0bf
reflect updates in tests
jenshnielsen Sep 24, 2022
02aef18
handle no and mixed groups of meas params
jenshnielsen Sep 24, 2022
cc8632c
Also test all list
jenshnielsen Sep 26, 2022
9a06075
check that we get int output from int data
jenshnielsen Sep 26, 2022
9d2a6fe
make error more explicit
jenshnielsen Sep 26, 2022
e45f0a7
Apply suggestions from code review
jenshnielsen Sep 26, 2022
552485f
extend sweeper test to cover 3 combined paramters
jenshnielsen Sep 26, 2022
fc6664e
Allow all permutations of parameters from a TogetherSweep
jenshnielsen Sep 26, 2022
cc4305b
typos
jenshnielsen Sep 26, 2022
4b9b4bf
Change datset_dependencies to a dict from name to parameters
jenshnielsen Sep 26, 2022
351e113
Add test with explicit mapping and parameterwithsetpoints
jenshnielsen Sep 26, 2022
b655a4f
Add missing types
jenshnielsen Sep 27, 2022
b6d8a53
Use new api in example
jenshnielsen Sep 27, 2022
a5f4bba
combine measure and sweep measure classes
jenshnielsen Sep 28, 2022
d7773f1
Change dependency logic
jenshnielsen Sep 28, 2022
6d6802f
update dataset_mapping example
jenshnielsen Sep 28, 2022
f7d25b1
fix typing
jenshnielsen Sep 28, 2022
335ce5b
fix typos
jenshnielsen Sep 28, 2022
e8d7c9c
make branching logic total
jenshnielsen Sep 28, 2022
091c478
simplify parameter expansion
jenshnielsen Sep 29, 2022
e110b2e
fix some warnings from pylint
jenshnielsen Sep 29, 2022
39221c6
We need not sweep over floats
jenshnielsen Sep 29, 2022
caa5f4a
add missing tests for error cases
jenshnielsen Sep 29, 2022
f4403fa
add basic test for post actions being called
jenshnielsen Sep 29, 2022
cc09b03
Add test that extra log info is added to log
jenshnielsen Sep 29, 2022
9bbf811
Apply suggestions from code review
jenshnielsen Sep 30, 2022
cea1665
Update qcodes/dataset/dond/do_nd.py
jenshnielsen Sep 30, 2022
ed9e189
adapt to changes from review
jenshnielsen Sep 30, 2022
ff8422b
Add test for default logging info
jenshnielsen Sep 30, 2022
d86e35c
Add test for multipe parameters in one ds
jenshnielsen Sep 30, 2022
1a5cfc5
correct black whitespace
jenshnielsen Sep 30, 2022
c8b35c5
simplify looping logic
jenshnielsen Sep 30, 2022
523a063
break too long lines
jenshnielsen Sep 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/changes/newsfragments/4325.improved
@@ -0,0 +1,2 @@
``dond`` has gained the ability to sweep multiple parameters at the same time enabling the creation of more complex
measurements.

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion qcodes/dataset/__init__.py
Expand Up @@ -18,7 +18,7 @@
from .dond.do_1d import do1d
from .dond.do_2d import do2d
from .dond.do_nd import dond
from .dond.sweeps import AbstractSweep, ArraySweep, LinSweep, LogSweep
from .dond.sweeps import AbstractSweep, ArraySweep, LinSweep, LogSweep, TogetherSweep
from .experiment_container import (
experiments,
load_experiment,
Expand Down Expand Up @@ -54,6 +54,7 @@
"LinSweep",
"LogSweep",
"Measurement",
"TogetherSweep",
"ParamSpec",
"SQLiteSettings",
"SequentialParamsCaller",
Expand Down
8 changes: 4 additions & 4 deletions qcodes/dataset/data_set.py
Expand Up @@ -310,7 +310,7 @@ def prepare(
*,
snapshot: Mapping[Any, Any],
interdeps: InterDependencies_,
shapes: Shapes = None,
shapes: Shapes | None = None,
parent_datasets: Sequence[Mapping[Any, Any]] = (),
write_in_background: bool = False,
) -> None:
Expand Down Expand Up @@ -531,9 +531,9 @@ def toggle_debug(self) -> None:
self.conn.close()
self.conn = connect(path_to_db, self._debug)

def set_interdependencies(self,
interdeps: InterDependencies_,
shapes: Shapes = None) -> None:
def set_interdependencies(
self, interdeps: InterDependencies_, shapes: Shapes | None = None
) -> None:
"""
Set the interdependencies object (which holds all added
parameters and their relationships) of this dataset and
Expand Down
4 changes: 2 additions & 2 deletions qcodes/dataset/data_set_in_memory.py
Expand Up @@ -398,7 +398,7 @@ def prepare(
*,
snapshot: Mapping[Any, Any],
interdeps: InterDependencies_,
shapes: Shapes = None,
shapes: Shapes | None = None,
parent_datasets: Sequence[Mapping[Any, Any]] = (),
write_in_background: bool = False,
) -> None:
Expand Down Expand Up @@ -700,7 +700,7 @@ def _set_parent_dataset_links(self, links: list[Link]) -> None:
self._parent_dataset_links = links

def _set_interdependencies(
self, interdeps: InterDependencies_, shapes: Shapes = None
self, interdeps: InterDependencies_, shapes: Shapes | None = None
) -> None:
"""
Set the interdependencies object (which holds all added
Expand Down
2 changes: 1 addition & 1 deletion qcodes/dataset/data_set_protocol.py
Expand Up @@ -85,7 +85,7 @@ def prepare(
*,
snapshot: Mapping[Any, Any],
interdeps: InterDependencies_,
shapes: Shapes = None,
shapes: Shapes | None = None,
parent_datasets: Sequence[Mapping[Any, Any]] = (),
write_in_background: bool = False,
) -> None:
Expand Down
27 changes: 4 additions & 23 deletions qcodes/dataset/descriptions/rundescriber.py
Expand Up @@ -30,14 +30,14 @@ class RunDescriber:
attributes when converting itself to dictionary.
"""

def __init__(self, interdeps: InterDependencies_,
shapes: Shapes = None) -> None:
def __init__(
self, interdeps: InterDependencies_, shapes: Shapes | None = None
) -> None:

if not isinstance(interdeps, InterDependencies_):
raise ValueError('The interdeps arg must be of type: '
'InterDependencies_. '
f'Got {type(interdeps)}.')
self._verify_interdeps_shape(interdeps, shapes)

self._interdeps = interdeps

Expand All @@ -49,32 +49,13 @@ def version(self) -> int:
return self._version

@property
def shapes(self) -> Shapes:
def shapes(self) -> Shapes | None:
return self._shapes

@property
def interdeps(self) -> InterDependencies_:
return self._interdeps

@staticmethod
def _verify_interdeps_shape(interdeps: InterDependencies_,
jenshnielsen marked this conversation as resolved.
Show resolved Hide resolved
shapes: Shapes) -> None:
"""
Verify that interdeps and shape are consistent
"""
for dependent, dependencies in interdeps.dependencies.items():
if shapes is not None:
shape = shapes.get(dependent.name)
if shape is not None:
if len(shape) < len(dependencies):
raise ValueError(f"Found inconsistency between "
f"InterDependencies and shape "
f"metadata. "
f"{dependent.name} has "
f"{len(dependencies)} dependencies "
f"but it's shape "
f"is given as {shape}")

def _to_dict(self) -> RunDescriberV3Dict:
"""
Convert this object into a dictionary. This method is intended to
Expand Down
4 changes: 2 additions & 2 deletions qcodes/dataset/descriptions/versioning/rundescribertypes.py
Expand Up @@ -33,7 +33,7 @@ class InterDependencies_Dict(TypedDict):
standalones: list[str]


Shapes = Optional[Dict[str, Tuple[int, ...]]]
Shapes = Dict[str, Tuple[int, ...]]


class RunDescriberV0Dict(TypedDict):
Expand All @@ -51,7 +51,7 @@ class RunDescriberV2Dict(RunDescriberV0Dict):


class RunDescriberV3Dict(RunDescriberV2Dict):
shapes: Shapes
shapes: Shapes | None
# dict from dependent to dict from depenency to num points in grid


Expand Down
2 changes: 1 addition & 1 deletion qcodes/dataset/dond/do_0d.py
Expand Up @@ -69,7 +69,7 @@ def do0d(
)

try:
shapes: Shapes = detect_shape_of_measurement(
shapes: Shapes | None = detect_shape_of_measurement(
measured_parameters,
)
except TypeError:
Expand Down
4 changes: 3 additions & 1 deletion qcodes/dataset/dond/do_1d.py
Expand Up @@ -114,7 +114,9 @@ def do1d(
)
try:
loop_shape = (num_points,) + tuple(1 for _ in additional_setpoints)
shapes: Shapes = detect_shape_of_measurement(measured_parameters, loop_shape)
shapes: Shapes | None = detect_shape_of_measurement(
measured_parameters, loop_shape
)
except TypeError:
LOG.exception(
f"Could not detect shape of {measured_parameters} "
Expand Down
4 changes: 3 additions & 1 deletion qcodes/dataset/dond/do_2d.py
Expand Up @@ -138,7 +138,9 @@ def do2d(

try:
loop_shape = (num_points1, num_points2) + tuple(1 for _ in additional_setpoints)
shapes: Shapes = detect_shape_of_measurement(measured_parameters, loop_shape)
shapes: Shapes | None = detect_shape_of_measurement(
measured_parameters, loop_shape
)
except TypeError:
LOG.exception(
f"Could not detect shape of {measured_parameters} "
Expand Down