fetch data sets based on run_id. Modif. in select_runs

[mlotfiben]

allow for selection of data set by giving one specific run_id and also it can be used with matching a given numbers of run_ids (example: run_id="110*").

[JelleAalbers]

Thanks Lotfi. It currently crashes because there is not yet a run_id argument in the select_runs function. You can add one with a default of None (and update the docstring).

It seems the logic for run_id is very similar to that of run_mode. Maybe it would work to replace both of these with one for loop, like so:

for field_name, pattern in (
        ('run_id', run_id), ('mode', run_mode)):

    values = dsets[field_name].values
    mask = np.zeros(len(values), dtype=np.bool_)
    for i, x in enumerate(values):
        if pattern_type == 'fnmatch':
            mask[i] = fnmatch.fnmatch(x, pattern)
        elif pattern_type == 're':
            mask[i] = bool(re.match(pattern, x))

    dsets = dsets[mask]

but I have not tested this.

[mlotfiben]

Hi Jelle, thanks. when I used the fork version, I forgot to add the run_id as argument to select_run. Actually we need to leave it as it is, because then if the user give one of them, either run_mode or run_id, the logic, as it is now will do the job. But if we use the loop then both should be given at the same time, which I think not the goal of the selection_runs. Please correct me if I am wrong. Best, Lotfi

…

On Thu, Jun 20, 2019 at 5:44 PM Jelle Aalbers ***@***.***> wrote: Thanks Lotfi. It currently crashes because there is not yet a run_id argument in the select_runs function. You can add one with a default of None (and update the docstring). It seems the logic for run_id is very similar to that of run_mode. Maybe it would work to replace both of these with one for loop, like so: for field_name, requested_value in ( ('run_id', run_id), ('mode', run_mode)): values = dsets[field_name].values mask = np.zeros(len(values), dtype=np.bool_) if pattern_type == 'fnmatch': for i, x in enumerate(values): mask[i] = fnmatch.fnmatch(x, requested_value) elif pattern_type == 're': for i, x in enumerate(values): mask[i] = bool(re.match(requested_value, x)) dsets = dsets[mask] but I have not tested this. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_AxFoundation_strax_pull_192-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DACXCDBIIT22VPRKOOFZMXYTP3OCSPA5CNFSM4HZREYM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYFOQSY-23issuecomment-2D504031307&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=scVlfZQUS1WkwFJdQyC8dg&m=vSp1cvD1jqxLgP_f1-PZmHHx2toN9pdljaDvqKt5HPM&s=5uD8ByiuniIf96SNE2HQgRt8UJG8iTX7gIms3AMG5M8&e=>, or mute the thread <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ACXCDBNR2JHIVX7UHWBXZZLP3OCSPANCNFSM4HZREYMQ&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=scVlfZQUS1WkwFJdQyC8dg&m=vSp1cvD1jqxLgP_f1-PZmHHx2toN9pdljaDvqKt5HPM&s=QNjjXG1Ft3Pudp9HF5AQ3q2sokheJzPfyHu2c6jQ_Xs&e=> .

[JelleAalbers]

Ah sorry, in the loop we would have to add a check for None, otherwise indeed you'd have to give both.

for field_name, pattern in (
        ('run_id', run_id), ('mode', run_mode)):
    if pattern is None:
        continue

Perhaps there are other modifications too (I did not test the loop code), but it would be great to avoid code duplication.

Speaking of testing, could you edit the unit test here https://github.com/AxFoundation/strax/blob/master/tests/test_core.py#L252 to test that this new feature works?

[JelleAalbers]

Thanks Lotfi! We discovered the unit test for select_runs is actually broken (it passes tautologically), I will fix it after merging this.