dials.scale exclude_datasets and experiment identifiers

[huwjenkins]

Removing a dataset with the exclude_datasets option to dials.scale appears to be able to trigger the potential issue mentioned in #902

e.g.

dials.scale integrated.{expt,refl} exclude_datasets=0

and subsequently running dials.two_theta_refine scaled.{expt,refl}

Crashes:

Traceback (most recent call last):
  File "/Users/huw/dials-v3-8-2/build/../modules/dials/command_line/two_theta_refine.py", line 560, in <module>
    run()
  File "/Users/huw/dials-v3-8-2/conda_base/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/Users/huw/dials-v3-8-2/build/../modules/dials/command_line/two_theta_refine.py", line 556, in run
    script.run(args)
  File "/Users/huw/dials-v3-8-2/build/../modules/dials/command_line/two_theta_refine.py", line 502, in run
    refiner = self.create_refiner(params, reflections, experiments)
  File "/Users/huw/dials-v3-8-2/build/../modules/dials/command_line/two_theta_refine.py", line 233, in create_refiner
    refman = TwoThetaReflectionManager(
  File "/Users/huw/dials-v3-8-2/modules/dials/algorithms/refinement/two_theta_refiner.py", line 62, in __init__
    super().__init__(*args, **kwargs)
  File "/Users/huw/dials-v3-8-2/modules/dials/algorithms/refinement/reflection_manager.py", line 397, in __init__
    refs_to_keep = self._id_refs_to_keep(reflections)
  File "/Users/huw/dials-v3-8-2/modules/dials/algorithms/refinement/reflection_manager.py", line 555, in _id_refs_to_keep
    raise DialsRefineConfigError(
dials.algorithms.refinement.DialsRefineConfigError: Please report this error to dials-support@lists.sourceforge.net: Experiment id 0 contains no reflections with valid scan angles

and the message that Experiment id 0 contains no reflections with valid scan angles seems to be correct:

>>>from dials.array_family import flex
>>>refl = flex.reflection_table.from_file('scaled.refl')
>>>[i for i in refl.experiment_identifiers()]
[(1, '66ed5bfb-c1b9-4fca-b253-222d3fcc93ab'), (2, '80519636-5728-4195-992a-d9d89530b9c6'), (3, 'cd387427-9daa-46db-916c-f5ab12fbd7a7')]

[jbeilstenedmands]

Thanks for reporting. I have a fix coming in, but for now you can use the following workaround and then it should work fine:

from dials.array_family import flex
refl = flex.reflection_table.from_file("scaled.refl")
refl.reset_ids()
refl.as_file("scaled.refl")

[huwjenkins]

Thanks for fix. I get same error with dials.refine. dials.two_theta_refine was just an example but I think it's only these 2 dials commands I'd be likely to run with the output experiments/reflections from dials.scale

dials.refine scaled.expt scaled.refl output.experiments=joint_refined.expt output.reflections=joint_refined.refl detector.fix_list=Tau2,Tau3 scan_varying=false

Experiment id 0 contains no reflections with valid scan angles

and I think #2007 only fixes dials.two_theta_refine?