refine_bravais_settings hangs on very large reflection tables

[ndevenish]

[Copied from email from @jmp1985]

... sent me these files (copied to /dls/mx-scratch/upc86896/2020/05/dials_refine_bravais_settings_error) which when run through dials.refine_bravais_settings result in errors in multiprocessing queue. I can confirm that I get the same error on an up to date build (copied below). The files otherwise look fine (as in there is nothing wrong with the files and they open in the reciprocal lattice viewer). Do you have any idea what this may be?

Best wishes
James

@rjgildea posted the output from running with current:

$ dials.refine_bravais_settings /dls/mx-scratch/upc86896/2020/05/dials_refine_bravais_settings_error/47_indexed.*
DIALS (2018) Acta Cryst. D74, 85-97. https://doi.org/10.1107/S2059798317017235
DIALS 3.dev.21-g6d4610270
The following parameters have been modified:

input {
  experiments = /dls/mx-scratch/upc86896/2020/05/dials_refine_bravais_settings_error/47_indexed.expt
  reflections = /dls/mx-scratch/upc86896/2020/05/dials_refine_bravais_settings_error/47_indexed.refl
}

Traceback (most recent call last):
  File "/dls/science/groups/scisoft/DIALS/CD/latest/dials-dev20200527/conda_base/lib/python3.6/multiprocessing/queues.py", line 240, in _feed
    send_bytes(obj)
  File "/dls/science/groups/scisoft/DIALS/CD/latest/dials-dev20200527/conda_base/lib/python3.6/multiprocessing/connection.py", line 200, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/dls/science/groups/scisoft/DIALS/CD/latest/dials-dev20200527/conda_base/lib/python3.6/multiprocessing/connection.py", line 393, in _send_bytes
    header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647

[graeme-winter]

<- has just stubbed toes on same

[graeme-winter]

Discussion: dumping reflection shoeboxes before spawning subprocesses would probably make this go away