DIALS 3.19.0

2024-04-17:

Features

• dials.predict: Allow usage when image data are not available. (#2553)
• dials.ssx_index: Allow use of sequences indexer, pink_indexer and low_res_spot_match indexing algorithms. (#2614)
• dials.export: Add option composition= to specify of asymmetric unit composition for SHELX .ins file output. (#2623)
• Add TOFSpotFinder to tailor default params to time of flight experiments and add additional reflection table data. (#2602)
• dxtbx: Add format reader for Jungfrau4M serial images from beamline ID29 at ESRF. (cctbx/dxtbx#659)
• dxtbx: Better handling for bad data in spectra calibration in XTC format. (cctbx/dxtbx#674)
• dxtbx: Add Bruker and miniCBF format readers for the ELDICO ED-1 electron diffractometer with DECTRIS QUADRO detector. (cctbx/dxtbx#682)
• dxtbx: FormatSMVTimePix_SU: Always mask out the central cross of virtual pixels. (cctbx/dxtbx#683)
• dxtbx: Add format reader for ISIS SXD detector. (cctbx/dxtbx#687)
• dxtbx: Detector distance can now be manually overridden for multi-panel detectors. (cctbx/dxtbx#698)
• dxtbx: Add format reader to read time of flight Laue data from MANDI. (cctbx/dxtbx#703)

Bugfixes

• Fix building on RHEL8 and other more recent distributions. (#1465)
• dials.index: Joint indexing is automatically set on for rotation data, off for still data. This can be overridden by explicit use of joint_indexing=. (#2605)
• dials.export format=shelx: Increased precision of unit cell parameters and their estimated standard deviations written to .ins file. (#2624)
• dials.ssx_index: Don't combine detector models if individually refined. (#2634)
• dials.scale: Fix error in propagation of partiality variance, when scaling still data. (#2642)
• dials.export: Fix crash when exporting scaled still-shot data. (#2646)
• dxtbx: Fix scan comparison for scan properties changes (cctbx/dxtbx#669)
• dxtbx: Eiger Support: Invert the module dimensions, only for older firmware versions. See https://media.dectris.com/230203-Release_Notes-DECTRIS_EIGER2.pdf for reference. (cctbx/dxtbx#676)
• dxtbx: FormatMRC: Better handling of extended headers. (ccpem/mrcfile#50), and extended headers are ignored if they contain junk values. (cctbx/dxtbx#679)
• dxtbx: Negative rotation angles are now allowed, the goniometer axis will be inverted if necessary. (cctbx/dxtbx#690)
• dxtbx: dials.import now uses natural sorting on input data, instead of strict sorting. (cctbx/dxtbx#697)
• dxtbx: Fix setting detector distance for single panel detectors that have a hierarchy. (cctbx/dxtbx#699)
• dxtbx: Fix error introduced in FormatSMVJHSim causing test failures. (cctbx/dxtbx#710)
• xia2.multiplex: Avoid space group analysis if given reference file. (xia2/xia2#770)
• xia2.ssx: Allow use of stills.indexer=sequences in ssx_index routine. (xia2/xia2#779)
• xia2.multiplex: Allow use of relative_length_tolerance= and absolute_angle_tolerance= (used by cosym) in multiplex. (xia2/xia2#786)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

DIALS 3.18.1

2024-03-26:

• wxPython was inadvertently reverted to 4.1 instead of 4.2 for python 3.9 and 3.10 releases. (#2636)

For a more complete list of changes and fixed bugs check the dials changelog.

There are no changes to cctbx, dxtbx or xia2 in this release.

DIALS 3.18.0

2024-02-22:

Features

• dials.slice_sequence: Add exclude_images_multiple= option to split a scan at calibration images, as an alternative to excluding those images in dials.integrate. (#2565)
• dials.index: Add indexing.method=pink_indexer for still images, based on Gevorkov Y et al. pinkIndexer – a universal indexer for pink-beam X-ray and electron diffraction snapshots <https://doi.org/10.1107/S2053273319015559>_. (#2604)
• dxtbx: XTC Format has a new parameter, spectrum_required=, to better handle spectra calibration for bad data. (cctbx/dxtbx#674)
• dxtbx: Add Bruker and miniCBF format readers for the ELDICO ED-1 electron diffractometer with DECTRIS QUADRO detector. (cctbx/dxtbx#682)
• dxtbx: FormatSMVTimePix_SU: Always mask out the central cross of virtual pixels that have been reconstructed from wide edge pixels. (cctbx/dxtbx#683)
• dxtbx: Add format reader for ISIS SXD detector. (cctbx/dxtbx#687)

Bugfixes

• dials.index: Correctly output imageset id when joint_indexing=False and max_lattices>1 (#2538)
• dials.index: Ensure the unindexed_reflections table in the indexer classes is updated during refinement. (#2545)
• dials.cosym: Ensure correct reindexing of low symmetry data to the correct setting. (#2548)
• dials.export: Unmerged MTZ output now uses gemmi, fixing various longstanding issues. (#2549)
• dials.symmetry/dials.cosym: Avoid crashes when unable to find consistent symmetry during cell reduction routine. (#2552)
• dials.sequence_to_stills: Avoid creating an experiment for an image with no reflections on it. (#2554)
• dials.reciprocal_lattice_viewer: Fix toggling of ids on spotfinding output when viewing multiple files. (#2566)
• dials.estimate_resolution: Fix bug causing fitting failures for isigma, misigma curves. (#2568)
• dials.reciprocal_lattice_viewer: Ensure the correct panel for the beam centre is recorded when loading models. (#2572)
• dials.reindex: When reindexing against a reference, use the best cell when determining lattice symmetry. (#2573)
• dials.cosym: Filter out datasets that can't be mapped through a consistent best cell, rather than exiting. (#2574)
• dials.export_bitmaps: Fix for use of imageset_index parameter on image sets that don't start at image numbers of 1. (#2591)
• Reorganize imports in export_mtz.py so that its gemmi-less objects can be used by other gemmi-less software. (#2592)
• dials.image_viewer: Fix for stills experiments viewing crash. (#2594)
• dxtbx: Importing the (deprecated and removed) dxtbx.datablock module failed to display warning properly. (cctbx/dxtbx#665)
• dxtbx: Fix scan comparison for scan properties changes. (cctbx/dxtbx#669)
• dxtbx: Eiger support: Following bug fix in DECTRIS file writer, invert the module dimensions, but only for older firmware versions. (cctbx/dxtbx#676)
• dxtbx: FormatMRC: Better handling of extended headers. (ccpem/mrcfile#50), and extended headers are ignored if they contain junk values. (cctbx/dxtbx#679)
• dxtbx: Fixed Scan.from_dict bug where some properties were not correctly parsed. (cctbx/dxtbx#688)
• dxtbx: In DXTBX do not assert the rotation angles for images are positive, and invert in import with an axis inversion if needed. (cctbx/dxtbx#690)
• dxtbx: For dials.import: use natural sorting on input data in preference to strict sorting (cctbx/dxtbx#697)
• xia2.multiplex: Avoid space group analysis, if given reference file. (xia2/xia2#770)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

DIALS 3.17.0

2023-11-03:

Features

• Circular dependencies have been removed between DIALS and cctbx.xfel by using the new serialtbx. (#2404)
• dials.import: Enable use of an image range selection when importing a still sequence. (#2490)
• dials.estimate_resolution: Improved CC½ fitting by using a weighted tanh fit. (#2499)
• dials.find_spots and dials.integrate now accept exclude_images_multiple=N, which will be expanded to the appropriate exclude_images= definition that excludes every Nth image from the data set. This is intended for the sitation where a scan of diffraction images is regularly interrupted by calibration images. (#2511)
• dials.ssx_integrate: Add mosaicity_max_limit= setting, to control what mosaicity limit is considered unphysically large. (#2518)
• dials.ssx_integrate: Add max_cell_volume_change_fraction= parameter to catch highly divergent cell refinements (#2521)
• dxtbx: Add nxmx_writer, a tool for converting any data dxtbx can read to NeXus data. (cctbx/dxtbx#615)
• dxtbx: Remove circular dependencies between dxtbx and cctbx.xfel, by using the new serialtbx. (cctbx/dxtbx#627)
• dxtbx: Set the beam probe to electron in both FormatNXmxED and FormatSER. (cctbx/dxtbx#661)
• xia2.ssx: Enable slicing a subset of images/templates using <filename>:start:end syntax. (xia2/xia2#740)
• xia2.ssx_reduce: Improved workflow for resolving indexing ambiguity. (xia2/xia2#761)
• xia2.ssx: Add starting_geometry= option, to set an initial geometry for further geometry refinement. (xia2/xia2#763)
• xia2.cluster_analysis: Add run_cluster_identification= option to toggle cluster identification analysis. (xia2/xia2#767)

Bugfixes

• dials.cosym: Fix handling of refinement weights, for sparse data with a low number of common reflections. This improves the handling of serial still data. (#2503)
• dials.reflection_viewer: Fix a wsPython-related type error. (#2506)
• dials.sequence_to_stills: Now works with statically-refined experiments. (#2516)
• dials.sequence_to_stills: Generate identifiers for the split experiments. (#2517)
• dials.sequence_to_stills: No longer crashes when operating on a scan that does not start at image 1. (#2520)
• dials.symmetry: Print reindexed unit cell in log output. (#2523)
• dials.index: Correctly index multiple sweeps for multi-sweep indexed input. (#2534)
• dials.ssx_integrate: Fix bug in foreground mask calculation, affects datasets where wavelength is much bigger or smaller than 1 Å. (#2535)
• dials.index: Fix lattice counting when indexing with >1 sweep and >1 lattice (#2539)
• dials.cosym/dials.scale: Handle potential quasi_normalisation failures (for serial data). (#2540)
• Bootstrap --cmake: Fix configuration when CCP4 is already loaded. Previously, this could cause a failure as it tried to resolve the CCP4 copy of HDF5 in order to determine version. (#2544)
• dxtbx.image_average: Better handle detector gain and pixel data type. (cctbx/dxtbx#660)
• dxtbx: Beam.probe is no longer reset if any geometrical override is provided at import. (cctbx/dxtbx#661)
• dxtbx: Pilatus 4: Do not invert module size that is correctly written in master file. (cctbx/dxtbx#663)
• dxtbx.plot_detector_models: Use noninteractive matpotlib backend, if using the pdf_file= option. (cctbx/dxtbx#664)
• xia2.multiplex: Automatically extend r_free flags for clusters and filtered datasets. (xia2/xia2#747)
• xia2.ssx_reduce: Improve data selection for indexing ambiguity resolution. (xia2/xia2#760)
• xia2.cluster_analysis: Fixed bug where interesting clusters were not identified, due to inconsistency in file paths. (xia2/xia2#764)
• xia2.multiplex: Fix duplicate-batch-offsets crash for multi-lattice data. (xia2/xia2#765)
• xia2.multiplex: Exit cleanly when supplied with still-shot data. (xia2/xia2#766)
• xia2.ssx_reduce: Fix test for potential accidental indexing ambiguities for non-MX space groups. (xia2/xia2#768)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

DIALS 3.16.1

2023-09-05:

• dials.merge: Fix potential for crash with r_free_flags.extend=True, if there is no new flags to extend. (#2491)
• dxtbx: Fix situation where a bad Beam.probe could cause undefined behaviour. (cctbx/dxtbx#656)
• dxtbx: Fix performance regression loading large experiment lists containing profile/scaling models. (cctbx/dxtbx#658)

For a more complete list of changes and fixed bugs check the dials and dxtbx changelogs.

There are no changes to cctbx or xia2 in this release.

DIALS 3.16.0

2023-08-14:

Features

• Support for building a CMake developer environment with bootstrap.py --cmake. This should be considered experimental, for now. (#2474)
• dials.export_bitmaps: Set resolution_rings.d_spacings= to manually specify the size of resolution rings. (#2461)
• dials.export: You can now specifying a wavelength tolerance when exporting multi-wavelength data to mtz. (#2471)
• dials.index: Experiment number is now shown when indexing multiple experiments. (#2470)
• dials.merge: Added stereographic projections to the html report, if merging more than one experiment. (#2476)
• dials.scale: Absorption correction is now switched off if all scattering angles are small (cos(2ϑ) ~ 1). (#2455)
• xia2.cluster_analysis: The cluster selection algorithm now handles edge cases more robustly. (xia2/xia2#744)
• xia2.multiplex: Add support for multi-wavelength processing. (xia2/xia2#755)
• xia2.multiplex: Added option reference=, to use a reference pdb for consistent indexing. (xia2/xia2#748)
• dxtbx: Add new Beam class dxtbx.model.PolychromaticBeam, for polychromatic/multi-wavelength/wide bandpass experiments. (cctbx/dxtbx#621)
• dxtbx: The model.Beam object now has a probe value to keep track of the type of radiation. (cctbx/dxtbx#647)
• dxtbx: Formats: Reflect move of Eiger detector from PETRA P14 to P13. (cctbx/dxtbx#626)
• dxtbx: Formats: CBFMini support for the EIGER2 16M detector at CHESS beamline ID7B2, which has an inverted rotation axis. (cctbx/dxtbx#649)
• dxtbx: Formats: Support for Eiger 9M on ESRF ID23-2, which has an undeclared vertical goniometer. (cctbx/dxtbx#651)
• dxtbx: Formats: Partial support for the Rigaku Oxford Diffraction file format, including support for multi-axis goniometers and faster decompression. (cctbx/dxtbx#645)

Bugfixes

• dials.combine_experiments: No longer requires reflection data. (#2323)
• dials.image_viewer: Fix crash when the detector is rotated more than 90° from the beam direction. (#2479)
• dials.import: Now ensures manual geometry updates are only applied once to each model. This ensures invert_rotation_axis=True will only invert the rotation axis once. (#2469)
• dials.index: indexing.refinement_protocol.mode=None now applies to both the scans and stills indexers. (#2456)
• dials.index: Now fails with a useful error message when there are too few spots for indexing. (#2457)
• dials.model_background: Works again for data sets that do not start from image "1". (#2405)
• dials.reciprocal_lattice_viewer: Handle cases where a beam centre cannot be calculated, such as a detector with a high 2θ swing angle. (#2483)
• dials.reindex: Fix crash reindexing against a reference file, for more than one input file. (#2439)
• dials.ssx_integrate: Address issue with ellipsoid angular models by replacing with new, correct, r-dependent models. (#2463)
• Updated DIALS to imageset slicing properly using 0-based indices. Previously, the slice had to be accessed with the original image indices. (#2411)
• dxtbx: Panel geometry definitions in PHIL are merged by panel id before constructing panels. (cctbx/dxtbx#299)
• dxtbx: flumpy: Fix case where incorrect flex.vec2, flex.vec3 could be generated. (cctbx/dxtbx#439)
• dxtbx: NXmx files with multidimensional arrays (images, modules, or both) are now handled. (cctbx/dxtbx#612)
• dxtbx: Slicing of imageset objects is now consistently 0-based, including for the sliced data accessor. Previously, the data accessor had to be accessed with the original index offsets. (cctbx/dxtbx#633)
• dxtbx: Formats: Add fix for Eiger / NXmx data from DLS i19-2, to correctly assign the image bit depth. (cctbx/dxtbx#652)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

DIALS 3.15.1

2023-06-29:

• dials.export_bitmaps: Fix the resolution_rings.fontsize= feature to work on Mac, and more reliably across platforms. (#2441)
• xia2: Don't import matplotlib in scipy.hierarchy.dendrrogram, causing errors on headless machines. (xia2/xia2#745)
• dxtbx.dlsnxs2cbf: Fix import overwritten by local variable. (cctbx/dxtbx#641)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

There are no changes to cctbx in this release.

DIALS 3.15.0

2023-06-13:

Features

• Python 3.11 is now supported as a bootstrap target. (#2338)
• dials.refine: Independent groups of experiments are now refined separately by default. The previous behaviour can be selected by using separate_independent_sets=False. (#2336)
• Support IOBS/SIGIOBS and FOBS/SIGFOBS columns when reading reference intensities from MTZ files. (#2386)
• dials.export_bitmaps: Optionally overlay ice rings. (#2393)
• dials.reindex: Allow reindexing against a reference file (mtz, pdb, cif) from which intensities can be extracted/generated (#2420)
• dials.refine: new option auto_reduction.ignore=True allows avoiding the parameter auto-reduction procedure for special cases. (#2431)
• dxtbx: Support for Bruker Photon detectors has been extended to include Photon-III. (cctbx/dxtbx#637)
• New: xia2.cluster_analysis: The clustering algorithm from xia2.multiplex is now available as a separated program, applicable to any merged data from dials (rotation or stills). (xia2/xia2#733)
• xia2.ssx: Report hit and indexing rates during processing. (xia2/xia2#735)
• xia2.ssx_reduce: Allow setting of the partiality threshold (default now 0.25). (xia2/xia2#743)

Bugfixes

• dials.export_bitmaps: Correctly display the requested number of resolution rings. (#2397)
• dials.scale: Fix crash in html plotting for scaling and filtering when R-pim/R-merge values contain 'None' values (#2410)
• dials.scale, dials.cosym: Use bulk solvent model when calculating intensities from a reference file. (#2414)
• dials.integrate: Fix multiprocessing on Windows with Python 3.11. (#2426)
• dials.ssx_integrate: Fix error in partiality calculation for ellipsoid integration algorithm. (#2429)
• dials.import: Don't print filename template for every image of an ImageSequence. (#2432)
• dxtbx: Rigaku Saturn SMV images with multi-axis crystal goniometers are now handled, instead of being silently ignored. With thanks to James Hester for this contribution. (cctbx/dxtbx#617)
• dxtbx: FormatCBFFull: If rotation angles are decreasing, then invert the rotation axis as well as the angles, to be consistent. (cctbx/dxtbx#623)
• xia2.ssx: Include solvent contribution when generating reference intensities from a model. Adds k_sol and b_sol parameters. (xia2/xia2#737)
• xia2.ssx: Fix error in progress reporting when no hits found, or when no images indexed in a batch. (xia2/xia2#739)
• xia2.cluster_analysis: Improved cluster selection algorithm to more robustly handle edge cases. (xia2/xia2#744)

For a more complete list of changes and fixed bugs check the dials, dxtbx and xia2 changelogs.

DIALS 3.14.2

2023-05-16:

• dxtbx: Compatibility fix for the DECTRIS Eiger FileWriter. Recent FileWriter versions split bit depth metadata into two separate items, bit_depth_readout from the NXmx standard, and the new bit_depth_image field. This adds support for the latter, and now passes the metadata through into image conversion. (cctbx/dxtbx#632)

For a more complete list of changes and fixed bugs check the dxtbx changelog.

There are no changes to cctbx, dials or xia2 in this release.

DIALS 3.14.1

2023-04-20:

• dials.export_bitmaps: Correctly scale resolution rings with image binning= set. (#2392)
• Revert GTK-message-supression change that was causing reciprocal_lattice_viewer errors. (#2395)

For a more complete list of changes and fixed bugs check the dials changelog.

There are no changes to cctbx, dxtbx or xia2 in this release.