dials.export: No longer allow (erroneous) MTZ export for multiple experiments with multiple space groups. (#1915)dials.export: No longer fails for XDS_ASCII and SADABS export withintensity=auto. (#1926)dials.report: Fix broken json output option. Include more graphs in json output. (#1932)
- Bootstrap support for MacOS M1 platforms. (#1841)
- New
dials.find_rotation_axisprogram optimises the rotation axis from strong spot positions prior to indexing. (#1884) dials.import: Allow importing templates with no template characters. (#1840)dials.stills_process: Performance improvements in Kapton absorption correction and in rare cases of highly mosaic crystals. (#1846)dials.image_viewerCoordinates are now given in fast, slow order. (#1849)dials.image_viewer: Crystal basis vectors are now shown in the same colour as their predictions. (#1855)dials.image_viewer: Add the option to display the rotation axis (#1856)dials.image_viewer: Draw resolution rings for curved detectors. (#1899)dials.import: Unhandled files are now by default ignored. This means that e.g.*.logfiles alongside images will no longer prevent a successful import. Setignore_unhandled=Falseto restore the previous behaviour. (#1881)dials.scale: Allow fixing of a particular correction with e.g.physical.correction.fix=absorption. (#1883)- Installer now accepts a
--raw-prefixoption to use the target destination directly, instead of in adials-X.Ysubdirectory. (#1896)
dials.compute_delta_cchalf: Fix crash when only using passing dataset/group. (#1892)dials.find_bad_pixels: Pixel coordinates are now reported in row-major order, and mask value is now set to 16, which corresponds internally to "noisy pixel". (#1876)dials.find_rotation_axis: removed unused parameteroptimise={True|False}. (#1898)dials.report: Don't show otherwise empty sections. (#1875)
- Improvements to 3DED tutorials. (#1850)
- SARS-CoV-2 main protease tutorial: process in C2 rather than I2 setting for consistency with published structures. (#1854)
- Removed outdated lysozyme nanocrystals tutorial. (#1877)
- Add an associated projects page to the website. (#1893)
- Bootstrap no longer allows creating Python 3.6 environments. (#1852)
dials.util.mp: deprecateparallel_map()function and remove previously deprecatedpreserve_exception_message=parameters. (#1860)
dials.reciprocal_lattice_viewer: In cases with multiple lattices, "Crystal Frame" now aligns all crystal frames, rather than just the first. Unindexed reflections are no longer shown in this mode. (#1868)
No significant changes.
This is the last release to support Python 3.6. Future releases will require a minimum of Python 3.7.
- DIALS bootstrap now creates a Python 3.9 environment by default (#1735)
- New program:
dials.reference_profile_viewerfor viewing reference profiles dumped bydials.integratewhen using thedebug.reference.output=Trueoption. (#1759) dials.combine_experiments: Unindexed reflections are now included in the combined output (#1760)dials.image_viewer: Image overlays are now accumulated over stacks of images (#1750)dials.image_viewer: Allow control of the basis vector scale from the settings window (#1780)dials.image_viewer: Better colour choice for text overlays. Labels will now be light grey on Black, or Dark grey on White. The previous settings were sometimes hard to read on narrow-contrast images. (#1781)dials.merge: Include DANO/SIGDANO columns in output .mtz whenanomalous=Trueandtruncate=True(#1809)dials.reciprocal_lattice_viewer: Show resolution on the "nearest point" label (#1770)dials.reciprocal_lattice_viewernow shows the path to the reflections in the title bar (#1771)dials.reciprocal_lattice_viewer: The default marker size now scaled automatically based on the data density (#1773)dials.scale: Always enable absorption correction if theabsorption_level=parameter is set. Previously it was only enabled for sweeps >= 60° or ifabsorption_correction=True. (#1793)dials.scale: Allow a shared absorption correction between sweeps if using the physical model, with the optionshare.absorption=True. Extra absorption correction plots have also been added; and multiple sweeps are now aligned to the same reference frame. (#1811)- API:
...scaling_library.scaled_data_as_miller_arraynow sets wavelength in the returnedmiller.array(#1808) reflection_table.match()now returnsflex.size_tindex arrays, instead offlex.int. (#1784)- New bootstrap option:
--condato install with miniconda instead of micromamba. (#1730)
dials.combine_experiments: Correctly preserve mapping to images. This affectsdials.image_vieweranddial.reciprocal_lattice_viewer. (#1093)dials.compute_delta_cchalf: Unwarranted precision in the output has been reduced (#1751)dials.find_spots: Fix counting of imagesets in histogram output (#1827)dials.image_viewer: Add buttons to clear unit cell and generic ring display (#1777)dials.image_viewer: Fix various minor behavioural bugs in the spot-finding and image type controls. (#1796)dials.import: Fail gracefully when # is missing from template. (#1840)dials.integrate: change default filename of debug reference profile toreference_profiles.pickle. (#1747)dials.integrate: Change default configuration so that unintegrated reflections are not retained. This helps reduce memory usage of downstream tools. Setoutput_unintegrated_reflections=Trueto restore the previous behaviour. (#1753)dials.integrate: ensure imageset_ids are always output. Affected use of image viewer, reciprocal lattice viewer on multi-sweep data. (#1762)dials.reciprocal_lattice_viewer: When starting withblack_background=False, ensure the rotation axis and beam vector are displayed in black. (#1540)dials.reciprocal_lattice_viewer: More robust beam centre control that works for multiple panel detectors (#1842)dials.refine_bravais_settings: correctly report mI Bravais settings (#1825)dials.split_experiments: Update the imageset_id column in the output reflection files. (#1792)- Don't fail
bootstrap.pyif a submodule is missing a reference (#1834) - Correctly handle reflection
imageset_idcolumn indials.scale,dials.cosym, anddials.symmetry. (#1763)
dials.anvil_correction: Made a small improvement to the developer documentation. (#1788)- Fix help string for
best_monoclinic_beta=parameter (fordials.cosym,dials.refine_bravais_settingsanddials.symmetry) (#1833) - Added a new tutorial on 3DED/MicroED data processing. (#1837)
- Add a "Getting started" page to the documentation on the website. (#1844)
- Add a tutorial on processing small molecule 3DED data. (#1847)
- The previously deprecated
dials.resolutionizercommand has been removed. Please usedials.estimate_resolutioninstead. (#1330) - The previously deprecated
dials.refineparametertrim_scan_edgeshas been removed. Please usescan_margin=...instead. (#1374) - The previously deprecated
Spotfinder()()interface has been removed. Please useSpotfinder().find_spots()instead. (#1484) - The previously deprecated
dials.util.masking.MaskGeneratorhas been removed. Please usedials.util.masking.generate_maskinstead. (#1569) - The bootstrap option
--mambahas become the default and will be removed in the future. (#1730) dials.anvil_correction: Drop compatibility support for SciPy < 1.4 (#1787)
- #1746, #1733, #1752, #1755, #1756, #1764, #1767, #1772, #1783, #1789, #1791, #1794, #1795, #1799, #1802, #1804, #1806, #1807, #1812, #1816, #1817, #1823, #1830, #1835, #1836, #1839
dials.stills_process: Fix case where imagesets and experiment filenames could potentially disagree (#1814)dials.scale: Fix incorrect output files, for targeted scaling with more than one target dataset. (#1815)dials.image_viewer: Fix opening datasets withload_models=False(#1818)
dials.image_viewer: Fix thebasis_vector_scale=parameter. (#1769)
dials.image_viewer: Fix display of spotfinding intermediates (threshold, dispersion, etc) when viewing multiple still experiments (#1734)dials.image_viewer: Stacking images no longer gives incorrect results for multi-sweep data beyond the first sweep (#1758)
No significant changes.
dials.integrate: Avoid crash when data is too large to process in memory, by splitting into subsets (#1392)- New bootstrap options:
--mambato install with micromamba, and--cleanto remove installation caches immediately after completion. (#1676) dials.find_spots_server: Faster filtering of reflections by resolution (#1680)dials.scale: Add optionerror_model.grouping=to control refinement of either individual or grouped error models during scaling (#1684)dials.scale: Addedphysical.absorption_level=[low|medium|high]option for automatic setting of suitable absorption correction parameters. (#1688)dials.cosym: Significantly faster calculation of Rij matrix of pairwise correlation coefficients (#1693)dials.sort_reflectionsanddials.merge_reflection_listsare now available without adev.prefix. (#1703)- New command:
dials.find_bad_pixelsto identify pixels which are identified as signal in >= 50% of images (#1710) dials.image_viewer: Add selector to choose between a new default "image" and traditional "lab" coordinate frames. "image" frame attempts to align the fast/slow axes of the detector panels to screen x and y coordinates, so overall detector rotations will mostly be invisible. "lab" frame is the previous projection, where rotated detectors will appear rotated.dials.export_bitmaps: Gained this sameprojection=option. (#1716)dials.find_spotsanddials.integrate: nproc= now works with N > 1 on Windows. (#1724)
- Fix rare crash in symmetry calculations when no resolution limit could be calculated (#1641)
dials.report: Add units of pixels / images to centroid difference histograms (#1677)dials.refine: Scan-varying refinement failed whentrim_scan_to_observations=Falsewas used. (#1686)dials.spot_counts_per_image: Show an explicit error if given data that isn't spotfinding output (i.e. unindexed reflections/experiments). (#1690)dials.integrate: Improved background model variance calculation for integrating detectors. (#1692)dials.stills_process: improve processing performance by preventing re-reading of image data (#1705)dials.background: Correctly identify signal pixels for integrating detectors, and respect pre-calculated masks. (#1726)dials.integrate: Fixed bug in memory-use calculation for multi-sweep integration runs (#1728)
- Remove remaining 'master' references in the documentation. (#1632)
- The previously deprecated
dials.util.masking.MaskGeneratornow prints a user warning. Please usedials.util.masking.generate_maskinstead. (#1643) dials.cosym: Remove clustering code as this is no longer a necessary part of determination of symmetry or reindexing operations, and serves no useful purporse. (#1647)dials.cosym:nproc=parameter is deprecated. The algorithm is much faster on single cores. (#1693)- The pytest option
--runslowwas retired. The tests that it triggered will now always run. (#1695) dev.dials.csvhas been deprecated. Similar functionality is available withdials.export format=json. (#1708)dials.util.mp: Thepreserve_exception_messageargument has been deprecated. (#1722)
- #1631, #1633, #1648, #1649, #1652, #1661, #1672, #1673, #1674, #1675, #1676, #1678, #1679, #1687, #1696, #1697, #1698, #1701, #1706, #1707, #1711, #1713, #1717, #1718, #1720
dials.scale: Fix crash when full-matrix minimisation is unsuccessful due to indeterminate normal equations. (#1653)dials.scale: Fix crash when no reflections remain after initial filtering. (#1654)dials.export: Fix error observed withformat=mmciffor narrow sweeps with low symmetry (#1656)- Fix image numbering inconsistency in ascii histogram of per-image spot counts (#1660)
dials.find_spots_server: Significant performance improvement for HDF5 grid scans. (#1665)
- Log messages from spot finding and integration no longer ignore logging level when using
nproc > 1. This mainly affects usage of dials from outside contexts. (#1645)
dials.cosym: Significantly faster via improved computation of functional, gradients and curvatures (#1639)dials.integrate: Added parametervalid_foreground_threshold=, to require a minimum fraction of valid pixels before profile fitting is attempted (#1640)
dials.cosym: Cache cases where Rij is undefined, rather than recalculating each time. This can have significant performance benefits when handling large numbers of sparse data sets. (#1634)dials.cosym: Fix factor of 2 error when calculating target weights (#1635)dials.cosym: Fix brokenengine=scipyoption (#1636)dials.integrate: Reject reflections with a high number of invalid pixels, which were being integrated since 3.4.0. This restores better merging statistics, and prevents many reflections being incorrect profiled as zero-intensity. (#1640)
dials.integrate: Profile-fitting improvements; Profile fitting will now be attempted on reflections with masked pixels, and the number of reflections qualifying for profile-fitting on multi-panel detectors has dramatically increased. (#1297)dials.import: When usingreference_models=, individual components of the model can be excluded withuse_beam_reference=,use_gonio_reference=anduse_detector_reference=. (#1371)flex.reflection_table.matchcan now match reflections with configurable distance and scaling between any 3-vector column in the reflection tables. The default is still"xyzobs.px.value". (#1398)dials.background: Add optionoutput.plot=to save an image to disk, instead of displaying interactively. Image files can now also be used directly. (#1537)dials.import: The defaulttolerance.scan.oscillation=is increased to 3% of the image width, in order to accommodate electron diffraction datasets with poor rotation stages. (#1543)dials.background: Add support for multiple imagesets (#1554)- dials.estimate_resolution: reject Wilson outliers to minimise effect of spurious observations from e.g. ice rings on the resulting resolution estimates (#1580)
dials.cosym: Use numpy in place of flex for large parts of cosym analysis (#1581)dials.cosym: Add option to use scipy L-BFGS-B <https://docs.scipy.org/doc/scipy/reference/optimize.minimize-lbfgsb.html> minimization engine (minimization.engine=scipy) (#1581)- New masking parameter
disable_parallax_correction=False. Set toTrueto speed up generation of resolution masks by disabling parallax correction (this is only likely to have significant effect when spotfinding is spread across many independent processes). (#1590) dials.image_viewer: New parameterbasis_vector_scale=to adjust the length of the basis vector overlay (#1598)dials.merge: add option to set wavelength_tolerance for MAD datasets (#1609)dials.reciprocal_lattice_viewer: Added an option to label the reciprocal lattice point nearest the centre (#1614)dials.scale: An additional outlier rejection based on normalised intensities has been added (#1627)
dials.image_viewer: Fix various display issues relating to viewing still images (#1463)dials.background: Fix crash when writing output plot with bad display configuration (#1550)dials.scale: Fix issue of error model not always being carried through after the profile/summation intensity combination step. (#1566)- Fail bootstrap step if the git checkout fails in a non-interactive or non-posix environment (#1572)
- Fixes working towards direct support of Windows builds:
- Describe how to fix gltbx build failures for development installations on non-RHEL distributions (#1561)
- Replace references to
.picklewith reflections /.reflin docstrings (#1619) - Add documentation for
dials.filter_reflectionsto the website. (#1625)
- Remove previously deprecated
use_trusted_range=parameter from masking configuration (#1156) - The main development branch of dials was renamed from 'master' to 'main'. (#1546)
dials.background: Theplot=parameter to interactively display the background plot has been removed. Useoutput.plot=to save to file instead. (#1554)- Remove
*.ofiles from the DIALS installer package (#1564) dials.util.masking.MaskGeneratoris deprecated in favour ofdials.util.masking.generate_mask(#1569)
- #1530, #1531, #1532, #1534, #1535, #1536, #1542, #1567, #1570, #1571, #1588, #1593, #1597, #1599, #1600, #1601, #1603, #1604, #1613, #1620, #1621, #1624, #1626, #1630
dials.import: Selecting a subset of images withimage_range=now works on stills (#1592)- `dials.search_beam_centre`: Dramatically improved execution time for large data sets (#1612)
dials.reindex: Write.reflfile output in the default "MessagePack" format for better compatibility with downstream programs (#1616)dials.scale: Fix rare memory crash from infinite loop, that could occur with very bad quality datasets (#1622)
dials.refine: More informative error message when reflections have weights of zero (#1584)
No changes to core DIALS in 3.3.3.
- Remove unnecessary call to
imageset.get_raw_data()while generating masks. This was causing performance issues when spotfinding. (#1449) dials.export: Allow data with either summation or profile fitted intensities to be exported. Previously, both were (erroneously) required to be present. (#1556)dials.scale: Fix crash if only summation intensities present andintensity_choice=combine(#1557)- Fix unicode logging errors on Windows (#1565)
dials.index: More verbose debug logs when rejecting crystal models that are inconsistent with input symmetry (#1538)
dials.stills_process: Fix spotfinding error "Failed to remap experiment IDs" (#1180)- Improved spotfinding performance for HDF5 when using a single processor. (#1539)
- DIALS is now using GEMMI. (#1266)
- Upgrade
h5pyrequirement to 3.1+ for SWMR-related functionality. (#1495) - Added support for small integer types to DIALS flex arrays. (#1488)
dials.estimate_resolution: Only use cc_half in default resolution analysis. (#1492)dials.export: Allow on-the-fly bzip2 or gzip compression for mmCIF output, because unmerged mmCIF reflection files are large. (#1480)dials.find_spotsanddials.integrateboth now havenproc=Autoby default, which uses the number of allowed/available cores detected. (#1441)dials.merge: Report<dF/s(dF)>, ifanomalous=True. An html report is also generated to plot this statistic. (#1483)dials.scale: Apply a more realistic initial error model, or load the existing error model, if rescaling. (#1526)dials.stills_process: allow using different saturation cutoffs for indexing and integration. Useful for using saturated reflections for indexing while still rejecting them during integration. (#1473)
- Internal: Logging metadata is now preserved when running spotfinding and integration across multiple processes. (#1484)
- Fix NXmx behaviour with h5py 3.1. (#1523)
dials.cosym: Choose the cluster containing the most identity reindexing ops by default. Under some circumstances, particularly in the case of approximate pseudosymmetry, the previous behaviour could result in reindexing operators being chosen that weren't genuine indexing ambiguities, instead distorting the input unit cells. (#1514)dials.estimate_resolution: Handle very low multiplicity datasets without crashing, and better error handling. (#1494)dials.export,dials.two_theta_refine: Updates to mmcif output to conform to latest pdb dictionaries (v5). (#1528)dials.find_spots: fix crash whennproc=Auto. (#1019)dials.image_viewer: Fix crash on newer wxPython versions. (#1476)dials.index: Fix configuration error when there is more than one lattice search indexing method. (#1515)dials.merge: Fix incorrect output of SigF, N+, N- inmerged.mtz. (#1522)dials.reciprocal_lattice_viewer: Fix error opening with wxPython 4.1+. (#1511)dials.scale: fix issues for some uses of multi-crystal rescaling iffull_matrix=False. (#1479)
- Update information on how to care for an existing development environment, and remove outdated information. (#1472)
- Each of the available indexing strategies in
dials.indexnow has some help text explaining how it works. You can view this help by callingdials.index -c -a1 -e1and looking formethodunderindexing. (#1519) - Include
__init__methods in autodoc generated library documentation. (#1520) dials.estimate_resolution: Improved documentation. (#1493)
dials.algorithms.spot_finding.finder.SpotFinder: Use of__call__to run spotfinding has been deprecated in favor ofSpotFinder.find_spots(experiments). (#1484)
dials.slice_sequence: Fix crash usingblock_size=option (#1502)dials.scale: Fix brokenexclude_images=option (#1509)
- Fix case where
dials.stills_processcould swallow error messages dials.cosym: Fix non-determinism. Repeat runs will now give identical results. (#1490)- Developers: Fix precommit installation failure on MacOS (#1489)
3.2 Branch releases will now use a fixed conda environment. This release is the first to use the same versions of all dependencies as 3.2.0.
dials.symmetry,dials.cosymanddials.two_theta_refine: Lowered default partiality_threshold from0.99to to0.4. The previous default could occasionally result in too many reflections being rejected for particularly narrow wedges. (#1470)dials.stills_processImprove performance when using MPI by avoiding unnecessary log file writing (#1471)dials.scale: Fix scaling statistics output of r_anom data. (#1478)
- DIALS development environments are now running Python 3.8 by default. (#1373)
- Add a scaled flag to the reflection table. Indicates which reflections are good after the scaling process. (#1377)
- Python warnings are now highlighted on the console log and written to log files (#1401)
- Exit error messages from commands will now be colourized (#1420)
- Change the way
dials.integratesplits data into blocks, to reduce unnecessary data reads, increasing performance up to 35% in some cases (#1396) - Add
dials.util.mp.available_coresfunction (#1430) dials.refine: Trimming scans to observations for scan-varying refinement can now be turned off, using the parametertrim_scan_to_observations=False(#1374)dials.refine: Change default toseparate_panels=False. This speeds up outlier rejection for multi-panel detectors. For metrology refinement this should be set toTrue(#1424)dials.merge: Add best_unit_cell option. If the best_unit_cell option is set indials.scale, this will now propagate to the merged mtz output file. (#1444)- DIALS bootstrap now allow creating a Python 3.9 environment (#1452)
- DIALS now uses pytype for limited static type checking. We hope that this will, over time, improve code quality. (#1364)
dials.stills_process: Addedprocess_percent=to restrict processing to a subset of data, sync reference geometry instead of overwriting it and handle composite spotfinding modes. (#1409)
dials.stills_process: Prevent memory usage getting too high by clearing the imageset cache during processing. (#1412)dials.find_spots_server: Return HTTP 500 instead of 200 when running fails (#1443)dials.find_spots_server: Fix multiprocessing-related crash on macOS with Python3.8 (#1447)dials.integrate: Fix failures when building with GCC 9 (#1456)dials.image_viewer: Fix deprecation warnings (#1462)dials.index: When using local index assignment, take into account phi in nearest neighbour analysis. This can significantly improve indexing rates in some cases with scans > 360° (#1459)dials.reindex: Show an error instead of crashing for bad reindex operations. (#1282)
- dials.refine: the parameter
trim_scan_edgesis renamedscan_marginand the former name is deprecated (#1374) - The developer command
dev.dials.show_test_failure_reasonswas removed. (#1436) - Remove clipper sources from new development installations (#1437)
- #1175, #1337, #1354, #1379, #1381, #1400, #1406, #1416, #1423, #1426, #1432, #1433, #1435, #1446, #1454, #1466, #1468
No changes to core DIALS in 3.1.4.
dials.integrate: fix integrator=3d_threaded crash if njobs > 1 (#1410)dials.integrate: Check for and show error message if shoebox data is missing (#1421)dials.refine: Avoid crash for experiments with zero reflections if the auto_reduction.action=remove option was active (#1417)
dials.merge: improve help message by adding usage examples (#1413)dials.refine: More helpful error message when too few reflections (#1431)
dials.stills_process: Add parametermax_images=to limit the number of processed images
dials.integrate: fix crash when run with integrator=3d_threaded (#1404)dials.integrate: Minor performance improvements (#1399)dials.stills_process: MPI performance improvements for large datasetsdials.stills_process: Fix error when using split logs
dials.scale: Prevent discarding of resolution limits in rare cases, which could cause incorrect symmetry determination, and worse results. (#1378)dials.cosym: filter out experiments with inconsistent unit cells (#1380)- Internally slicing experiments now works if image range doesn't start at 1 (#1383)
- Restore missing I/sigma(I) resolution estimate log output (#1384)
dials.image_viewer: "Save As" button now works, for single panels- Fix developer
libtbx.precommitinstallation error (#1375)
- Supports Python 3.7 and 3.8. Python 3.6 remains the default. (#1236)
- Switch DIALS environment to use conda compilers. For development environments, a new
dialsscript, located above the build directory, replaces the existing 'setpaths'-family of scripts. This means that all commands within the conda environment will now be available. (#1235) - New command:
dials.missing_reflectionsto identify connected regions of missing reflections in the asymmetric unit. (#1285) - Improvements to image stacking in
dials.image_viewer:- add pull-down selector for stacking mode
- add modes for mean and max
- add command-line selection for stacking mode
- rename
sum_imagescommand-line option tostack_images(#1302)
- Reduce volume of output in
dials.integrate; histograms and other less important information only shows in debug output. Pass the-vvoption to restore the previous behaviour (#1319) dials.integrate: Experimental feature: Specifyingoutput_unintegrated_reflections=Falsediscards unintegrated data from output reflection file, for smaller output and faster post-processing (#1343)- Rename
dials.resolutionizercommand todials.estimate_resolution, and includes a html report. Writing png plot output is now turned off by default (passingplot=Truewill restore this behaviour). (#1330) dials.scalenow separates anomalous pairs during error model analysis (#1332)dials.background: Add parametercorrected=to optionally use pedestal-and-gain corrected data (#1348)dials.combine_experiments: Add optionoutput.max_reflections_per_experiment=, to reject experiments with too many reflections (#1369)
dials.image_viewer: Shoeboxes are now shown rotated with rotated detector panels. (#1189)dials.index: In multi-lattice indexing, ensure that reflections where refinement fails are flagged as unindexed. (#1350)dials.scale: Reflections excluded from scaling are no longer permanently excluded from any subsequentdials.scalejobs. (#1275)dials.scale: When usingintensity_choice=combine(the default), don't exclude reflections that only have one of summed or profiled intensities available, but not both. (#1300)dials.split_experiments: Don't generate extra leading zeros in the output filename when not required e.g.output_09.expt->output_9.expt(#1316)dials.plot_reflections: Fix invisible white spots on white background. (#1346)
dials.find_spots: Deprecatespotfinder.filter.use_trusted_range=(#1156)setpaths.shand related scripts in newly created DIALS development environments are made obsolete and will no longer work. (#1235)dials.show: Removeshow_image_statistics=parameter. Useimage_statistics.show_raw=for equivalent output (#1306)- Log files will omit timings unless the relevant dials program was run with
-v(#1313)
dials.scale: Allow usage ofmode=image_groupwithfiltering.method=deltacchalfwhen only providing a single data set (#1334)dials.import: When using a template and specifying an image_range, missing images outside of the range will not cause a failure (#1333)dials.stills_process: Show better error message in specific spotfinding failure case (#1180)
- Developer tool: On posix systems, sending SIGUSR2 to DIALS commands will now print a stack trace (#1277)
- HTML reports: Plot bin centres instead bin minimum for d_min line plots vs. resolution (#1323)
dials.export: Fix inconsistency in mtz export when given a non-reference (e.g. I2 or primitive) setting (#1279)dials.refine_bravais_settings: Fix crash with large (>2gb) reflection tables and reduce memory use (#1274)dials.scale: Fix bug in outlier rejection code causing misidentification of outliers (with outlier_rejection=standard).dials.scale: Fix outlier rejection formula to avoid overconfidence in spuriously low values
- Fix crash in scaling error model handling (#1243)
- dials.reciprocal_lattice_viewer: Add an option to show lattice(s) in the crystal rather than laboratory frame. (#1259)
- Support for mtz project_name in export and scaling
- dials.reciprocal_lattice_viewer: fix multiple experiment view for integrated data (#1284)
- Show more useful output when crashing in C++ code (#659)
- dials.image_viewer: for the unit cell tool, rename parameters for consistency and add a new show_hkl option to filter displayed powder rings to select only those of interest. (#1192)
- In dials.integrate: changed the background box size multiplier to be a parameter (sigma_b_multiplier) - setting to small values significantly reduces memory requirements. (#1195)
- dials.image_viewer: add an overlaying showing pixels marked as strong by the spot-finding operations. That is, the pixels picked out by the "threshold" image. (#1200)
- dials.scale report file was renamed from scaling.html to dials.scale.html dials.symmetry report file was renamed from dials-symmetry.html to dials.symmetry.html (#1202)
- dials.report output file was renamed from dials-report.html to dials.report.html (#1206)
- dials.image_viewer: faster navigation between different image types. (#1213)
- Crystal model now has a new recalculated_unit_cell attribute. This allows it to store a post-refined unit cell (e.g. from dials.two_theta_refine) in addition to that from traditional geometry refinement (which was used for prediction). Downstream programs such as dials.scale and dials.export will now use the recalculated unit cell where appropriate. (#1214)
- New best_monoclinic_beta parameter for dials.refine_bravais_settings and dials.symmetry. Setting this to False will ensure that C2 is selected in preference to I2, where I2 would lead to a less oblique cell (i.e. smaller beta angle). (#1226)
- New scaling model, model=dose_decay, implementing a shared exponential decay component for multicrystal experiments (#1183)
- Avoid empty "Unable to handle" messages on failed dials.import (#600)
- Functions from dials.export now raise exceptions on errors rather than exit. This improves their use elsewhere (such as in dials.scale). (#1205)
- Ensure dials.index chooses the C2 setting with the smallest beta angle (#1217)
- Fix propagation of best_unit_cell and application of resolution cutoffs in dials.scale and export_mtz. Add a new mtz.best_unit_cell parameter to dials.export (#1248)
- Make some of the DIALS tools furthest downstream (
dials.scale,dials.symmetry,dials.mergeanddials.report) more robust in the case of very few reflections. (#1263)
- dials.image_viewer: Add a choice between displaying the raw or the corrected image. (#634)
- Automatically choose between the simple and glm background determination algorithms, depending on whether the detector appears to be integrating or counting. (#706)
- Allow adjustment of font size for overlay text, such as Miller indices and resolution ring values. (#1074)
- Keep goniometer and scan objects in indexing of still data, if provided in input (#1076)
- Experimental:
dials.image_viewercan be remotely controlled via a ZeroMQ endpoint with thezmq_endpointPHIL parameter. Initially, the viewer can be commanded to load new images. This requires the (optional)pyzmqpackage. (#1085) - Programs now generate a unique identifier for each experiment created, and reflection tables are linked via the experiment_identifiers map (#1086)
- Introduce dials.anvil_correction to correct the absorption of the incident and diffracted X-ray beam by the diamond anvils in a pressure cell. Call dials.anvil_correction on the output of dials.integrate and then proceed to use post-integration tools as normal, just as though the sample had been measured in air. (#1090)
- Map of detector efficiency for photon counting detectors as a function of detector position added to report, based on the qe value applied at the end of integration. (#1108)
- Significantly reduce the amount of memory required to write .refl output files (#1115)
- Add maximum_trusted_value=N option to spot finding to temporarily allow override of trusted range, e.g. to find overloaded spots in spot finding. (#1157)
- array_family.flex interface has changed: background and centroid algorithms are set via public properties. Instead of flex.strategy use functools.partial with the same signature. as_miller_array() raises KeyError instead of Sorry. .extract_shoeboxes() lost its verbosity parameter, use log levels instead. (#1158)
- dials.stills_process now supports imagesets of length > 1 (e.g. grid scans) (#1174)
- Fixed prediction on images numbered zero, so integrating works correctly. (#1128)
- Fix an issue (#1097) whereby aggregating small numbers of reflections into resolution bins could sometimes result in empty bins and consequent errors. (#1130)
- Ensure that restraints are ignored for parameterisations that are anyway fixed (#1142)
- Fix dials.search_beam_centre to ensure that the correct detector models are output when multiple detector models are present in the input. Fix dials.search_beam_centre n_macro_cycles option (previously it was starting from the original geometry every macro cycle). (#1145)
- dials.find_spots_server no longer slows down 3x when using resolution filters (#1170)
- We now fully support Python 3 environments.
- MessagePack is now the default reflection table file format. Temporarily, the environment variable
DIALS_USE_PICKLEcan be used to revert to the previous pickle-based format, however this will be removed in a future version. (#986) - new option for dials.show 'show_shared_models=True' displays which beam, crystal, and detector models are used across experiments (#996)
- Import still image sequence as N experiments dereferencing into one image set rather than one experiment. (#1014)
- Add reflection_table.get method for defaulted column access (#1031)
- Don't use -2 to indicate masked pixels, except for DECTRIS detectors where this is to be expected. (#536)
- No longer show pixels that are above the trusted range upper bound as "saturated" on the "variance" image. (#846)
- Correctly account for scan-varying crystals while providing a scan range to dials.integrate (#962)
- Ensure that generated masks do not include pixels that are overloaded on a few images, but only pixels that are always outside the trusted range. (#978)
- Rewritten parameter auto-reduction code for dials.refine provides finer-grained fixing of individual parameters rather than whole parameterisations and correctly takes constrained parameters into account (#990)
- Fix output of predictions in dials.refine. A recently-introduced bug meant that the updated predictions weren't being copied to the output reflections file. (#991)
- Allow scan-varying refinement where either the crystal cell or orientation is fixed. (#999)
- Respect batch= option to dials.symmetry - can reduce time taken for finding the symmetry for large data sets. (#1000)
- Scan-varying refinement no longer fails when the scan is wider than the observed reflections (e.g. when the crystal has died). Instead, the scan is first trimmed to match the range of the diffraction. (#1025)
- If convert_sequences_to_stills then delete the goniometer and scan. (#1035)
- Correctly account for scan-varying crystals in dials.slice_sequence (#1040)
- Eliminate systematic absences before applying change of basis op to minimum cell in dials.symmetry. (#1064)
- Add "Extending DIALS" page to developer documentation (#893)
- The command dials.analyse_output was removed. Its replacement, dials.report, will give you more useful output. (#1009)
- Support exporting multi-dataset and still experiments to XDS_ASCII (#637)
- Replace default spotfinder with improved dispersion algorithm (#758)
dials.reportnow displays oscillation data with units and more significant figures (#896)- A new program,
dials.sequence_to_stillsto create split a sequence into a separate still Experiment for every scan point in the sequence, splitting reflections as necessary. (#917) - Moved
dials.export format=besttodials.export_bestas that one needed access to the format object, the rest do not, and havingdials.exportwork in the general case seems like a better idea... (#921) - Unified logging output for dials programs - logs are no longer split into .log and .debug.log. Use -v to get debug output. (#923)
- New command
dials.resolutionizer(replacesxia2.resolutionizer). Add support forexpt/reflindials.resolutionizer. (#933) - Changed the selection of reflections used for determination of the reflection profile parameters in integration. Now uses reflections which were previously used in refinement rather than all reflections, resulting in a speed improvement for large data sets and a negligible difference in the quality of the integrated results. (#942)
dials.image_viewernow allows the choice betweendispersion_extended(new default) anddispersion(old default) thresholding algorithms for investigating the effect of different spot-finding parameters. (#948)dials.rs_mappernow respects masked regions of images (including the trusted range mask). (#955)
- Fix and reinstate normalisation option in
dials.option(#919)