yt 4.1.0

Summary

This is the first feature release in the yt 4.x series.
It contains about a year of work by many contributors, with many new features and improvements to the code base. It is also the first release to officially support Python 3.11
Some deprecations from yt < 4 are now expired, some minor backwards incompatibilities are expected. Anything broken on purpose in this version should correspond to a warning in yt 4.0.x, anything beyond that should be considered a regression; please report any unexpected breaking change.

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

Note that binaries for Python 3.11 are only available via pip.

🌟 Highlights

Newly supported data formats

yt 4.1 ships with 3 brand new frontends !

• #1990 CFRadial, by @zssherman and @chrishavlin
• #3638 Chimera, by @Ronan-Hix
• #3663 Cholla, by @chummels , @evaneschneider , and @matthewturk

Improved loading for "ambiguous" data formats

yt.load sometimes raises YTAmbiguousDataType when more than one data format appears to be valid. Ideally this should never happen, but in some cases it is very frequent, in particular with BoxLib data, for which we support many hard-to-distinguish flavours.
In order to mitigate this issue, yt.load now accepts a hint keyword argument. For instance

yt.load("det_x_plt00000", hint="castro")

Stretched grid support

yt now has experimental support for "stretched" grid schemes (grids with arbitrarily varying cell width).
It is currently not compatible with AMR.

The following contains a non-exhaustive list of changes

🐛 Bug fixes

#2538 Combining set_zlim and set_unit methods doesn't work as expected (fixed in #3849, by @neutrinoceros )
#3901 Cannot plot data containing both finite and infinite values and use set_zlim (fixed in #3849, by @neutrinoceros)
#3986 Restore and fix flexible linear threshold guessing in symmetric logarithmic plots, by @chrishavlin (issue #3944)
#4021 Fix multiple issues with magnetic field derived fields definitions, by @neutrinoceros
#4075 Fix scaling in off-axis projections of filtered gas particles, by @jzuhone
#4098 Correcting deceptive typo in an error message, by @chummels
#4149 Fix a bug in the computation of metallicity for the generation of X-ray emission fields, by @geoyuqiu

🚀 New features

#3346 Allow to set units from config file, by @cphyc
#3421 Allow stream frontends to accept callables, by @matthewturk
#3443 Directly open compressed files¹, by @cphyc
#3640 Add swap_axes, flip_horizontal and flip_vertical plot modifier methods, by @chrishavlin
#3666 Implement hint keyword argument for yt.load to help lifting ambiguities in data formats, by @neutrinoceros (issues #3510 by @mameehan5 and #3005 by @zingale)
#3686 Allow 'factor' argument in plot annotation methods to accept 2-tuples to distinguish x/y resolutions, by @neutrinoceros
#3756 Expose FRB filtering API, by @cphyc and @neutrinoceros
#3812 Allow for different normalizations of magnetic fields in CGS², by @jzuhone (issue #3471 by @forrestglines)
#3822 Add support for varying colors in quiver annotations, by @neutrinoceros
#3940 Add "min" method to ProjectionPlot, by @rjfarber
#3489 yt.ProjectionPlot can now perform off-axis projections³, by @neutrinoceros (also #3723 and #3737)
#3817 Add support for VelocityCallback and MagFieldCallBack in spherical coordinates, by @neutrinoceros
#3989 Add support for projecting a field's standard deviation, by @jzuhone
#4063 Add data_source argument to covering grid, by @chrishavlin
#4119 Allow passing a dataset as a data source to ParticlePhasePlot, by @neutrinoceros
#4121 Add public API render methods to PlotContainer and FixedResolutionBuffer, by @neutrinoceros
#4142 ENH: add API to switch raytracing engine at runtime, by @neutrinoceros

experimental features

#2567 First cut at showing multiple transfer functions on VR, by @zingale
#2998 Enable variable cell widths for grid datasets #2998, by @matthewturk
#3849 Add ability to use colorbar norms other than linear, log and symlog, by @neutrinoceros (issue #3840)

♻️ Optimisations and refactors

Performance optimisations

#2416 Stop preallocating particles, by @matthewturk and @chrishavlin, issue #2383 by @zhafen
#3439 Cache _determine_fields for a given dataset by @Xarthisius
#3469 Optimise memory layout for particle oct container, by @themousepotato
#4029 Only compute field suggestions if YTFieldNotFound isn't caught, by @neutrinoceros
#3935 Import optional dependencies lazily⁴, by @neutrinoceros
#4025 Optimise startup time, by @neutrinoceros
#4037 First steps toward utilizing the octree infrastructure as an engine for block-structured data, by @BolunThompson
#4071 Optimise some grid selection operations by skipping unnecessary symbolic unit computations, by @yipihey
#4079 Optimise radius calculation, by @yipihey (+ #4091 by @brittonsmith)

User experience and developer experience improvements

#3065 Make Scene.save less verbose, by @neutrinoceros
#3436 Improve clarity of error message for region slicing with 2D dataset, by @neutrinoceros
#3495 Warn in case a Dataset subclass is redefined, by @neutrinoceros
#3519 Improve user input validation for PLViewerMPL._setup_origin, by @neutrinoceros
#3525 Allow direct dataset instantiation using the user path '~' glyph, consistently with yt.load, by @neutrinoceros
#3527 Improve formatting for YTUnidentifiedDataType exception, by @neutrinoceros
#3621 Improve verbosity of yt.enable_parallelism in case of error, by @neutrinoceros

Quality improvements and refactors

#3526 Particle reader cleanup, by @matthewturk
#3546 Setup basic type checking and fix existing type errors, by @neutrinoceros
#3617 Factorise coordinate handler, by @cphyc
#3624 Factorise code in halo containers, by @cphyc
#3626 Decouple yt.config from initialisation⁵, by @neutrinoceros
#3772 Avoid storing relative paths in Dataset attributes, by @neutrinoceros
#3920 Improve formatting consistency in YTException error messages, by @neutrinoceros
#3957 Refactor plot callbacks registration, by @neutrinoceros (issue #3945)
#3998 Update to fields used in Enzo answer testing, by @jwise77
#4116 Avoid subclassing builtin dict, by @neutrinoceros
#4057 Cleanup dead attribute Dataset.file_style, by @neutrinoceros
#4125 Stop using deprecated np.int, by @neutrinoceros
#4143 Updating Firefly-dependent areas to match API changes and PyPI packaging, by @mtryan83

📚 Documentation

#3395 and #3412 Avoid using pylab in docs, by @neutrinoceros
#3608 Document how Dataset._is_valid works, by @neutrinoceros (issue #1018 by @ngoldbaum)
#3627 Remove a false statement from yt.funcs.rootonly's docstring, by @neutrinoceros (issue #2054 by @DTHaydon)
#3642 Fix a broken example (change of api in the Athena frontend), by @neutrinoceros (issue #3641)
#3813 Update contribution guidelines, by @neutrinoceros (issue #2554 by @munkm)
#3926 Update broken link to re-licensing blog post, by @tracykteal
#3982 Fix broken annotate_particles example, by @chrishavlin (issue #3968 by @neutrinoceros)
#4080 Document plot callbacks extensibility, by @neutrinoceros (issue #4072)
#4124 Update instructions and redirections for installing cartopy, by @neutrinoceros
#4148 Add more info to magnetic fields documentation, by @jzuhone

🤖 Frontends

Frontend-specific improvements and bug fixes

[AMRVAC]
#3508 Optimise reading speed, by @neutrinoceros

[AREPO]
#3815 Add three new derived fields, by @jzuhone
#3258 New fields for Arepo and Gadget frontends, by @jzuhone
#4120 Add Arepo stellar photometric fields , by @jzuhone

[BoxLib]
#3515 Add a missing close statement, by @mameehan5
#4112 allow reading of 1D particle data, by @dpgrote

[Enzo-E]
#3914 Update Enzo-E particle mass/density distinction, by @brittonsmith
#4097 Store the Enzo-E version number in parameters attribute of EnzoEDataset, by @mabruzzo
#4099 Improved Enzo-E frontend's determination of fluid properties, by @mabruzzo

[Gadget]
#3258 New fields for Arepo and Gadget frontends, by @jzuhone
#3455 Make sure we add the Parameters group to hvals if it's present, by @jzuhone

[GAMER]
#4070 Use lookback time in cosmological simulation, by @koarakawaii

[RAMSES]
#3332 Allow user customization of RAMSES field loading, by @cphyc

💣 Deprecations

Expired deprecations and known-breaking changes

#3240 Deprecation cycle (4.0 -> 4.1), by @neutrinoceros
#3957 Deprecate dict argument plot_args in annotate_* methods where a **kwargs is sufficient, by @neutrinoceros
#3970 Turn ambiguous fields warnings into errors[^8], by @neutrinoceros (issues #3416, and #3235 by @cphyc)

#4142 changed the default ray tracing engine from "embree" to "yt" (builtin engine). Previous versions of yt would switch to embree at startup if pyembree was installed. This version requires that embree be explicitly required, either with configuration file:

[yt]
ray_tracing_engine = "embree"

or at runtime

from yt.visualization.volume_rendering.api import set_raytracing_engine
set_...

1. this feature requires ratarmount as an optional dependency to yt, currently only Linux is supported, see #3749 ↩

2. at the moment this is only used in Athena and Athena++ frontends ↩

3. yt.SlicePlot is now a class (previously it was a function). Instantiating OffAxisProjectionPlot (resp OffAxisSlicePlot) directly is now unnecessary (but still supported) ↩

4. with the exception of netCDF4, which is still imported at startup time if available ↩

5. a consequence of this patch is that yt will no longer create configuration files on startup if it can't find one ↩

yt 4.0.5

Summary

This is the fifth bugfix release in the 4.0.x series.
This release fixes a critical bug with SPH particle selection in bounding boxes.

This release contains no new deprecation or removal and should be fully backward compatible, we recommend all users to upgrade.

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

Changes

• BUG: Broadcast missing attribute #3973 by @brittonsmith
• BUG: Fix broken annotate_particles example #3982 by @chrishavlin (issue #3968 by @neutrinoceros)
• BUG: volume rendering: sigma_clip normalization #3984 by @chrishavlin (issue #3979 by @zingale)
• BUG: fix a ResourceWarning (file is never closed) #3997 by @neutrinoceros
• BUG: fix for composite volume rendering with octree sources #4011 by @chrishavlin
• BUG: fix a typo in warning message #4017 by @neutrinoceros
• BUG: setting GeoAxes extent #4040 by @chrishavlin (issue #4039)
• BUG: Fix flux calculation on surface when there is no fluxing_field #4042 by @jzuhone
• BUG: Fix intersections between SPH particles and regions when the boundary is periodic #4050 by @jzuhone (issue #3916 by @benopp99)

Requirements

Python 3.6 to 3.10 is supported. The next feature release, yt 4.1, will require Python 3.7 or newer.

yt 4.0.4

Summary

This is the fourth bugfix release in the 4.0.x series.
This release fixes a critical incompatibility with the upcoming release of Matplotlib (3.6). This should have no immediate effect but allow yt to be installed alongside Matplotlib 3.6.
It is our first release to be distributed for Mac ARM (M1) on PyPI.

This release contains no new deprecation or removal and should be fully backward compatible, we recommend all users to upgrade.

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

Changes

• BUG: fix compatibility with GADGET2 files using '.gad.' template #3899 by @neutrinoceros
• BUG: fix an UnboundLocalError #3898 by @neutrinoceros
• BUG: fix an upcoming incompatibility with matplotlib 3.6 #3912 by @neutrinoceros
• BLD: fix a build error on Windows #3925 by @neutrinoceros

Requirements

Python 3.6 to 3.10 is supported. The next feature release, yt 4.1, will require Python 3.7 or newer.

yt 4.0.3

Summary

This is the third bugfix release in the 4.0.x series.
This release fixes a critical bug in particle indexing, as well as many small 2D visualisation bugs. It also contains a couple frontend specific bugfixes, most notably for RAMSES.
It is our first release to be distributed for Mac ARM (M1) on conda-forge.

This release contains no new deprecation or removal and should be fully backward compatible, we recommend all users to upgrade.

How to upgrade

To upgrade from PyPI, run

python -m pip install --upgrade yt

or, with conda

conda update --channel conda-forge yt

Changes are categorised below.

Particle Indexing

Fix a bug in bitmap index for particle datasets #3788 by @langmm . This was the result of some checks of the number of refined zones in the bitmap index being set to higher than it should be.

This bug was responsible for some particle data files not being selected correctly, which manifested e.g. as "holes" in SlicePlots, and were first identified with AREPO data (see issue #3672 by @jzuhone)

2D Plotting

Symlog colorbars (used for plotting data with negative values)

We made some adjustments in how yt internally chooses to use symlog over log norms for 2D plots.

• BUG: add missing MPL version check to a temporary hack #3754 by @neutrinoceros (issue #3170 by @chrishavlin)
• BUG: fix heuristic to switch from log scale to symlog scale #3793 by @neutrinoceros (issue #3791)
• BUG: fix a bug where automatically computed symlog linear threshold would sometimes cause a crash in matplotlib #3859 by @neutrinoceros (issue #3858, original report by @BenWibking)
• BUG: fix a bug where setting zmin/zmax on a 2D plot with symlog norm would cause an undesired change in linthresh #3888 by @neutrinoceros (issue #3887 by @vdbma)

Plotting non-cartesian data

• BUG: fix pixelize_cylinder #3782 by @neutrinoceros (issue #3781)
• BUG: fix a bug in pixelize_cylinder where buffer shape was read backwards #3795 by @neutrinoceros (issue #3794)
• BUG: don't colorize pixels that are not completely within the data domain in pixelize_cylinder routine #3818 by @neutrinoceros
• BUG: fix plot bounds for slices in cylindrical coordinates #3820 by @neutrinoceros (issue #3819)
• BUG: fix a bug in aitoff-hammer pixelizer where some pixels were skipped on the central meridian and the equator #3824 by @neutrinoceros (issue #3823)
• BUG: fix a bug where PlotWindow origin parameter wasn't handled correctly in non-cartesian geometries #3863 by @neutrinoceros (issue #3862 , original report by @vdbma)

Misc

• Phase plots are now properly masked #2504 by @matthewturk
• Format error string #3780 by @Yash-10
• BUG: fix a bug in masking invalid values in line integral convolution callback #3739 by @neutrinoceros (issue #3738)
• BUG: fix an issue where missing files would be indexed without verification #3816 by @neutrinoceros (issue #2819 by @joostvan)
• BUG: fix a typesetting bug with fractionnal units in colorbar labels for volume rendering #3755 by @neutrinoceros
• BUG: fix buffer masking in cartesian geometry #3856 by @neutrinoceros (issue #3854)
• BUG: fix a regression where calling annotate_timestamp on a plot from a dataset with code units would crash #3870 by @neutrinoceros (issue #3869, original report by @vdbma)

Frontend specific patches

Flash

• BUG: Track blockless particles #3683 by @matthewturk

OWLS

• Fix Electron number density field #3874 by @jzuhone (issue #3838 by @cphyc)

RAMSES

• BUG: also validate unresolved symlinks (RAMSES) #3786 by @mtrebitsch (issue #3785)
• BUG: fix error messages in RAMSESDataset validation #3801 by @neutrinoceros (issue #3800)
• BUG: bring back support to group folder structure in RAMSES #3811 by @cphyc (issue #3785 by @mtrebitsch)
• Add missing ionized species #3841 by @cphyc
• Fix definition of particle age/times #3864 by @cphyc (issue #3861 by @lconaboy)
• BUG: Use cm_per_mpc instead of 3.08e24 #3868 by @Iconaboy (issue #3867)

Requirements

Python 3.6 to 3.10 is supported. The next feature release, yt 4.1, will require Python 3.7 or newer.

Some small adjustments to yt's dependencies were done for this release, and listed here to the intention of downstream packagers

• The unmaintained toml package isn't required anymore, yt now require tomli and tomli_w instead (issue #3830 by @henryiii)
• pyyaml and IPython were previously declared as hard requirements when they are in fact not strictly necessary. pyaml is now used in tests instead of pyyaml, and IPython is now considered an optional dependency.
• jinja2 is now pinned in docs builds (#3860 by @Xarthisius)

yt 4.0.2

Summary

This is the second bugfix release in the 4.0.x series.
Most notably, this release fixes a number of compatibility issues with recent versions of Matplotlib (3.4 and 3.5). It is also the first release to officially support Python 3.10.

This release contains no new deprecation or removal and should be fully backward compatible, we recommend all users to upgrade.

How to upgrade

To upgrade via PyPI, run

python -m pip install --upgrade yt

or via conda-forge

conda update --channel conda-forge yt

Most impactful changes are categorised below. See the 4.0.2 milestone for a an comprehensive view of the release's content.

Matplotlib compatibility

typesetting issues

• BUG: Set the display name for new fields #3282 by @chummels (issue #3269)
• BUG: fix default math font in yt plots against matplotlib >= 3.4 #3520 by @neutrinoceros (issue #3514)
• BUG: fix LaTeX typesetting for fractional ylabels in LinePlot #3637 by @neutrinoceros (issue #3636)

colorbar layout in plots using (sym)log norms

• BUG: fix two bugs with symlog colorbar ticks for matplotlib 3.5 #3556 by @neutrinoceros (issues #3554 and #3560)
• BUG: fix for matplotlib 3.5 + symlog + linthresh value outside data range #3565 by @neutrinoceros (issue #3564)
• MNT: matplotlib 3.5 compat #3670 by @neutrinoceros (issue #3558)

2D Plotting

• BUG: Checking if extrema are all None in PhasePlot #3432 by @chummels (issue #3431)
• BUG: fix a bug where 2D plots in spherical geometries would always use code-units in plot axes #3618 by @neutrinoceros (issue #2913 by @n-claes)
• BUG: fixes for Hammer-Aitoff projection (r-normal) 2D viz in spherical geometries #3628 by @neutrinoceros (issue #3610)
• BUG: fix figure layout in case the long axis is vertical #3633 by @neutrinoceros
• BUG: fix a bug where PlotContainer.save would wrongfully ignore specified field extension #3687 by @neutrinoceros (issue #3632)

Documentation

• DOC: add an exhaustive list of deprecations to the yt4differences page #3665 by @neutrinoceros (issue #3451)
• DOC: fix an error in AMRVAC specific documentation #3716 by @neutrinoceros

Frontend specific patches

Gadget

• Make sure we add the Parameters group to hvals if it's present #3455 by @jzuhone
• BUG: Take into account endianness when reading Gadget binary files #3688 by @cphyc (issue #3676 by @olebole)

Athena

These changes should actually not affect Athena users, they are only meant to open the design space for new frontends supporting vtk files (issue #3001)

• BUG: implement AthenaDataset._is_valid #3424 by @neutrinoceros
• ENH: loosen header matching for Athena vtk files to allow compatibility with single variable 'vtk outputs' #3490 by @neutrinoceros

GDF

• dimensionality doesn't imply which dimension is reduced #3630 by @Xarthisius (issue #3629)

Misc

• BUG: add support for setting up derived fields from lambdas #3440 by @neutrinoceros (issue #3434 by @forrestglines)
• BUG: fix combining FITS images #3456 by @jzuhone
• BUG: fix two bugs in yt.visualization.eps_writer #3496 by @neutrinoceros

Python 3.10 support

yt 4.0.2 is the first release for which we provide wheels for Python 3.10
We note that part of our test suite has yet to be ported from nosetest to pytest. Because nosetest isn't compatible with Python 3.10 (and has been unmaintained for years), this means that we currently run only a (large) subset of our tests with Python 3.10
In the unlikely event of a regression, please fill in a bug report

Requirements

For posterity, it may be worth noting that some of yt's minimal requirements were adjusted for this release:

• IPython 2.0 or newer is now required (previously 1.0)
• Matplotlib 2.1 or newer is now required (previously 2.0.2)
• cmyt (ColorMaps from yt) is a new small dependency that was extracted from yt's main code base, this should have no impact on yt users.

Python 3.6 is still supported for this release. The next feature release, yt 4.1, will require Python 3.7 or newer.