Skip to content

Tack 0.6.0

Choose a tag to compare

@berkgeveci berkgeveci released this 28 Oct 18:32
· 697 commits to main since this release
e13e67d

The headline: the dataset layer takes data in the shapes simulations write
it. Meshes in pieces, over devices and MPI ranks, through
tack.communication, a subset of DIY with Tack arrays as messages: ghost
cells and global ids, merging, probes and particles that cross pieces,
statistics over ranks, GPU-aware MPI on CUDA and HIP. Point clouds, with a
point locator, SPH, deposits, friends-of-friends halos and periodic boxes.
Block-structured AMR and octrees. Physical units on fields and parameters,
and time on datasets, with filters that stream steps. The filters are
written with public names only, so users can write their own (the
Visualization Guide's "Write a Filter" chapter). Around them: polyhedra and
polygons through probe, gradients and advection; voxelization, cell quality
and feature edges; a two-level cell locator that took the streamlines of the
ONERA M6 wing (46.9M cells) from minutes to tens of milliseconds on every
GPU, 20-33x faster than Viskores' on CUDA, each building its locator; a
prototype NVIDIA Warp backend; and a performance page against VTK and
Viskores.

Validated on Linux (CPU; CUDA on an RTX 4060 Ti and an RTX 6000 Ada; HIP on
two MI210s; Level Zero on an Intel Max 1100), macOS (CPU and Metal on an M1
Max) and Windows Server 2022 (CPU, and CUDA on a T4 in TCC mode), with debug
mode, both id widths of the dataset tests and MPI over 2 and 3 ranks,
together with tack-examples' ports, among them four production datasets (up
to 46.9M cells and 512 pieces). The dataset filters agree with VTK's at 10k
to 1M cells. Against 0.5, two known costs remain on Windows' CPU: filters
over about 10k cells are 5-22% slower on two threads, where the first
fan-out costs more than it saves, and converting a mesh from VTK is 1.2-1.6x
slower.

Added

  • A performance page in the documentation: Tack's filters against VTK and
    Viskores at 20M hexahedra and tetrahedra on the CPU and the GPU (an EPYC
    7453 with an RTX 6000 Ada, an M1 Max with its GPU), streamlines on CUDA
    against Viskores' on the ONERA M6 wing and an aneurysm, and four
    production datasets against VTK. Every Tack output is checked against
    VTK's before its time counts.
  • CellLocator(data, levels=1): the cells binned on one uniform grid of
    about density bins a cell, every bin a leaf, as Viskores'
    CellLocatorUniformBins and VTK's vtkStaticCellLocator bin them; the
    default, levels=2, is the two-level grid (Changed). VTK and Viskores let
    the caller choose the locator too. On a small, even mesh one level searches
    fewer candidates (tack-examples' fire port: 6.6 a location, against 12 for
    two levels); probe, advect and streamlines take it as locator=. On
    a mesh graded toward a body in a far field (the ONERA M6) a few bins hold
    most of the cells, and two levels are the choice. Both find the same cells
    and coordinates, bit for bit.
  • td.merge_points(data, tolerance=0): coincident points made one, as VTK's
    vtkStaticCleanUnstructuredGrid and Viskores' CleanGrid merge them.
    Points at most tolerance apart join, and so do chains of them; each set
    becomes its smallest point, with that point's position, point fields,
    ghost marks and global id, and the cells stay the same cells on the
    merged points. It is for meshes written without merging their points. A
    simulation's 512 pieces store each point about 2.6 times, so matching
    points across pieces numbered 16.9M of them; merged piece by piece first,
    generate_global_ids numbers 6,310,285, the merged mesh's count. Point
    counts match VTK's static clean, exactly and with a tolerance.
  • A "Write a Filter" chapter in the Visualization Guide: five built-in
    filters (point_elevation, threshold, shrink, jump, triangulate)
    and entropy over pieces rewritten with public names only, each section
    on what that kind of filter must keep (relative positions, the device,
    implicit arrays, quantities as parameters, units of results, ghost marks
    through carry, keep_marks or drop_marks, hidden cells, owned values
    over pieces). Its program, docs/examples/data_filters.py, holds every
    rewrite to the built-in in every part, and runs in validate.py.
  • tolerance= on generate_global_ids, ghost_cells and merge_pieces
    (VTK's vtkGenerateGlobalIds has one): points of different pieces at most
    that far apart are copies, and so are chains of them, for pieces whose
    copies of a shared point differ in their last bits. Points are binned into
    cells a little over two tolerances wide, each sent to its own cell and the
    ones its box reaches, through the same sort over the pieces as exact
    matching; owners then agree along chains by exchanges between partners.
    The default, 0, matches exactly as before. On a simulation's 512 pieces in
    f64, merged first with merge_points, a tolerance of 1e-12 numbers
    6,310,285 points (the merged mesh's), where exact matching numbers
    6,311,919, and gives VTK's ghost cell count.
  • What the built-in filters are written with is public, so a filter of
    your own is written the same way: td.carry and its maps (td.Same,
    td.Take, td.Pieces, td.Interpolate), td.keep_marks and
    td.drop_marks, td.visible, td.owned_values (which takes a field's
    name too), td.Set.from_flags (a selection a kernel flags, for
    extract_cells) and td.units (quantities as parameters; the units of
    differences, products and SI names). On the Data Model page. An audit
    that rewrote built-ins with public names only found three silent errors a
    user's filter made without them: a dataset built by hand dropped the
    record of lost ghost marks (reductions over pieces then counted shared
    points twice), a difference of degrees Celsius kept the unit, and a
    hidden cell was processed. test_public_filters.py keeps six such
    rewrites equal to the built-ins, every part of the result. td.carry is
    now the function; the module is still tack.dataset.carry for
    from ... import.
  • td.streamlines(..., step_unit="length"): steps of a fixed arc length
    along the velocity's direction, as VTK's vtkStreamTracer steps by
    default, with steps * step_size the longest path and time still the
    time the particle takes (1 / |v| integrated beside the path by the same
    weights). The default, "time", is unchanged. A time step sized for the
    fastest speed made tiny steps where the flow is slow: the carotid port's
    median step was 0.0005 long against VTK's 0.1, and GPU streamlines from
    a few dozen seeds were 4-100x VTK's time. With length steps the four
    streamline ports take VTK's steps, match its paths to 2e-7 (carotid
    2e-4), and run in 0.16-0.57x its time on 8 CPU threads.
  • A Visualization Guide chapter, Communicate Between Blocks, on writing
    an algorithm over pieces with tack.communication: collectives,
    exchanges, a ghost Plan that makes a diffusion over blocks give the
    whole line's bits, Routing points to the block that holds them, and
    sort_by_key. Its examples run in validate.py and, unchanged but for
    the world, under mpiexec. The Runtime API contract gains a
    Communication section: what is collective, what does not depend on how
    blocks are assigned, and the errors.
  • Two reference pages in the Visualization Guide, generated from the
    docstrings: Algorithms, every tack.dataset algorithm by what it does,
    each group with a table of what it gives, what it takes (pieces, AMR,
    octrees) and its VTK and Viskores counterparts; and Data Model, the
    classes, constructors and sources, and the VTK, MFEM and Warp interop.
    A test fails when an exported name has no entry. The site's API pages now
    see tack-vis (a Griffe extension, docs/_ext/tack_namespace.py): before,
    nothing under tack.dataset or tack.interop could be rendered.
  • A prototype NVIDIA Warp backend (tack.device("warp"), the warp
    extra): kernels lower to generated @wp.kernel source from Tack's typed
    IR and run on Warp's CPU or CUDA device, on its current stream, with
    Tack's integer and floating-point semantics, wrap_ptr, DLPack and debug
    bounds checks; workgroups on CUDA only, no tack.inspect
    (Backend.supports_inspect). Its arrays are wp.arrays. CI's warp
    job runs it and the interop on Warp's CPU device.
  • tack.interop.warp: from_warp/to_warp share arrays both ways (Warp's
    vector and matrix types as Tack's), and from_simdata/to_simdata
    exchange datasets with warp-simdata. Point clouds, curves, surface meshes,
    OpenFOAM meshes and patches, CGNS sections (one shape, MIXED, NGON_n,
    NFACE_n), EnSight parts, corner-point grids and FLASH axisymmetric data
    (an AMRDataSet of revolved boxes, coordinates implicit) come across
    without copies, CGNS's 1-based and MIXED connectivities read lazily;
    other models through warp-simdata's VTK conversion.
  • An unstructured topology of one shape whose offsets count in steps of the
    shape's points shares its connectivity as the shape's rows rather than
    gathering them, so an imported mesh stays one copy through the filters.
    Polyhedral, polygonal and polyline topologies keep implicit arrays as
    given.
  • Implicit booleans: td.Union(f, g, ...), td.Intersection(...) and
    td.Difference(...) combine implicit functions as VTK's
    vtkImplicitBoolean does (the smallest value, the largest, the first's or
    the rest's negated), on the device, wherever a function is taken (clip,
    slice, extract_geometry, extract_points, implicit_values); held to VTK in
    every implicit-function comparison.
  • td.surface_normals(data, consistent=True): a surface's polygons wound
    alike -- each connected region as its first cell, by union-find with
    parities over the edges on the device -- and its unit normals on cells and,
    summed and made unit, on points, as VTK's vtkPolyDataNormals without
    splitting (matched to it on spheres, scrambled windings, curved quads and
    polygons). A surface no winding makes consistent is refused.
  • td.extract_extent(data, extent, rate=1): a box of a structured grid's
    points (VTK's VOI, in the whole grid's indices), every rate-th, as a
    structured grid of the same kind -- an image stays uniform, a rectilinear
    grid's axes are cut, a curvilinear grid's points taken; a k-plane of a 3D
    grid is a 2D grid. Matched to vtkExtractVOI, vtkExtractRectilinearGrid
    and vtkExtractGrid point for point and cell for cell.
  • vtk_to_dataset reads vtkPolyData -- what VTK's surface readers (STL,
    PLY, OBJ, .vtp) and its contours give -- as the unstructured grid of its
    cells in VTK's order: triangles and quads an UnstructuredTopology, larger
    polygons a PolygonalTopology, lines a PolylineTopology, triangle strips
    as their triangles (wound alike, each taking its strip's cell data). It
    was refused, and needed vtkAppendFilter first.
  • contour(..., dimension=): 3, 2 or 1, the cells to contour -- by default
    the mesh's highest -- each dimension's alone, never mixed as VTK mixes
    them: the boundary triangles beside a mesh's tetrahedra give lines with
    dimension=2. Contours of lines and polylines (dimension=1, or a 1D
    mesh) are vertex cells at the crossings, a crossing at a point whose value
    is the level that point once; they were refused. Matched to VTK per
    dimension on Viskores' mixed-cell-shapes.vtk.
  • Voxelization: td.voxelize(data, grid, fields=None, samples=4), each voxel
    of an image the fraction of its volume the data covers and each field's
    mean over it, by sampling through probe, so for datasets, pieces, AMR
    hierarchies and octrees alike.
  • Cell quality: td.cell_quality(data, measures), the scaled Jacobian,
    Jacobian ratio, condition number, edge ratio and shortest and longest edge
    of every cell, each 1 on its shape's ideal cell. Verdict's definitions
    (matched to VTK's vtkMeshQuality where it defines them), extended to every
    shape: order-2 cells along their curved edges, wedges and pyramids, and
    polyhedra and polygons by their fan tetrahedra and triangles.
  • Feature edges: td.feature_edges(data, angle=30.0, boundary=True, feature=True, non_manifold=True, manifold=False), a surface's edges by the
    cells meeting there and their angle, as VTK's vtkFeatureEdges (matched
    edge for edge), as line cells on the same points with an EdgeKind cell
    field.
  • clip of polyhedral topologies: each cut cell becomes the polyhedron of its
    kept part, its faces clipped once (so the output is conforming) and its
    iso-polygons (López's, as contour traces them) as caps; volumes equal the
    shape-based clip's for planes.
  • Time on datasets, for filters that stream steps (in situ): DataSet(..., time=, time_unit=), kept by every filter as the origin is; pieces and AMR
    boxes at two times are refused; VTK's DATA_TIME_STEP both ways.
    tack.dataset.temporal holds what the temporal filters share: a step cache
    that copies what it keeps (copy=False keeps references) and the time
    between steps, converted between units.
  • Running statistics and time derivatives over streamed steps:
    td.RunningStatistics(fields) (per point or cell mean, variance, minimum and
    maximum over the steps so far, by Welford's method on the device) and
    td.TimeDerivative(fields) (the backward difference since the previous
    step, in the field's unit over the time unit), each fed update(data) a
    step at a time, piece by piece for a PartitionedDataSet; one mesh over the
    steps (a new topology object over the same mesh is the same mesh).
  • Advection through streamed steps: td.Advection(field, seeds) carries
    particles from step to step, the velocity linear in time between the kept
    previous step and the current one (each probed in its own mesh, so meshes
    may change; over ranks for pieces), with inject(points, time=) releasing
    particles at any time; td.extend_paths joins each step's particles into
    paths; td.Pathlines and td.Streaklines return the current lines every
    step.
  • Physical units, a first pass: Field(..., unit=) holds the values' unit, a
    string in unyt's canonical spelling (None is unknown), and the spatial
    unit is the geometry's (data.unit; uniform_grid, rectilinear_grid,
    point_cloud and DataSet take unit=). Filters that keep a quantity keep
    its unit; the rest give None. Two known spatial units do not meet (probe,
    pieces, AMR boxes). convert(data, unit, field=None) converts the geometry,
    the origin in double precision, or a field, offsets included. The VTK
    interop maps UNITS_LABEL. Tack knows the US survey foot (ftUS).
    tack-vis now depends on unyt (pure Python), imported on first use.
  • Derived units: gradients (per spatial unit), jumps, dot and cross (by SI
    name where one fits: N m is J), composite, cell and face measures, edge
    lengths, fluxes and their divergence, perot, statistics
    (Statistics.unit, weight_unit and unit_of(name)), deposits (counts,
    per_volume), SPH density and sums, halo masses and radii, outlier
    distances, normals, particle times. A difference of degrees Celsius is in
    kelvin; such values are not multiplied. log_values refuses a dimensioned
    field. interpolate(..., unit=) states a function's unit.
  • Quantities as parameters: lengths (voxel_downsample(cloud, "10 cm"),
    radii, periodic boxes, smoothing lengths, softening, slice_plane's
    origin), thresholds and isovalues (threshold(data, "T", "20 degC", ...))
    and step sizes take a string, a (value, unit) pair or a unyt or pint
    quantity, converted on the host; a plain number is as before. A mass,
    density or velocity given as one quantity brings its unit to the result.
  • Octrees, a first pass: td.OctreeDataSet, a grid of octrees (VTK's
    HyperTreeGrid with branching factor 2) stored as a linear octree for GPUs,
    each node its tree, level and Morton key, sorted in pre-order; leaves,
    centres and measures from the keys in kernels. Cell data on every node, as
    RAMSES and HyperTreeGrids store it; non-leaves REFINEDCELL, so statistics
    and entropy count each region once. OctreeDataSet.from_cells takes cells'
    levels and positions (RAMSES, yt); vtkHyperTreeGrid converts both ways
    through each tree's breadth-first descriptor.
  • An octree's leaves as cells and its probe: octree.leaves() is an
    unstructured dataset of voxels (pixels in 2D) with the fields as cell data,
    coinciding corners one point (sorted on the finest level's lattice), hanging
    ones kept; probe(octree, where) takes each point's leaf by one binary search
    of the keys, as VTK's vtkHyperTreeGridProbeFilter, and over MPI ranks sends
    each point to the rank whose trees hold it. The boolean octree.leaves() of
    the first pass is now octree.is_leaf.
  • Weighted statistics: statistics(data, name, weights="volume" or a field)
    and entropy(..., weights=), for datasets, pieces, AMR hierarchies and
    octrees: a volume-weighted mean over cells of different sizes, a mass from
    a density. Statistics.weight is the weights' sum.
  • Block-structured AMR, a first pass: td.AMRDataSet(levels, ratios), levels of
    structured boxes placed by their topologies' offset, the hierarchy in index
    space as AMReX's, so boxes may be images, rectilinear or curvilinear grids
    (VTK's AMR holds the first two). Made with its marks: REFINEDCELL on cells
    a finer box covers (as VTK's BlankCells, which marks Tack's export alike),
    REFINEDPOINT (a new bit, 8; VTK has none) on points a finer level also
    holds, and siblings' shared points DUPLICATEPOINT, so each region and
    location counts once in statistics and entropy. Filters that make cells
    leave hidden and refined cells out (Ghosts.MASKED_CELLS). Overlapping
    boxes, improper nesting and images off their level's lattice are refused.
    vtkOverlappingAMR both ways, amr_gaussian_pulse (VTK's source), and the
    boxes' metadata gathered over MPI ranks.
  • AMR ghost cells: ghost_cells(amr, layers) grows each box, a ghost from the
    sibling holding it, else from the coarser level (injection for cells, linear
    interpolation for points and for a grid's positions, so rectilinear and
    curvilinear boxes grow too), hidden where neither holds it;
    update_ghosts(amr, ...) refreshes them, and average_down(amr, name) puts
    fine data into the coarse cells and points it covers. Each is a kept
    tack.communication.Plan and a weighted sum, so they run over devices and
    MPI ranks.
  • probe(amr, where): each point in the finest level's cell holding it, over
    every rank's boxes; onto an image, the hierarchy resampled, equal to VTK's
    vtkAMRResampleFilter (which leaves points on the domain's upper faces
    unset).
  • A dataset's origin: DataSet.origin, three doubles on the host, where its
    geometry's coordinates start. Kernels see only the geometry's values
    (tack.f32, or tack.f64 where the device has it), so points far from
    zero, such as a LiDAR survey at UTM coordinates, keep millimetres in single
    precision. Filters keep it. Implicit functions, probe points, seeds,
    elevation points and functions of position take world coordinates,
    converted in double precision; transform moves the origin;
    world_positions() and rebased(origin); vtk_to_dataset(..., origin=),
    and dataset_to_vtk writes world positions. Pieces whose positions meet
    must share one origin.
  • td.PointLocator: points in uniform bins sorted by bin, for radius
    neighbours (CSR) and the k nearest, deterministic (no atomics), and a device
    class for one's own kernels. On it, voxel_downsample (centroid, first or
    nearest point per voxel), outliers (radius or statistical, marked; VTK's
    vtkRadiusOutlierRemoval and vtkStatisticalOutlierRemoval keep the same
    points) and point_normals (PCA over the k nearest or a radius, signed
    toward a viewpoint or up; within 0.1 degrees of vtkPCANormalEstimation).
  • On the point locator: connected_components(data, radius=, min_size=)
    (Euclidean clusters, friends-of-friends; VTK's
    vtkEuclideanClusterExtraction one to one), deposit (points onto an image
    by nearest grid point or cloud in cell: counts, sums, means, minima, maxima,
    per volume; LiDAR surface models), and sph_interpolate/sph_density
    (VTK's four SPH kernels in 1 to 3 dimensions, Shepard normalization; within
    float rounding of vtkSPHInterpolator). All deterministic.
  • Periodic boxes in the point locator (PointLocator(..., periodic=box)):
    bins tile the box and wrap, distances to the nearest image; the point
    filters take periodic=.
  • halo_catalog: cosmology halos from friends-of-friends groups, ported from
    Viskores' cosmotools and ParaView's LANL halo finder: count, mass (per
    particle), centre of mass, most bound particle and its potential (with
    softening), mean velocity and dispersion, radius, in periodic boxes. The most
    bound particle is exact, with Viskores' pruned search on every large halo,
    tightened to bound each particle against each far bin. Viskores' two test
    cases give the same halos and centres.
  • Point clouds in pieces. ghost_points(pds, radius= | k=) gives each piece
    copies of the other pieces' points within a radius of its box, or exactly
    those among its points' k nearest (images across a periodic box's faces),
    with fields, sets and global ids; update_ghosts refreshes them. Normals,
    the radius outlier test and the SPH filters then run per piece with the
    whole's answers. connected_components(pds, radius=, min_size=, periodic=), outliers, voxel_downsample, deposit and halo_catalog
    take a partitioned point cloud, in one process or across MPI ranks: pieces
    cut in index order give the whole's clusters, centroids and catalogues bit
    for bit. Regions over pieces are now joined from the regions that cross
    pieces alone and numbered by a sort over the pieces, so no rank holds every
    region; min_size works on meshes in pieces too, counting each point once.
  • tack.communication.Plan lets a block send to itself, as a piece in a
    periodic box is its own neighbour.
  • td.point_cloud(positions, fields): a dataset of one implicit vertex cell
    per point (stored nowhere), float64 positions on an origin of their own,
    integer fields (a LAS classification, intensities) kept in their type. The
    first step of point clouds and SPH (the vault's proposal).
  • merge_pieces(pds): the pieces of a partitioned dataset as one dataset,
    each owned cell and each point once, as vtkMergeBlocks after
    vtkRemoveGhosts. Points matched and numbered as generate_global_ids
    does; hidden cells and global ids kept. Structured pieces that tile a box
    come back as that box's grid, split's input exactly; others as an
    unstructured dataset. Across MPI ranks it is gathered on a root rank.
  • connected_components(data): cells connected through shared points, as
    VTK's vtkConnectivityFilter (all regions, numbered alike) and Viskores'
    CellSetConnectivity: region on the cells and point_region on the
    points, by union-find on the device. Of a PartitionedDataSet too: each
    piece labels its own, the copies of shared points join regions across
    pieces, and regions are numbered across every piece and MPI rank, as on
    the whole dataset where the pieces carry its global ids.
  • ghost_cells(pds, layers=n) makes any number of layers, each grown from
    the points of the layer before (cells of any piece, so pieces thinner than
    n cells are fine), equal to split's and vtkGhostCellsGenerator's for
    two and three layers. Structured pieces stay structured: pieces tiling a
    box of cells become their blocks grown by the layers, with their offsets,
    values from their owners and grown coordinates (uniform, rectilinear or
    explicit), split's ghost layers exactly; update_ghosts works on them.
  • probe of a PartitionedDataSet: each point goes to the lowest piece one
    of whose own cells holds it (holding ahead of the locator's tolerance),
    is evaluated there, and its values come back, equal to probing the whole
    dataset, with or without ghost cells, on any devices or MPI ranks.
    where may be a PartitionedDataSet, giving one back.
  • tack.communication.Routing: points to the block that holds them (by the
    blocks' boxes and the caller's ranked test), values sent with them
    (send) and answers returned in the points' order (reply). Particles
    across pieces now route through it.
  • Particles across pieces: advect and streamlines of a
    PartitionedDataSet route each particle to the piece owning the cell
    that holds it, advance it there, and hand it on when a whole step ends in
    a ghost cell, in rounds until none is in flight, on any devices or MPI
    ranks (each rank its own seeds). With a ghost layer wider than a step the
    paths are the whole mesh's up to rounding (checked: positions within
    1.2e-6, the same steps and status); without ghost cells particles cross
    by the short step to the boundary. streamlines gives each piece the
    stretches it traced, with seed and segment.
  • advect takes its own output as seeds and goes on from there: times,
    steps and status carried, stopped particles left as they are.
  • statistics and entropy of a PartitionedDataSet work across MPI
    ranks: each rank passes its pieces (PartitionedDataSet(..., world=)),
    and the pieces' moments and histograms are gathered and combined in the
    pieces' global order, so the result has the same bits however the pieces
    are shared out, a rank with none included, and as in one process.
  • Global ids: data.global_ids (tack.dataset.GlobalIds), an i64 per
    point and per cell, the same on every piece holding it. VTK's GlobalIds
    attributes come in as them (no longer as fields) and go back out;
    filters carry them where an output entity is an input entity.
    generate_global_ids(pds) makes them as vtkGenerateGlobalIds does
    (equal ids and duplicate marks on pieces without ghost cells; on pieces
    with ghost cells it keeps the owners the marks say, where VTK reassigns
    them), in one process or across MPI ranks, and ghost_cells carries them
    into the ghost layer, matching points by them.
  • ghost_cells(pds) gives each unstructured piece of a partitioned dataset
    a layer of its neighbours' cells, as VTK's vtkGhostCellsGenerator does:
    the same cells, points, marks (a shared point belongs to the lowest piece
    holding it) and values, checked against it and against split's ghost
    layers. Pieces may be on any devices, or on MPI ranks (each rank its own
    pieces; PartitionedDataSet(..., world=)), the same code through
    tack.communication: boundary points matched by a distributed sort of
    their positions, cells sent to neighbours by exchange.
    update_ghosts(pds, *fields) refreshes ghost values from their owners
    through the two Plans ghost_cells keeps, so a second filter that needs
    neighbours (point gradients of projected values) is right on every owned
    point.
  • tack.communication.sort_by_key(blocks, keys, values): every block's keys
    sorted together, the global order cut into one range per block, every
    copy of a key on one block (a sample sort, as DIY's), in one process and
    across MPI ranks.
  • tack.communication: communication between blocks (units of work with
    global ids, each on a device, such as a partitioned dataset's pieces), the
    patterns that algorithms over pieces are written on. Blocks.exchange
    sends messages, each a dict of named Tack arrays, from any block to any
    block, arriving on the destination block's device; all_reduce,
    all_gather and exclusive_scan combine small host values over every
    block in global-id order, so a sum has the same bits however blocks are
    assigned; link_by_bounds finds neighbours by overlapping boxes; a Plan
    keeps an exchange (elements gathered on one block, scattered on another)
    for reuse, as ghost values are refreshed from their owners. DIY's model,
    with Tack arrays as the messages. World() is one process, blocks on any
    of its devices; World(comm) spans an mpi4py communicator's ranks (the
    new mpi extra), staging arrays through the host or, with
    gpu_aware=True, handing them to MPI through DLPack. The same cases run
    on both, and a CI job runs them under mpiexec.
  • contour takes a sequence of values, and slice and slice_plane take
    one as value (parallel planes for slice_plane): every value's contour
    in one dataset, made in one pass over the cells, each output cell's value
    in a cell field contour_value. On every path: shapes, polyhedra, 2D
    lines and per-cell (L2) lines. 10 values on 1M hexahedra: 38 ms against
    80 for ten contours (CPU). As warp-simdata's compute_many, but one
    dataset, as VTK's and Viskores' contour filters give.
  • topology.validate() checks the arrays a topology is built from, on their
    device, reading back a few integers: offsets that start at 0, never run
    backwards and end at their members' length, ids within range (points,
    faces, sides), and an unstructured topology's types and point counts. It
    raises ValueError saying what is wrong and how often. Debug mode
    (tack.set_debug, TACK_DEBUG) checks every topology's ids when it is
    made, before anything is derived from them: a bad id from a reader or a
    caller was read or written out of bounds, silently. As warp-simdata's
    validate_datasets.
  • for_each(kernel, data, space, ...) runs a kernel over a space's nodes, each
    once (DataSet.node_views): views over points, edges, quad faces or cells
    whose entities have node_count(i) nodes with node_dof(i, j) and
    node_position(i, j). interpolate is now one kernel over them, with no
    table of writers and no sort, and gives the same values; an L2 geometry,
    which gives a shared node no position of its own, keeps the first cell's.
  • A @tack.device_class object, or one it holds, can be passed to a
    @tack.func or to another template's method -- helper(obj, i),
    b.combine(a, i), helper(self), helper(self.base) -- not only to a
    kernel: the callee is specialized for it, as a method is for self.
  • A device function may return f(x) where f returns several values,
    vectors among them, and a name may hold such a result until it is
    unpacked (hit = closest_hit(o, d), then depth, normal, color = hit).
  • Partitioned datasets, the first phase of the partitioned-data proposal.
    tack.dataset.PartitionedDataSet(partitions) is a dataset in pieces, an
    ordered list of DataSets as VTK's vtkPartitionedDataSet, each piece
    with its own topology, fields, sets and device. Work on one piece takes
    the piece: the caller iterates. statistics and entropy also take a
    partitioned dataset and a field's name, and combine the pieces (moments
    pairwise; histograms over the whole's range), each piece reduced on its
    own device. split(data, pieces) cuts a structured grid by index ranges
    into pieces that keep the geometry's kind and carry their block of the
    point, cell and L2 fields and the sets. vtk_to_dataset and
    dataset_to_vtk convert vtkPartitionedDataSet and
    vtkPartitionedDataSetCollection (a dict of names to partitioned
    datasets).
  • Filters carry ghost marks, the second phase of the partitioned-data
    proposal. They process a piece's ghost cells, as VTK's do, and what they
    make takes the owner of what it came from: an output cell its source
    cell's, a point on an input edge the edge's, a point inside a cell the
    cell's, a surface face its cell's, and a face two cells share or an edge
    the owner of its smallest point, by position. Over pieces with a ghost
    layer, the owned part of every filter's output is the whole's output, and
    statistics and entropy of the outputs are the whole's (checked for
    every filter that makes cells or points, chained, on polyhedra, in 2D,
    over split's pieces and VTK's). split(data, pieces, ghost_layers=n)
    adds ghost layers; remove_ghosts(data) takes the duplicate and hidden
    cells out, keeping the points a piece owns, before rendering or writing
    one file. Filters never process hidden cells. Outputs of pieces without
    ghost cells, which cannot know what their neighbours make on the faces
    and edges they share, record that they lost the marks, so reductions
    refuse them, saying why.
  • Ghost marks: every cell and point that several pieces hold has one owner,
    and the other copies are ghosts, marked in data.ghosts (a
    tack.dataset.Ghosts of u8 marks per cell and per point, in
    VTK's vtkGhostType bits). vtk_to_dataset takes a vtkGhostType array
    as the marks rather than as a field, and dataset_to_vtk writes it back;
    split marks the points its pieces share as duplicates in all but one;
    statistics and entropy leave out duplicate and hidden cells and
    points, so over pieces with one owner per point they give the whole's
    answer. Filters that keep the cells and points keep the marks; the others
    do not carry them yet, and their outputs record that they lost them, so
    statistics and entropy refuse those outputs rather than count shared
    cells and points once per piece.
  • Examples: 45_flow.py (gradients, flow_quantities, advect,
    streamlines and perot on a swirl whose answers are known exactly) and
    46_cut_and_sample.py (clip, extract_geometry, extract_points,
    slice and probe with implicit functions and a reused CellLocator),
    each checking its results. The dataset tour now runs and checks the
    algorithms no example showed: implicit_values, point_elevation,
    log_values, point_ids, cylindrical and spherical (and back),
    mask_points, extract_edges, tetrahedralize, as_polygons, perot
    and shrink.

Fixed

  • HIP arrays take hipMalloc memory through a cache of dropped arrays'
    buffers, reused by size class in stream order, instead of the device's
    stream-ordered pool. On ROCm 7.1.1 the pool slowed with every allocation
    it had made (63 to 600 µs a hipMallocAsync over the 512 eta pieces'
    global ids, 2.7 ms when trimmed), and that port stalled for two hours;
    hipMalloc stays at 17 µs, and the cache spares the hipFree that waits
    for the device. The cache holds at most a quarter of the device, never
    memory handed outside Tack nor MPI arrays, and an allocation that runs out
    frees what it holds first (Backend.release_memory, which Metal's pool
    answers too). TACK_HIP_ALLOCATOR=pool still selects the pool.
  • A dropped dataset frees its device memory at once. A topology interned
    its spaces in a dict and each space holds its topology, so every dataset
    with a field was a reference cycle, freed only when Python's cycle
    collector ran, which counts objects, not device memory: on a 16 GB GPU
    the 20M-cell checks ran out of memory with dropped datasets still on the
    card. The topology holds its spaces weakly. A topology's cached cell and
    point locators still hold a dataset of it, a cycle left for now.
  • An allocation that runs out of device memory collects Python's garbage
    and tries once more. A dropped array held in a reference cycle keeps its
    memory until the cycle collector runs, which counts objects, not device
    memory: a topology's cached locator holds a dataset of that topology, so
    a mesh dropped after a probe or streamlines stayed on the GPU, and a
    16 GB card ran out with dropped meshes on it. Every backend raises
    DeviceMemoryError (a MemoryError) when an allocation runs out (CUDA,
    HIP, Level Zero, Metal; NumPy's MemoryError on the CPU), and
    tack.empty collects, waits for the device and retries once.
  • The CPU fans out fewer elements than it has threads when each is dear
    enough to repay it. The threshold was floored at the thread count, so on
    jura's 28 threads the ONERA M6's 17 streamlines ran serially on every
    call (30 ms, against 7-12 on 8 threads), and a short range of dear
    elements never fanned out from 10 threads up. The floor is two elements,
    the fewest a fan-out can split; above it the costs decide.
  • Advection on a GPU runs in launches of a bounded number of steps, each
    going on where the last paused, sized to about 0.2 s at the last launch's
    pace (the first takes 16 steps). One launch carried every particle's whole
    path, and a GPU driver resets a device whose launch runs past its limit:
    the ONERA M6's streamlines lost headroom's Level Zero device every time
    (7.5 s there; 2 s by default on Windows), and a lost device takes every
    array of the process with it. The office port's one launch took 4.9 s.
    A pause keeps each particle's position, time, steps and status, whether it
    stepped in this call (a hand-off between pieces needs it) and the cell it
    was last found in, so the paths are one launch's, bit for bit, on the CPU
    and on Metal. The CPU still runs one launch.
  • A particle pushed across a mesh boundary that landed inside and went on
    took its next whole steps from a stale state: its RK4 stages placed at the
    bisection's last trial size rather than the step's, and, when a trial's
    stage had left the mesh, starting mid-step on the trial's stage
    velocities. Some such particles then stopped as if seeded outside
    (SUCCESS cleared). Particles that run along a curved wall, an
    aneurysm's, changed path: 82 of 20,000 seeds there.
  • carry, visible, owned_values and orient run on their dataset's
    device. Made public for filters of one's own (#163), the first three had
    only been called from inside algorithms that had already made that device
    the default; called directly they allocated on the default device, and
    failed with "arrays on two devices" (six of test_public_filters'
    rewrites in a CUDA run, after an earlier test left CUDA the default).
    orient failed the same way on a topology on another device, and
    without_hidden and refuse_hidden, which the built-ins call, allocate
    on their dataset's device too. The "Write
    a Filter" chapter's triangulate makes its ConstantArray on the data's
    device.
  • A compiled CPU kernel bound to another device's arrays raises a
    TypeError naming the kernel and the argument, where it wrote device
    pointers from the host and segfaulted (CUDA arrays, met through the Warp
    backend's run of test_parallel_dims.py). Arrays bind through
    DeviceBuffer.host_address, which only host memory has: the CPU's, and
    Warp's on its CPU device. A kernel call never got that far, since
    dispatch picks a device that can address every array.
  • A repeated extract_surface of a mesh whose faces were never derived
    costs only the carry again. The fresh boundary (#157), which finds the
    surface without deriving every face, kept nothing, so every call paid
    for it: on jura, 20M hexahedra's warm surface went from 5.7 ms to 447 on
    the CPU (1.5 to 28 on CUDA). The topology now keeps the surface it found;
    5M hexahedra's repeated call on an M1 Max takes 0.3 ms, against 55-150.
  • np.from_dlpack of a GPU array (CUDA, HIP, Level Zero) raised
    SystemError in place of NumPy's own error: the capsule's destructor, a
    ctypes callback, met and cleared the pending exception. C now gets native
    destructors.
  • point_normals gave a line of points (no plane) an arbitrary normal, and
    on CUDA [0, 0, 1]: the least eigenvalue counts as repeated within the
    closed form's rounding, not only when exactly equal.
  • A mesh with cells of more than one dimension (a surface's triangles among
    tetrahedra, as Gmsh and Exodus meshes carry): cell_geometry measures each
    cell in its own dimension, and net_flux, perot and divergence give its
    lower-dimensional cells zero; all three failed inside kernel resolution
    ("no @tack.func method 'side_position'"), and perot left those cells
    unwritten. Found with Viskores' test datasets, now in the tests
    (_viskores_datasets.py, measures held to Viskores' expected values).
  • as_polyhedra refuses an inside-out cell beside an upright one, which no
    polyhedral face can record, saying so; contour and clip of the result
    failed later, with faces that "do not close" and a pointer to
    check_winding, which passes such cells.
  • tetrahedralize and triangulate of a collapsed cell (repeated points, a
    pyramid written as a hexahedron) leave out its pieces with a repeated
    point, which measure nothing; VTK keeps them, but their repeated faces made
    every filter that derives faces refuse the output.
  • vtk_to_dataset of a grid with a point array and a cell array of one name
    (VTK keeps them apart) kept only the cell array, so a contour of the name
    found nothing: the cell array is now the field "<name> (cells)", with a
    warning. Viskores' mixed-cell-shapes.vtk has such a pair.
  • contour and clip of polyhedra with a NaN in the field: the tracing took
    a NaN as neither above nor below the level, and its faces did not close
    ("check_winding"); a NaN is now below every level, as the shape path's
    cases have it, and an inside clip keeps it, so the clipped volumes are the
    shape path's on both sides.
  • contour of a 2D mesh with a NaN in the field: the first crossing was
    found with < iso but crossings paired with >= iso, so fewer lines were
    written than counted and the rest held whatever empty gave them -- read
    as indices, a bus error now and then. A NaN is below the level there too.
  • Point filters on GPUs (the validation round on mustafar-linux, headroom
    and illyad, 2026-10-25):
    • PointLocator.neighbors and nearest ran on the default device when no
      Tack array was among their arguments (the locator's own points, or host
      query points), and failed with arrays on two devices. The locator now
      has its dataset's device, as CellLocator does.
    • voxel_downsample(keep="nearest") broke a tie the other way on CUDA, HIP
      and Level Zero: a voxel of two points has them equidistant, and a GPU
      fused the squared distance into multiply-adds. Distances that decide a
      neighbour, the k nearest, a friends-of-friends link or a voxel's point
      are now named products (squared_length), rounded alike on every
      backend, as implicit functions' values already were.
    • Level Zero compiles with -cl-fp32-correctly-rounded-divide-sqrt: its
      f32 division (2.5 ulp) and square root (3 ulp) put centroids and means a
      bit off CUDA's, HIP's and the CPU's.
    • The docs said the point filters' sums had the same bits on every backend.
      They have them from run to run; counts, labels, neighbours and kept
      points are the same on every backend, and sums agree to rounding.
  • GPU-aware MPI with HIP arrays: Tack's HIP arrays come from the
    stream-ordered pool (hipMallocAsync), which MVAPICH-Plus 4.1's UCX took
    for host memory (an assertion, or a segfault, at the first send; illyad).
    World(comm, gpu_aware=True) now hands such an MPI device copies in
    hipMalloc memory and receives into it (Backend.mpi_direct,
    Backend.allocate_mpi_array); the exchange runs at 36.8 GB/s a rank on
    the MI210s, against 44.6 for hipMalloc arrays sent as they are.
  • GPU-aware MPI with CUDA arrays, the same way: Cray MPICH 9.1 could not
    export Tack's CUDA pool memory (cuMemAllocAsync) to another rank on the
    node ("Invalid count") and libfabric refused it across nodes (Perlmutter's
    A100s). Sends go as copies in cuMemAlloc memory and receives land in it:
    22.2 GB/s a rank over CUDA IPC on one node, 5.9 across two. World's
    docstring says what such an MPI needs of the launch: each rank's device
    made before MPI starts, and every GPU of a node visible to its ranks.
  • slice and slice_plane of a curved dataset (an order-2 geometry) raised
    NotImplementedError since #93 made contour refuse order-2 fields: the
    implicit function's values are in the geometry's space. They are now
    projected into order-1 H1 first, so the cells are cut linearized at their
    corners, as before #93. Example 43, which contoured an order-2 field, now
    contours its order-1 interpolant. Found on mustafar's CUDA round.
  • Face derivation on AMD GPUs: _sort_buckets sorts a bucket of up to 16
    entries in a local array and writes it back once, instead of by insertion
    in global memory, where each shift is a store the next comparison reads.
    1M hexahedra on an MI210: the sort 2.48 to 0.76 ms, faces() 9.2 to 7.5,
    a fresh extract_surface 11.6 to 9.8; unchanged on an H100 and the CPU.
    Found on illyad's round.
  • tetrahedralize over the pieces of a partitioned dataset splits each face
    pieces share along the same diagonal on both sides. A structured piece
    alternated its hexahedra by the parity of its own (i, j, k), so a piece
    starting at an odd index split the shared quads the other way; it now adds
    the topology's offset, the grid index of its first point, which split
    sets and vtk_to_dataset takes from an extent (StructuredTopology(..., offset=)). Other pieces split from each cell's corner of smallest local
    id, which differs between pieces; they now rank corners by global id where
    the dataset has global ids. A single dataset's tetrahedra are unchanged.
  • The CPU threading policy kept fanning out kernels whose threads contend
    (many atomics on a few counters: a cell-type histogram over one type, a
    hot bucket), at 8-9x their serial time on 28 cores. A fan-out measured
    slower per element than serial was folded into the serial estimate, and
    the threshold still credited threads with a 10% gain. Now, in a kernel
    with atomics, such a fan-out is settled by one whole-range serial run: if serial wins, the kernel is
    contended and runs serially, with the back-off rechecks still trying
    threads now and then. jura: histogram 41.2 to 4.8 ms (one thread 4.6);
    yavin, one atomic counter over 1M elements: 39.2 to 2.3 ms. Found and
    fixed on jura.
  • Implicit functions (Plane, Sphere, Cylinder, Box, Planes) give
    the same bits on every backend: each product in value is named, so the
    float contract rounds it on its own and no backend fuses it into a sum.
    A slice through grid points made 1015 triangles on Metal and 883 on the
    CPU (Metal's fused multiply-adds left noise about zero where the CPU had
    exact zeros, which contour merges); both now make 883.
  • as_polyhedra and extract_surface scanned their faces' point counts (4
    a face) into the mesh's own id type, which a mesh is given for 12 per
    cell: a structured grid of 170M cells, or 120M hexahedra that share no
    points, still on 32-bit ids, wrapped the offsets. They now widen to 64 bits
    when the faces need it. Found by an audit of every scan in Tack; the other
    46 were covered by their bounds.
  • A point just outside a pyramid, beside its apex, was located in no cell
    though within the locator's tolerance: Newton runs off there. Within 0.2%
    of the pyramid's size of the apex, it is now placed just past the apex,
    outside the hold margin, so it is near the pyramid (a cell holding it
    still ranks first) and takes the apex's value.
  • A point at a pyramid's apex was located in no cell: every (r, s) at
    t = 1 maps to the apex, so Newton's Jacobian is singular there and it
    did not converge. Pyramid.world_to_parametric now takes a point within
    shapes.APEX_SNAP (1e-6 of the pyramid's size) of the apex to be at
    (0.5, 0.5, 1). In vtkCellTypeSource's pyramid grid, all 18 apexes were
    found by no cell; both searches find them now. A point just outside a
    pyramid beside its apex, on the mesh's boundary, is still not found.
  • CellLocator.find, probe and advection: which cell held a point on a
    face, edge or vertex that cells share was the sign of rounding noise, so
    the searches from the points and from the cells could put a mesh point in
    different cells, on the CPU (hexahedra, tetrahedra, wedges, pyramids) as on
    CUDA. A cell now holds a point within 1e-5 of its parametric domain
    (shapes.HOLD_TOLERANCE), so every cell that has a shared point holds it
    and the smallest id wins, in both searches and on every backend.
  • CUDA: a product bound to a name, or returned from a device function, was
    fused into a later statement's sum (NVRTC's --fmad=true fuses across
    statements: all of 100k values on an RTX 4060 Ti), against the float
    contract's one rounding per named value. The CUDA generator now emits
    such products as __fmul_rn/__dmul_rn; a product inside a sum may
    still fuse, as the contract permits.
  • DLPack: Tack set argument types on ctypes.pythonapi's capsule
    functions, which are one object for the whole process, so its types and
    another library's replaced each other. Imported after Warp, Tack made
    Warp's capsule destructor raise ArgumentError at exit; imported before
    it, Warp's made Tack's destructor miss its capsule, and every array
    exported through DLPack and never consumed stayed pinned, its device
    memory never freed. Tack now binds the functions with prototypes of its
    own and touches nothing shared.
  • contour read only an order-2 H1 field's corner values, so a quadratic
    that rises inside a cell from zero at its corners gave no surface,
    silently. It now refuses an order-2 field (NotImplementedError) on
    every path, until cells are subdivided for contouring.
  • CUDA: Tack works in the device's primary context (retained, and released
    when its last backend goes) instead of creating its own. PyTorch, CuPy and
    Warp use the primary context, so a launch of Tack's after one of them made
    it current no longer fails with CUDA_ERROR_INVALID_HANDLE. A context an
    embedding application made current is still adopted as before.
  • CellLocator.find, and so probe, located a point in the lowest-numbered
    cell within the tolerance (0.1% of a cell's box diagonal), whether or not
    it held the point, and extrapolated there. On a Delaunay mesh of 16k
    tetrahedra, 2006 of 75k probes were wrong, by up to 1.7. Now a cell that
    holds the point ranks ahead of one it only comes near, and advection ranks
    its cells the same way. On GPUs without 64-bit atomics (Metal, Level Zero)
    the locator's limit halves, to 2^30 − 1 cells.
  • probe grouped its points with two radix sorts whenever some lay outside
    the mesh (they share one bucket): 57 launches. buckets.bucket_groups
    counts, scans and scatters instead, leaving the unread bucket unfilled:
    19 launches, and on an RTX 4060 Ti 10.5 → 4.0 ms for 100k points.

Changed

  • Breaking: the VTK interop needs VTK 9.7.1 or later (the interop
    extra requires vtk>=9.7.1), and uses VTK's pythonic API throughout:
    keyword constructors, properties, dict-like point and cell data, implicit
    arrays. A one-component vtkConstantArray or vtkAffineArray in point or
    cell data becomes a ConstantArray or CountingArray field, and back.

  • key_order, which merges contour and slice crossings, keeps the bucket
    sort on CUDA, HIP and Level Zero until buckets outnumber keys 64 to 1
    (8 to 1 on the CPU and Metal, as before). On CUDA the buckets were
    1.3-1.7x faster for a contour or slice of hexahedra at 13-20 buckets a
    key, where the radix sort had made them slower since it was introduced
    for memory (5M hexahedra's contour 5.4 to 7.8 ms); on the CPU radix is
    faster there (0.7-0.8x), and Metal goes either way.

  • The Visualization Guide and advect's docstring say when to advect on
    the CPU: each particle is one thread, so below a few thousand seeds a CPU
    copy of the mesh is usually faster than a GPU (the aneurysm's 25 seeds
    8.4 ms on 28 cores against 32 on an RTX 6000 Ada; 20,000 seeds 871
    against 331).

  • CellLocator takes its cells' bounds on the device, a GPU's threads
    each over every k-th box and the CPU's over a block, and reads back only
    the partial results. It read every box back and reduced them with NumPy:
    on the ONERA M6 (46.9M cells) that was 1.0 s of the locator's 1.4 s
    build on an M1 Max's GPU, and over PCIe on a discrete GPU (jura's CUDA
    build took 4.9 s, against Viskores' 3.0). The M6's build is now 0.45 s on
    Metal and 1.7 s on the CPU (2.7 s); the bounds, so the grid and every
    result, are the same bit for bit.

  • The cell locator bins cells on two levels, as Viskores'
    CellLocatorTwoLevel does: a coarse uniform grid of about 32 cells a bin,
    each bin a uniform grid of leaves sized by its count, about 8 cells a leaf.
    One level of about a cell a bin held up to 2.1M cells in a bin near the
    ONERA M6 wing (46.9M cells graded toward the wing in a far field a hundred
    chords wide), so a location there scanned 164k candidates on average, and
    a GPU's threads waited for their group's worst: Tack's streamlines on the
    M6 took 27.9 s for one seed on CUDA against Viskores' 3.5 s, and 230 s for
    17 on an M1 Max's GPU. Now the largest leaf holds 13,918 cells, and the 17
    streamlines take 0.36 s on Metal and about 4 ms warm on the CPU (1.5-2.3 s
    before). The build is faster (M6 5.6 s to 2.8 s on the CPU) with fewer
    entries (8.8 to 5.7 a cell); the aneurysm's 8.7M tetrahedra build in 0.68
    s (0.83), 12.9 entries a cell (7.8), and probe and streamlines there are
    as fast as before. Traces and probes are the same, bit for bit: the leaves
    only propose cells, and the search ranks them as before. Viskores sizes
    leaves at 2 cells; a bin's count is of the boxes overlapping it, a
    tetrahedron's box about six times its volume, and at 2 the aneurysm's
    tetrahedra were each entered 34 times (296M entries). A bin's leaves a
    side are rounded, where Viskores truncates: 32 cells a bin over 8 a leaf is
    1.59 a side, which truncated left every bin of a small, even mesh one leaf.
    tack-examples' fire port (10,462 cells, about 41 a bin) scanned 43
    candidates a location, against one level's 6.6, and its streamlines took
    twice as long on CUDA, HIP and Level Zero (jura's CUDA: 55 ms to 105).
    Rounded, 12 candidates (jura measured 83 ms with leaves about that size); the
    aneurysm 99 (110) at 15.6 entries a cell, the M6 unchanged, and the
    streamlines on the CPU as fast. CellLocator(data, levels=1) (Added) is
    the one level again, for meshes like the fire's.

  • The CPU threading policy fans out a short range of costly elements on its
    first dispatch, and balances it. A range too short for a timing slice ran
    whole on one thread, and the first recheck ran it whole again; fanned out,
    17 elements went to 8 threads in chunks of 3. Now one element is timed
    first: a cheap one changes nothing, a costly one (100 µs or more) starts
    slices until one lasts 20 ms, and the rest fans out; a recheck believes a
    sample that long; and a fan-out's chunks hold at most about 20 µs of
    estimated work, so costly elements go one a claim. 17 streamlines on the
    ONERA M6 wing: the first two calls 8.8 and 8.5 s to 2.4 and 2.1, steady
    2.7 to 1.6 (M1 Max, 8 threads); an aneurysm's 25 steady 1.5 to 1.1-1.4 ms.

  • Streamlines and advect try the cell a particle was last found in before
    searching the locator's bins. The cell is taken only where it holds the
    point well inside (by 1e-3 in its parametric coordinates, a hundred times
    the tolerance for holding a point), where no other cell of a mesh whose
    cells do not overlap can hold it, so the traces are the search's bit for
    bit; near a face, and in pyramids, surfaces and polyhedra, the bins are
    searched as before. RK4 streamlines on an aneurysm's 8.7M tetrahedra went
    from 7.4 ms to 1.7 (VTK on 8 threads: 3.6), and on the ONERA M6 wing's
    46.9M cells, with its boundary layer of wedges, from 4.9 s to 1.0 (VTK:
    3.9).

  • Matching points across many pieces in one process no longer sends a
    message from every piece to every other. tack.communication.sort_by_key
    joins the blocks' sorted keys and cuts them where the splitters do when
    every block is on this rank and one device, and the matching behind
    generate_global_ids, ghost_cells and merge_pieces hands each piece
    its copies on the host there. The copies of a value are grouped from the
    sorted keys, sorting by value only where two values hash alike, instead
    of np.unique over rows. On a simulation's 512 pieces (5.5M voxels),
    global ids went from 83 s to 9.0 (VTK on 8 threads: 10.4) and a layer of
    ghost cells from 184 s to 28.7 (VTK: 27.3). Across MPI ranks the messages
    are as before.

  • merge_pieces of unstructured pieces gathers cells' point ids in one go
    instead of slicing them a cell at a time in Python: 8.0 s to 0.9 s to merge
    an aneurysm's ten pieces (8.7M tetrahedra).

  • A few crowded points no longer send a mesh's face derivation to the slow
    sorted path. Faces go into buckets by lowest point, and one thread sorts
    each bucket. Any bucket over 256 faces used to fall back to the sort, and
    an aneurysm's 8.7M real tetrahedra had one point of 266 among 1.5M. The
    test is now the sorting work: the buckets' squared sizes at most 512 per
    face, and no bucket over 8192. A fresh mesh's extract_surface there went
    from 930 ms to 350 (VTK on 8 threads: 532).

  • Cold compiles cost less. Walking the IR uses an explicit stack, not
    nested generators, which handed each node up once per level of depth.
    Lowering copies syntax trees directly rather than through copy.deepcopy.
    Weak literal locals are found in one rewrite of the body, and copy
    propagation leaves alone the statements it has nothing to replace in.
    A cold probe of a mesh of five shapes went from 2.5 s to 2.1, with the
    generated code identical (LLVM IR hashed for 470 variants). Point links
    sort each bucket by entry alone, without reading keys every entry of a
    bucket shares.

  • Filters hold less memory at scale, measured beside VTK's by the new
    benchmarks/vtk_compare/memory.py, which samples the allocator's bytes in
    use through one call in a process of its own. At 20M hexahedra on the CPU:

    • contour's peak is 183 MB, down from 671 (VTK 74); slice_plane's is
      294, down from 725 (VTK 319). Merging a few million crossings no longer
      allocates bucket counts and offsets over every point: key_order radix
      sorts keys that are far fewer than their buckets. The per-cell counts are
      scanned in place.
    • extract_surface of a fresh mesh peaks at 2.1 GB, down from 4.0 (VTK
      1.5); faces() holds less while it works too. Past 20M hexahedra the
      fresh boundary takes the points in ranges, holding at most 2 GiB of face
      records at once, for the same surface. At 100M hexahedra it takes 1.46 s
      and peaks at 4.7 GB, against VTK's 1.47 s and 7.6 GB; in one pass it
      took 3.9 s. A face record no longer
      stores its lowest point, which its bucket already gives. Each bucket
      counts its faces and numbers them in place of per-record flags, run ids
      and slots, and the surface's rows come from each boundary face's cell
      rather than a slot for every (cell, face).
    • Cell to point of a cell field on cells of one shape reads the cells'
      values through the point links, rather than a contribution for every
      corner (640 MB at 20M). At 20M hexahedra its peak went from 1.7 GB to
      1.07 (VTK 0.99), and its warm time from 25 ms to 12 (VTK 96); at 100M
      tetrahedra it peaks at 1.9 GB against VTK's 3.4.

    Every filter runs at 50M and 100M hexahedra and tetrahedra.

  • A fresh mesh's first filters cost less at scale. Faces are derived by a
    counting sort of the (cell, face) records into buckets of their lowest
    point, each record placed once and each bucket sorted in place, with
    rows and sides from the cells in their order (the same faces, id for id);
    extract_surface of a mesh whose faces are not derived keeps the faces
    of one cell from those records without deriving the whole incidence; and
    point links of a one-shape mesh bucket its connectivity as it stands. At
    20M hexahedra on 8 threads, a fresh extract_surface went from about
    430 ms to 193 (VTK 249), at 20M tetrahedra from about 300 to 166 (VTK
    216).

  • Metal reuses the buffers of arrays that have died (1 MiB or more, by
    capacity): a new buffer's pages are mapped and cleared on first touch at
    about 13 GB/s, so a filter's large temporaries cost more than its kernels
    (640 MB took 50 ms to first fill). At 20M hexahedra on an M1 Max,
    threshold went from 67.7 to 25.4 ms, contour 41.6 to 28.7, slice 50.1 to
    32.3, cell centers 14.7 to 6.8 (VTK on 8 threads: 152, 73, 59, 301). A
    reused buffer is not cleared, so tack.zeros fills it; a new array's
    contents were always unspecified.

  • Small meshes pay less around their kernels: filters keep their constant
    tables (clip's cases, corner orders) on each device instead of uploading
    them per call (_common._table); exclusive_scan(..., close=True) writes
    the total at output[n] in the scan's own launch, so CSR offsets need no
    launch of their own; and clip scans its five kinds of counts in one
    launch below 2048 cells. A clip of 360 triangles went from 1.0 ms to 0.49
    (VTK's vtkClipPolyData 0.14), from 28 launches to 17.

  • extract_surface of a 2D mesh, by default, is the mesh itself -- its cells
    passed through, hidden ones left out -- as VTK's geometry filter passes
    polygons; it was the boundary edges, which extract_surface(data, boundary_faces(data)) still gives.

  • gradients of 2D cells -- triangles, quads, pixels and polygons, in a 2D
    mesh or among 3D cells -- lie in the cell's plane, as VTK's and Viskores'
    do (matched to vtkGradientFilter and vtkCellDerivatives on a curved
    surface, and to Viskores' UnitTestGradientExplicit); they were zero. A
    polygon's is Green-Gauss over its edges, exact for a linear field. Lines
    and vertices still give zero.

  • cell_geometry and cell_quality of shape cells no longer derive the
    mesh's faces: a shape's faces walk from its tables (_FaceWalk is in every
    shape cell's domain view), so on a fresh mesh of 1M tetrahedra volumes take
    6.5 ms instead of 24 (VTK's vtkCellSizeFilter 39 ms); polyhedra and
    polygons still derive theirs. Deriving faces computes each side's
    orientation in the pass that matches the sides (1.5 ms less at 1M
    tetrahedra).

  • uniform_grid takes an origin and spacing of one value per size (a 2D
    image no longer needs a third spacing) and a 2 x 2 direction for a 2D
    image. DataSet.rebased keeps an image an image, its corner moved in double
    precision, and a rectilinear grid its axes: a raster at survey coordinates
    rebased near them holds its points exactly, and stays one to deposit onto.

  • The documentation's datasets and visualization chapters are now the
    Visualization Guide, a tab of its own (docs/vis-guide/) with
    tack.dataset's reference and design records; the User's Guide is about
    kernels. Pages moved without redirects. New chapters: Point Clouds and
    Particles (LiDAR cleaning and gridding, SPH resampling, halos in a periodic
    box, each a checked example) and Algorithms Across Pieces (split from Work
    with Data in Pieces, with running over MPI ranks and point clouds in
    pieces).

  • Breaking: Planes(origins, normals) follows the data's precision, as
    the other implicit functions do, and takes no dtype: filters evaluate a
    copy of its planes in the geometry's floating type on its device, made
    once. A plane at x = 0.1 was off by about 1e-9 on f64 data unless
    dtype=tack.f64 was asked for.

  • tetrahedralize makes tetrahedra that meet face to face, as VTK's
    vtkDataSetTriangleFilter does. Viskores' fixed tables split each
    hexahedron the same way, so neighbours cut their shared face along
    different diagonals: on a 2 x 2 x 2 cube of hexahedra, extract_surface
    of the tetrahedra returned every inner face too (96 triangles for 48).
    Now a structured grid's hexahedra take 5 tetrahedra, the two mirror splits
    alternating with the parity of (i, j, k), and any other cell is the cone
    from its corner of smallest point id over the faces without it, each
    quadrilateral face split from its own smallest corner, whatever order the
    cells and points come in: hexahedra and voxels give 6 tetrahedra (were 5),
    wedges 3, pyramids 2, as VTK's counts.

  • probe, advect and streamlines given no locator reuse the one the
    mesh's topology keeps (cached_locator), instead of building one per
    call: probing 100k points in 1M hexahedra again takes 13 ms instead of
    48 on an M1 Max's CPU. A locator records its geometry's state, and is
    rebuilt (the kept one) or refused (one passed in) once Tack has written
    the positions, through a write count on every array's storage
    (DeviceBuffer.version, which kernel launches and host writes keep, at
    no measurable cost per launch). A locator passed in now serves any
    dataset of the same topology and geometry, not only the dataset it was
    made from. Writes through memory shared with another library are not
    seen.

  • contour and slice where the isovalue equals a point's value: the
    crossings of every edge at that point are now that point, once, and a
    triangle, polygon or line left with two corners at one point is dropped,
    as warp-simdata does. Before, each edge's crossing there was its own
    point, and the cells between them had no area or length: 702 of 45k
    triangles on a 1M-hexahedron grid whose plane of points lay on the level,
    23% of the triangles on 10k tetrahedra. The area is unchanged; the cell
    and point counts now differ from VTK's and Viskores', which keep them.
    On every path: shapes, polyhedra, 2D lines and per-cell (L2) lines.

  • A topology made from a stored cell-types array groups its cells 16× faster
    on the CPU at 1M cells (36 → 2.2 ms), 7× at 100k: the types are counted by
    chunks into private tables, with no atomics. With one counter per type, a
    mesh of one type, as most readers give, put every worker on one atomic
    counter. A fresh unstructured mesh paid this before its first filter
    (found comparing with warp-simdata on the CPU).

  • gradients and flow_quantities work on polyhedra, which have no
    reference element: each cell's gradient is Green–Gauss over its faces,
    each fanned into triangles from the mean of its points, which is exact for
    a linear field and, on a parallelepiped, the trilinear gradient at its
    center. At the points it is the average of the cells' gradients, as on
    shapes. Polygons and polylines get zero, as lower-dimensional cells do.

  • cell_centers and values_at_centers work on polyhedra, polygons and
    polylines: a cell's center is its cell point, the mean of its distinct
    points, and point data takes the mean of their values there (the value
    of an interpolant linear on tetrahedra fanned from that point, so exact
    for a linear field); cell data is its own value.

  • CellLocator and probe work on polyhedra, locating points in
    sub-tetrahedra: each face fanned from its points' mean, the fans from
    the mean of the cell's points. Point data is linear on each (exact for
    a linear field, continuous across faces). find returns (cells, tets, pcs) there; field views evaluate with tet_value(c, t, pc), and the
    cell view gives locate, cell_point, face_point and tet_jacobian.
    A cell not star-shaped from its cell point is refused.

  • advect and streamlines run through polyhedra, located in the same
    sub-tetrahedra (the kernel's _PolyhedralMesh reuses the views'
    location code), with point velocities linear on each and cell
    velocities constant: RK4 through a linear field is exact, and leaving
    the mesh bisects and pushes out as on shapes.

  • Implicit topology arrays. An UnstructuredTopology's types, offsets
    and connectivity may be implicit arrays, kept as they are rather than
    stored: a ConstantArray of types for cells all of one shape, a
    CountingArray of offsets for cells all of one size, and a
    CountingArray connectivity for cells that each have their own points
    (tack.dataset.topology.uniform_cells makes the first two). Filters
    whose cells are all one shape now give them: contour and slice,
    tetrahedralize, triangulate, extract_edges, the point extractions
    and probe (vertex cells: all three implicit), and threshold,
    extract_cells, extract_geometry and mask of a topology of one shape.
    A million triangles store their 12 MB of connectivity and nothing else,
    against 5 MB more before. dataset_to_vtk and vtk_to_dataset map them
    to and from VTK's vtkConstantArray and vtkAffineArray, through VTK's
    pythonic API (ug.cells = (type, vtkCellArray(offsets=..., connectivity=...)), cells.offsets_array, grid.cell_types). Code that
    reads a topology's arrays as tack.Arrays should use numpy(),
    np.asarray or tack.arrays.materialize, which accept both.

  • Breaking: CellLocator.find returns Locations(cells, pcs, tets), a
    named tuple of three on every dataset, where it returned (cells, pcs),
    or (cells, tets, pcs) on polyhedra. tets is None except on
    polyhedra. cells, pcs = locator.find(p) becomes
    cells, pcs, _ = locator.find(p), or reads .cells and .pcs.

  • CellLocator.find and probe search from the points where cells can be read
    by id (tack.dataset.by_id: an order-1 geometry over the linear shapes,
    structured grids, polyhedra): one launch over the points instead of two
    passes over the cells and two groupings, and probe reads point and cell
    data by cell id, one launch per field. Advection shares the same search
    and evaluation. Probing 100k points warm: 13.5 → 5.8 ms (10k cells) and
    30 → 16 ms (1M) on an M1 Max's CPU, 15.3 → 8.3 ms and 23.5 → 6–8 ms on its
    GPU. The cell-side search remains for what by_id cannot serve, and for a
    fresh locator's first query of few points in many cells, which it answers
    without building the cell bins; both give the same answer.

  • Structured grids derive their faces and edges by arithmetic, in one pass
    over the cells with no sort: ids, sides, orientations and signs identical to
    the general derivation's. On a grid of 1M hexahedra, faces 24 → 7.9 ms and
    edges 22 → 9.5 ms on an M1 Max's CPU, 32 → 13.6 ms and 35 → 7.8 ms on its
    GPU; up to 7× on small grids.

  • CellLocator, probe and advection run on polygons, and CellLocator and
    probe on polylines. A polygon is fanned from the mean of its points into
    triangles, and a point within the tolerance of one is located there (held
    when its projection falls in it), point data linear on each (exact for a
    linear field); a polyline locates on its segments, point data linear along
    them. Polygons not star-shaped from their mean point are refused, as such
    polyhedra are.