Skip to content

Hex quantization silently returns garbage (INT64_MIN-scale values) for NaN/Inf input instead of erroring #36

Description

@gcol33

Hex quantization silently returns garbage (INT64_MIN-scale values) for NaN/Inf input instead of erroring

Defect

grid_math.h's quantize_rotation_classI/II/III_A/III_B (called from aperture.cpp's hex_quantize_ap3/4/7, exposed via rcpp_aperture.cpp's quantize_impl backing cpp_hex_quantize_ap3/4/7) feed x, y straight into CubeCoord::round_to_nearest() (src/cube_coordinates.h:51-72), which calls std::llround(q) with no finiteness check. std::llround(NaN) is unspecified behavior and yields a large garbage sentinel rather than an error.

This is the same bug class fixed for issue #6, but that fix only covered coordinate_transforms.cpp's quantize_class1, which explicitly checks std::isfinite(x) || !std::isfinite(y) and short-circuits to (0,0) (lines 233-239). rcpp_aperture.cpp's checked_ij() guards the output i/j after casting, but quantize_impl() never validates icosa_triangle_x/icosa_triangle_y before calling hexify::hex_quantize.

Reproduction

cpp_hex_quantize_ap7(NaN, NaN, resolution = 3)
#             i             j
# -9.223372e+18 -9.223372e+18

No error, no warning — a plausible-looking but completely bogus (i, j) cell is returned silently.

Root cause

Two parallel, duplicated hex-quantization implementations exist (coordinate_transforms.cpp's anonymous-namespace quantize_class1/class2/class3i/class3ii vs. grid_math.h's quantize_rotation_classI/II/III_A/III_B), and only one received the NaN guard. This is the exact "no copy-paste across specialized functions" duplication this project's own engineering principles flag — consolidating into one shared, NaN-guarded quantization primitive would prevent this class of gap recurring.

Fix

Add the same std::isfinite() guard used in coordinate_transforms.cpp::quantize_class1 to the grid_math.h quantization path (or better: deduplicate the two implementations into one shared, NaN-guarded helper, per this project's own no-copy-paste-across-specialized-functions principle).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions