Characterize limits of DD orientation predicate soundness over arbitrary doubles (#1106)

[grootstebozewolf]

Summary

This PR adds comprehensive test infrastructure and empirical characterization of the robustness limits of JTS's orientation predicate (Orientation.index), directly addressing the core request in #1106.

It does not change any public API or production behavior.

Key Findings

• The current DD-backed implementation (CGAlgorithmsDD.orientationIndex → DD) is sound for coordinates roughly in the band [2^-511, 2^511].
• Outside this band it fails for a fundamental IEEE-754 reason:
  • Overflow (|coord| ≳ 2^512): intermediate products in DD.selfMultiply become +∞, the determinant evaluates to NaN, and signum() returns 0 (collinear) for genuinely non-collinear input.
  • Underflow (|coord| ≲ 2^-512): products flush to zero → same incorrect collinear result.
• These are characterization results, not endorsements of the limitation. A fully robust predicate over all finite doubles would need coordinate scaling or a wider-range exact path.

The work also validates the historical 64-bit integer soundness claim for integer coordinates |c| ≤ 2^25 (the domain for which a short machine-checkable argument exists).

What This PR Delivers

• DDCounterexampleHunter — a powerful adversarial search framework with multiple generators (nearCollinear, productCollision, extremeMagnitude, etc.) and three side-by-side predicates (current DD, legacy RobustDeterminant, naive double).
• OrientationDDRobustnessTest — JUnit suite that proves the hunter is effective, asserts soundness inside the safe band (millions of cases), and explicitly demonstrates the overflow/underflow failures.
• KNOWN_COUNTEREXAMPLES + extremeMagnitude generator — reproducible demonstrations of the exact failure modes.
• ShewchukExpansionExactnessTest — property-based verification of the Shewchuk expansion primitives (exactness vs BigDecimal + structural invariants of fast_expansion_sum_zeroelim). Also documents a masked porting detail in the zero-elimination routine.
• RocqRefRunner + RocqRefRunnerTest — self-contained exact oracles:
  • refSignExact(...) using BigDecimal (exact for any finite dyadic double — the R² reference)
  • Fast certified 64-bit path for the integer domain + BigInteger cross-check
  • loadProofCases(...) that validates imported Rocq vectors against the Java reference (a corrupt export cannot silently weaken the tests)
• orientation_proof_vectors.txt — drop-in slot for future exports from the Rocq orientation-soundness development (currently populated with manually-derived certified cases).

All new tests are fast enough for CI while the main methods in the hunter support very large-scale evidence gathering.

Practical Impact

For all normal GIS, CAD, graphics, and most scientific workloads the coordinate magnitudes are many orders of magnitude below the failure threshold. This work documents a real theoretical limitation with excellent evidence rather than discovering a bug that affects everyday users.

Relationship to Formal Methods

The RocqRefRunner is designed as a bridge to the separate Rocq (Coq) orientation-soundness proof development. The Java side can consume and independently validate proof vectors, allowing the formal guarantee (for the integer domain) to be exercised as ordinary JUnit tests.

Testing

• New JUnit tests in modules/core.
• Run with: mvn test -pl modules/core -Dtest=OrientationDDRobustnessTest,ShewchukExpansionExactnessTest,RocqRefRunnerTest
• The hunter's main method provides extended evidence (millions of adversarial cases).

Contribution Checklist

• Eclipse Contributor Agreement signed
• Commits signed off (-s)
• Suggested labels: enhancement, jts-core, tests

---

Addresses #1106 (orientation soundness over arbitrary doubles).

Feedback welcome on preferred mitigation strategies (pre-scaling, adaptive BigDecimal fallback on detected overflow, etc.) if/when a production fix is desired in the future.