[flang][runtime] Fix off-by-one error in EX0.0 output editing

[klausler]

The maximum number of significant hexadecimal digits in EX0.0 REAL output editing is 29, not 28. Fix by computing it at build time from the precision of REAL(16).

[llvmbot]

@llvm/pr-subscribers-flang-runtime

Author: Peter Klausler (klausler)

Changes

The maximum number of significant hexadecimal digits in EX0.0 REAL output editing is 29, not 28. Fix by computing it at build time from the precision of REAL(16).

---

Full diff: https://github.com/llvm/llvm-project/pull/85428.diff

1 Files Affected:

• (modified) flang/runtime/edit-output.cpp (+4-1)

diff --git a/flang/runtime/edit-output.cpp b/flang/runtime/edit-output.cpp
index b474c8cd91bae4..7267540370fc07 100644
--- a/flang/runtime/edit-output.cpp
+++ b/flang/runtime/edit-output.cpp
@@ -9,6 +9,7 @@
 #include "edit-output.h"
 #include "emit-encoded.h"
 #include "utf.h"
+#include "flang/Common/real.h"
 #include "flang/Common/uint128.h"
 #include <algorithm>
 
@@ -700,7 +701,9 @@ bool RealOutputEditing<KIND>::EditEXOutput(const DataEdit &edit) {
   if ((editWidth == 0 && !edit.digits) || editDigits == 0) {
     // EX0 or EXw.0
     flags |= decimal::Minimize;
-    significantDigits = 28; // enough for 128-bit F.P.
+    static constexpr int maxSigHexDigits{
+        (common::PrecisionOfRealKind(16) + 3) / 4};
+    significantDigits = maxSigHexDigits;
   }
   auto converted{
       ConvertToHexadecimal(significantDigits, edit.modes.round, flags)};

[vdonaldson]

Thanks!