mod.rs

//! Converting decimal strings into IEEE 754 binary floating point numbers.
//!
//! # Problem statement
//!
//! We are given a decimal string such as `12.34e56`. This string consists of integral (`12`),
//! fractional (`34`), and exponent (`56`) parts. All parts are optional and interpreted as zero
//! when missing.
//!
//! We seek the IEEE 754 floating point number that is closest to the exact value of the decimal
//! string. It is well-known that many decimal strings do not have terminating representations in
//! base two, so we round to 0.5 units in the last place (in other words, as well as possible).
//! Ties, decimal values exactly half-way between two consecutive floats, are resolved with the
//! half-to-even strategy, also known as banker's rounding.
//!
//! Needless to say, this is quite hard, both in terms of implementation complexity and in terms
//! of CPU cycles taken.
//!
//! # Implementation
//!
//! First, we ignore signs. Or rather, we remove it at the very beginning of the conversion
//! process and re-apply it at the very end. This is correct in all edge cases since IEEE
//! floats are symmetric around zero, negating one simply flips the first bit.
//!
//! Then we remove the decimal point by adjusting the exponent: Conceptually, `12.34e56` turns
//! into `1234e54`, which we describe with a positive integer `f = 1234` and an integer `e = 54`.
//! The `(f, e)` representation is used by almost all code past the parsing stage.
//!
//! We then try a long chain of progressively more general and expensive special cases using
//! machine-sized integers and small, fixed-sized floating point numbers (first `f32`/`f64`, then
//! a type with 64 bit significand, `Fp`). When all these fail, we bite the bullet and resort to a
//! simple but very slow algorithm that involved computing `f * 10^e` fully and doing an iterative
//! search for the best approximation.
//!
//! Primarily, this module and its children implement the algorithms described in:
//! "How to Read Floating Point Numbers Accurately" by William D. Clinger,
//! available online: <http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.45.4152>
//!
//! In addition, there are numerous helper functions that are used in the paper but not available
//! in Rust (or at least in core). Our version is additionally complicated by the need to handle
//! overflow and underflow and the desire to handle subnormal numbers. Bellerophon and
//! Algorithm R have trouble with overflow, subnormals, and underflow. We conservatively switch to
//! Algorithm M (with the modifications described in section 8 of the paper) well before the
//! inputs get into the critical region.
//!
//! Another aspect that needs attention is the ``RawFloat`` trait by which almost all functions
//! are parametrized. One might think that it's enough to parse to `f64` and cast the result to
//! `f32`. Unfortunately this is not the world we live in, and this has nothing to do with using
//! base two or half-to-even rounding.
//!
//! Consider for example two types `d2` and `d4` representing a decimal type with two decimal
//! digits and four decimal digits each and take "0.01499" as input. Let's use half-up rounding.
//! Going directly to two decimal digits gives `0.01`, but if we round to four digits first,
//! we get `0.0150`, which is then rounded up to `0.02`. The same principle applies to other
//! operations as well, if you want 0.5 ULP accuracy you need to do *everything* in full precision
//! and round *exactly once, at the end*, by considering all truncated bits at once.
//!
//! FIXME: Although some code duplication is necessary, perhaps parts of the code could be shuffled
//! around such that less code is duplicated. Large parts of the algorithms are independent of the
//! float type to output, or only needs access to a few constants, which could be passed in as
//! parameters.
//!
//! # Other
//!
//! The conversion should *never* panic. There are assertions and explicit panics in the code,
//! but they should never be triggered and only serve as internal sanity checks. Any panics should
//! be considered a bug.
//!
//! There are unit tests but they are woefully inadequate at ensuring correctness, they only cover
//! a small percentage of possible errors. Far more extensive tests are located in the directory
//! `src/etc/test-float-parse` as a Python script.
//!
//! A note on integer overflow: Many parts of this file perform arithmetic with the decimal
//! exponent `e`. Primarily, we shift the decimal point around: Before the first decimal digit,
//! after the last decimal digit, and so on. This could overflow if done carelessly. We rely on
//! the parsing submodule to only hand out sufficiently small exponents, where "sufficient" means
//! "such that the exponent +/- the number of decimal digits fits into a 64 bit integer".
//! Larger exponents are accepted, but we don't do arithmetic with them, they are immediately
//! turned into {positive,negative} {zero,infinity}.

#![doc(hidden)]
#![unstable(
    feature = "dec2flt",
    reason = "internal routines only exposed for testing",
    issue = "none"
)]

use crate::fmt;
use crate::str::FromStr;

use self::num::digits_to_big;
use self::parse::{parse_decimal, Decimal, ParseResult, Sign};
use self::rawfp::RawFloat;

mod algorithm;
mod num;
mod table;
// These two have their own tests.
pub mod parse;
pub mod rawfp;

macro_rules! from_str_float_impl {
    ($t:ty) => {
        #[stable(feature = "rust1", since = "1.0.0")]
        impl FromStr for $t {
            type Err = ParseFloatError;

            /// Converts a string in base 10 to a float.
            /// Accepts an optional decimal exponent.
            ///
            /// This function accepts strings such as
            ///
            /// * '3.14'
            /// * '-3.14'
            /// * '2.5E10', or equivalently, '2.5e10'
            /// * '2.5E-10'
            /// * '5.'
            /// * '.5', or, equivalently, '0.5'
            /// * 'inf', '-inf', 'NaN'
            ///
            /// Leading and trailing whitespace represent an error.
            ///
            /// # Grammar
            ///
            /// All strings that adhere to the following [EBNF] grammar
            /// will result in an [`Ok`] being returned:
            ///
            /// ```txt
            /// Float  ::= Sign? ( 'inf' | 'NaN' | Number )
            /// Number ::= ( Digit+ |
            ///              Digit+ '.' Digit* |
            ///              Digit* '.' Digit+ ) Exp?
            /// Exp    ::= [eE] Sign? Digit+
            /// Sign   ::= [+-]
            /// Digit  ::= [0-9]
            /// ```
            ///
            /// [EBNF]: https://www.w3.org/TR/REC-xml/#sec-notation
            ///
            /// # Known bugs
            ///
            /// In some situations, some strings that should create a valid float
            /// instead return an error. See [issue #31407] for details.
            ///
            /// [issue #31407]: https://github.com/rust-lang/rust/issues/31407
            ///
            /// # Arguments
            ///
            /// * src - A string
            ///
            /// # Return value
            ///
            /// `Err(ParseFloatError)` if the string did not represent a valid
            /// number. Otherwise, `Ok(n)` where `n` is the floating-point
            /// number represented by `src`.
            #[inline]
            fn from_str(src: &str) -> Result<Self, ParseFloatError> {
                dec2flt(src)
            }
        }
    };
}
from_str_float_impl!(f32);
from_str_float_impl!(f64);

/// An error which can be returned when parsing a float.
///
/// This error is used as the error type for the [`FromStr`] implementation
/// for [`f32`] and [`f64`].
///
/// # Example
///
/// ```
/// use std::str::FromStr;
///
/// if let Err(e) = f64::from_str("a.12") {
///     println!("Failed conversion to f64: {}", e);
/// }
/// ```
#[derive(Debug, Clone, PartialEq, Eq)]
#[stable(feature = "rust1", since = "1.0.0")]
pub struct ParseFloatError {
    kind: FloatErrorKind,
}

#[derive(Debug, Clone, PartialEq, Eq)]
enum FloatErrorKind {
    Empty,
    Invalid,
}

impl ParseFloatError {
    #[unstable(
        feature = "int_error_internals",
        reason = "available through Error trait and this method should \
                  not be exposed publicly",
        issue = "none"
    )]
    #[doc(hidden)]
    pub fn __description(&self) -> &str {
        match self.kind {
            FloatErrorKind::Empty => "cannot parse float from empty string",
            FloatErrorKind::Invalid => "invalid float literal",
        }
    }
}

#[stable(feature = "rust1", since = "1.0.0")]
impl fmt::Display for ParseFloatError {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        self.__description().fmt(f)
    }
}

fn pfe_empty() -> ParseFloatError {
    ParseFloatError { kind: FloatErrorKind::Empty }
}

fn pfe_invalid() -> ParseFloatError {
    ParseFloatError { kind: FloatErrorKind::Invalid }
}

/// Splits a decimal string into sign and the rest, without inspecting or validating the rest.
fn extract_sign(s: &str) -> (Sign, &str) {
    match s.as_bytes()[0] {
        b'+' => (Sign::Positive, &s[1..]),
        b'-' => (Sign::Negative, &s[1..]),
        // If the string is invalid, we never use the sign, so we don't need to validate here.
        _ => (Sign::Positive, s),
    }
}

/// Converts a decimal string into a floating point number.
fn dec2flt<T: RawFloat>(s: &str) -> Result<T, ParseFloatError> {
    if s.is_empty() {
        return Err(pfe_empty());
    }
    let (sign, s) = extract_sign(s);
    let flt = match parse_decimal(s) {
        ParseResult::Valid(decimal) => convert(decimal)?,
        ParseResult::ShortcutToInf => T::INFINITY,
        ParseResult::ShortcutToZero => T::ZERO,
        ParseResult::Invalid => {
            if s.eq_ignore_ascii_case("nan") {
                T::NAN
            } else if s.eq_ignore_ascii_case("inf") || s.eq_ignore_ascii_case("infinity") {
                T::INFINITY
            } else {
                return Err(pfe_invalid());
            }
        }
    };

    match sign {
        Sign::Positive => Ok(flt),
        Sign::Negative => Ok(-flt),
    }
}

/// The main workhorse for the decimal-to-float conversion: Orchestrate all the preprocessing
/// and figure out which algorithm should do the actual conversion.
fn convert<T: RawFloat>(mut decimal: Decimal<'_>) -> Result<T, ParseFloatError> {
    simplify(&mut decimal);
    if let Some(x) = trivial_cases(&decimal) {
        return Ok(x);
    }
    // Remove/shift out the decimal point.
    let e = decimal.exp - decimal.fractional.len() as i64;
    if let Some(x) = algorithm::fast_path(decimal.integral, decimal.fractional, e) {
        return Ok(x);
    }
    // Big32x40 is limited to 1280 bits, which translates to about 385 decimal digits.
    // If we exceed this, we'll crash, so we error out before getting too close (within 10^10).
    let upper_bound = bound_intermediate_digits(&decimal, e);
    if upper_bound > 375 {
        return Err(pfe_invalid());
    }
    let f = digits_to_big(decimal.integral, decimal.fractional);

    // Now the exponent certainly fits in 16 bit, which is used throughout the main algorithms.
    let e = e as i16;
    // FIXME These bounds are rather conservative. A more careful analysis of the failure modes
    // of Bellerophon could allow using it in more cases for a massive speed up.
    let exponent_in_range = table::MIN_E <= e && e <= table::MAX_E;
    let value_in_range = upper_bound <= T::MAX_NORMAL_DIGITS as u64;
    if exponent_in_range && value_in_range {
        Ok(algorithm::bellerophon(&f, e))
    } else {
        Ok(algorithm::algorithm_m(&f, e))
    }
}

// As written, this optimizes badly (see #27130, though it refers to an old version of the code).
// `inline(always)` is a workaround for that. There are only two call sites overall and it doesn't
// make code size worse.

/// Strip zeros where possible, even when this requires changing the exponent
#[inline(always)]
fn simplify(decimal: &mut Decimal<'_>) {
    let is_zero = &|&&d: &&u8| -> bool { d == b'0' };
    // Trimming these zeros does not change anything but may enable the fast path (< 15 digits).
    let leading_zeros = decimal.integral.iter().take_while(is_zero).count();
    decimal.integral = &decimal.integral[leading_zeros..];
    let trailing_zeros = decimal.fractional.iter().rev().take_while(is_zero).count();
    let end = decimal.fractional.len() - trailing_zeros;
    decimal.fractional = &decimal.fractional[..end];
    // Simplify numbers of the form 0.0...x and x...0.0, adjusting the exponent accordingly.
    // This may not always be a win (possibly pushes some numbers out of the fast path), but it
    // simplifies other parts significantly (notably, approximating the magnitude of the value).
    if decimal.integral.is_empty() {
        let leading_zeros = decimal.fractional.iter().take_while(is_zero).count();
        decimal.fractional = &decimal.fractional[leading_zeros..];
        decimal.exp -= leading_zeros as i64;
    } else if decimal.fractional.is_empty() {
        let trailing_zeros = decimal.integral.iter().rev().take_while(is_zero).count();
        let end = decimal.integral.len() - trailing_zeros;
        decimal.integral = &decimal.integral[..end];
        decimal.exp += trailing_zeros as i64;
    }
}

/// Returns a quick-an-dirty upper bound on the size (log10) of the largest value that Algorithm R
/// and Algorithm M will compute while working on the given decimal.
fn bound_intermediate_digits(decimal: &Decimal<'_>, e: i64) -> u64 {
    // We don't need to worry too much about overflow here thanks to trivial_cases() and the
    // parser, which filter out the most extreme inputs for us.
    let f_len: u64 = decimal.integral.len() as u64 + decimal.fractional.len() as u64;
    if e >= 0 {
        // In the case e >= 0, both algorithms compute about `f * 10^e`. Algorithm R proceeds to
        // do some complicated calculations with this but we can ignore that for the upper bound
        // because it also reduces the fraction beforehand, so we have plenty of buffer there.
        f_len + (e as u64)
    } else {
        // If e < 0, Algorithm R does roughly the same thing, but Algorithm M differs:
        // It tries to find a positive number k such that `f << k / 10^e` is an in-range
        // significand. This will result in about `2^53 * f * 10^e` < `10^17 * f * 10^e`.
        // One input that triggers this is 0.33...33 (375 x 3).
        f_len + e.unsigned_abs() + 17
    }
}

/// Detects obvious overflows and underflows without even looking at the decimal digits.
fn trivial_cases<T: RawFloat>(decimal: &Decimal<'_>) -> Option<T> {
    // There were zeros but they were stripped by simplify()
    if decimal.integral.is_empty() && decimal.fractional.is_empty() {
        return Some(T::ZERO);
    }
    // This is a crude approximation of ceil(log10(the real value)). We don't need to worry too
    // much about overflow here because the input length is tiny (at least compared to 2^64) and
    // the parser already handles exponents whose absolute value is greater than 10^18
    // (which is still 10^19 short of 2^64).
    let max_place = decimal.exp + decimal.integral.len() as i64;
    if max_place > T::INF_CUTOFF {
        return Some(T::INFINITY);
    } else if max_place < T::ZERO_CUTOFF {
        return Some(T::ZERO);
    }
    None
}