Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Fix float data conversion to string #1632

Merged
merged 2 commits into from

3 participants

@strk

Fixes precision digits, closing #430
Also avoids forcing a trailing '.0', closing #1627

Compatibility is still not 100% matching 2.0.x because in two cases:

  • Scientific notation is never used
  • Decimal digits after 16th are not printed

See http://strk.keybit.net/tmp/report_full.txt

@strk strk Fix float data conversion to string
Fixes precision digits, closing #430
Also avoids forcing a trailing '.0', closing #1627
b51b357
@strk

NOTE: this commit breaks a test, but I think the test is bogus, as it expects a floating number 1 to be represented with a trailing zero, which is what #1627 was filed to complain about (backward compatibility issue)

@springmeyer
Owner

Can you create a C++ test file to go along with this commit to capture all the testing work and prevent regressions?

@strk

ok, working on it

@strk

One thing: should I expect what the stringstream version gave us or can we upgrade expectance of some of the results ?

For example stringstream did use scientific notation for some values, while the karma version never does.
Also the karma version forces a limit on the number of decimal digits to std::numeric_limits<T>::digits10 + 1

@strk strk Add test for backward compatibility of double to string conversion
The testcase is ready to host more conversion tests but is currently
really only targetting the double-to-string.

refs #430, #1632
9416a57
@strk

So I added the test expecting 100% backward compatibility. But the code isn't there yet (would need more overrides)

@strk

One of the incompatibilities (number of decimal digits printed) is IMHO due to a bug in boost:
https://svn.boost.org/trac/boost/ticket/7785

We could workaround it by writing our own call_n function, but I dunno if it's worth the trouble.

@strk

I don't get how TravisCI doesn't see the test failures, btw, there must be something broken there

@springmeyer
Owner

Travis does not run the tests

@springmeyer
Owner

@artemp - could you review this?

@artemp
Owner

@strk - Thanks for implementing this!

@artemp artemp merged commit e60a3f7 into mapnik:master

1 check passed

Details default The Travis build passed
@artemp
Owner

@strk @springmeyer - couple issues :

log10 (0.0) returns -inf
log10 of negative number returns nan

@strk

artemp: could you update the testcase with those numbers ? I'll look at them on monday if nobody beats me to it

@artemp
Owner

@strk - done in 9453d93

@strk

@artemp I tried those number against my offline tester and they don't seem to be giving any problem.
I do see the -inf return, but it's converted to 0 when cast to integer, and I do not get any nan on negative values.
This is with GCC 4.6.3

I see no problem with handling those cases (check for zero and fabs the N in precision function), but being unable to reproduce the problem I'm not the best person to do that.

@artemp
Owner

@strk - ok, I'll have a look

@strk

@artem I worked on a version also doing scientific notation, so I'll be requesting a new pull shortly, also fixing the inf/nan cases.

The more I work with karma the less I think it's appropriate to perform parametrized formatting. All those compile-time policies don't seem to play nicely with runtime determined policies...

@strk

This is the updated report_full.txt with my latest changes (for scientific notation):
http://strk.keybit.net/tmp/report_full.txt

I think we do the 9.000000000000001e-05 better than stringstream as the input number was 0.9e-4

@artemp
Owner

@strk - great, btw this what works for me in wkt generator:

template <typename T>
struct wkt_coordinate_policy : karma::real_policies<T>
{
    typedef boost::spirit::karma::real_policies<T> base_type;
    static int floatfield(T n) { return base_type::fmtflags::fixed; }
    static unsigned precision(T n)
    {
        if (n == 0.0) return 0;
        return static_cast<unsigned>(15 - boost::math::trunc(log10(std::fabs(n))));
    }
    template <typename OutputIterator>
    static bool dot(OutputIterator& sink, T n, unsigned precision)
    {
        if (n == 0) return true;
        return base_type::dot(sink, n, precision);
    }
    template <typename OutputIterator>
    static bool fraction_part(OutputIterator& sink, T n
                       , unsigned adjprec, unsigned precision)
    {
        if (n == 0) return true;
        return base_type::fraction_part(sink, n, adjprec, precision);
    }
};
@PetrDlouhy PetrDlouhy referenced this pull request from a commit in PetrDlouhy/mapnik
@strk strk Add test for backward compatibility of double to string conversion
The testcase is ready to host more conversion tests but is currently
really only targetting the double-to-string.

refs #430, #1632
8a0fdf9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 6, 2012
  1. @strk

    Fix float data conversion to string

    strk authored
    Fixes precision digits, closing #430
    Also avoids forcing a trailing '.0', closing #1627
Commits on Dec 11, 2012
  1. @strk

    Add test for backward compatibility of double to string conversion

    strk authored
    The testcase is ready to host more conversion tests but is currently
    really only targetting the double-to-string.
    
    refs #430, #1632
This page is out of date. Refresh to see the latest.
View
8 include/mapnik/util/conversions.hpp
@@ -68,7 +68,11 @@ struct double_policy : boost::spirit::karma::real_policies<T>
{
typedef boost::spirit::karma::real_policies<T> base_type;
static int floatfield(T n) { return base_type::fmtflags::fixed; }
- static unsigned precision(T n) { return 16 ;}
+ static unsigned precision(T n) { return 15 - trunc(log10(n)); }
+ template <typename OutputIterator>
+ static bool dot(OutputIterator& sink, T n, unsigned precision) {
+ return n ? *sink = '.', true : false;
+ }
};
@@ -77,7 +81,7 @@ template <>
inline bool to_string(std::string & str, double value)
{
namespace karma = boost::spirit::karma;
- typedef boost::spirit::karma::real_generator<double, double_policy<double> > double_type;
+ typedef karma::real_generator<double, double_policy<double> > double_type;
std::back_insert_iterator<std::string> sink(str);
return karma::generate(sink, double_type(), value);
}
View
76 tests/cpp_tests/conversions_test.cpp
@@ -0,0 +1,76 @@
+#include <boost/version.hpp>
+#include <mapnik/util/conversions.hpp>
+#include <boost/detail/lightweight_test.hpp>
+#include <iostream>
+
+int main( int, char*[] )
+{
+ using mapnik::util::to_string;
+
+ std::string out;
+
+ // Test double
+
+ to_string(out, double(1));
+ BOOST_TEST_EQ( out, "1" );
+ out.clear();
+
+ to_string(out, double(0.1));
+ BOOST_TEST_EQ( out, "0.1" );
+ out.clear();
+
+ to_string(out, double(0.123));
+ BOOST_TEST_EQ( out, "0.123" );
+ out.clear();
+
+ to_string(out, double(1e-06));
+ BOOST_TEST_EQ( out, "1e-06" );
+ out.clear();
+
+ to_string(out, double(1e-05));
+ BOOST_TEST_EQ( out, "0.00001" );
+ out.clear();
+
+ to_string(out, double(0.0001));
+ BOOST_TEST_EQ( out, "0.0001" );
+ out.clear();
+
+ to_string(out, double(0.0001234567890123456));
+ BOOST_TEST_EQ( out, "0.0001234567890123456" );
+ out.clear();
+
+ to_string(out, double(1000000000000000));
+ BOOST_TEST_EQ( out, "1000000000000000" );
+ out.clear();
+
+ to_string(out, double(100000000000000.1));
+ BOOST_TEST_EQ( out, "100000000000000.1" );
+ out.clear();
+
+ to_string(out, double(1.00001));
+ BOOST_TEST_EQ( out, "1.00001" );
+ out.clear();
+
+ to_string(out, double(1234000000000000));
+ BOOST_TEST_EQ( out, "1234000000000000" );
+ out.clear();
+
+ to_string(out, double(1.234e+16));
+ BOOST_TEST_EQ( out, "1.234e+16" );
+ out.clear();
+
+ // Test int
+
+ to_string(out, int(2));
+ BOOST_TEST_EQ( out, "2" );
+ out.clear();
+
+ if (!::boost::detail::test_errors()) {
+ std::clog << "C++ exceptions: \x1b[1;32m✓ \x1b[0m\n";
+#if BOOST_VERSION >= 104600
+ ::boost::detail::report_errors_remind().called_report_errors_function = true;
+#endif
+ } else {
+ return ::boost::report_errors();
+ }
+}
Something went wrong with that request. Please try again.