Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Num.float_of_num depends on Arith_status.set_floating_precision #6896

Closed
vicuna opened this issue Jun 7, 2015 · 2 comments

Comments

Projects
None yet
1 participant
@vicuna
Copy link

commented Jun 7, 2015

Original bug ID: 6896
Reporter: @edwintorok
Status: closed (set by @xavierleroy on 2017-02-16T14:15:04Z)
Resolution: fixed
Priority: normal
Severity: minor
Target version: 4.03.0+dev / +beta1
Fixed in version: 4.03.0+dev / +beta1
Category: documentation
Tags: junior_job
Related to: #5021
Monitored by: @gasche

Bug description

Num.float_of_num depends on the precision set with Arith_status.set_floating_precision, but that is not immediately obvious from the documentation: "This parameter is the number of digits displayed when approx_printing is on". approx_printing is not on in the example below.

Please document in Num.float_of_num that it depends on Arith_status.set_floating_precision, and in Arith_status.set_floating_precision that it influences Num.float_of_num.

Steps to reproduce

#use "topfind";;
#require "num";;
(* enable this to increase float_of_num precision:
Arith_status.set_floating_precision 13;; *)
let a = 32012049433 and b = 8 in
let n = Num.div_num (Num.Int a) (Num.Int b) in
let f = Num.float_of_num n in let f0 = (float_of_int a) /. (float_of_int b) in
Printf.eprintf "%g\n" (f -. f0)

@vicuna

This comment has been minimized.

Copy link
Author

commented Jun 7, 2015

Comment author: @gasche

Arguably the implementation is wrong : I think the behaviour you observe is an artefact of the implementation of (Ratio.float_of_ratio r) as (float_of_string (string_of_ratio r)). There are certainly direct ways to convert a rational into a string, but at least there should be a way to obtain a float (or string representation of a float) that provides the maximal precision available for double.

@vicuna

This comment has been minimized.

Copy link
Author

commented Jul 24, 2015

Comment author: @xavierleroy

Fixed in commit 16247 by a complete reimplementation of float_of_big_int and float_of_ratio. The old implementation (going through strings) was embarrassingly bad.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.