Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Float default precision #9547

Closed
mjonss opened this issue Jul 11, 2022 · 2 comments · Fixed by #9581
Closed

Float default precision #9547

mjonss opened this issue Jul 11, 2022 · 2 comments · Fixed by #9581
Assignees
Labels
area/sql-infra Indicates that the Issue or PR belongs to the area of sql-infra and sql-metadata.

Comments

@mjonss
Copy link
Contributor

mjonss commented Jul 11, 2022

Error Report

Please answer the following questions before submitting your issue. Thanks!

  1. What is the URL/path of the document related to this issue?
    https://docs.pingcap.com/tidb/stable/data-type-numeric

  2. How would you like to improve it?
    The documentation speaks of default precision like this:
    Note
    As in MySQL, the FLOAT data type stores approximate values. For values such as currency, it is recommended to use the DECIMAL type instead. In TiDB, the default precision of the FLOAT data type is 8 bits, but in MySQL, the default precision is 6 bits. For example, assuming that you insert 123456789 and 1.23456789 into columns of the FLOAT type in both TiDB and MySQL, when you query the corresponding values in MySQL, you get 123457000 and 1.23457, while in TiDB, you get 123456790 and 1.2345679

Here I think "precision" mean number of decimal digits (base-10) the value is rounded to and it is not 6 or 8 bits, but rather 6 or 8 digits (in base 10).

(I intentionally left out the deeper technical details of the protocol and how it is stored internally).

@shichun-0415 shichun-0415 added the area/sql-infra Indicates that the Issue or PR belongs to the area of sql-infra and sql-metadata. label Jul 12, 2022
@shichun-0415
Copy link
Contributor

@mjonss Thank you for the feedback. @djshow832 Could you please take a look at this issue? Thanks.

@djshow832
Copy link
Contributor

https://dev.mysql.com/doc/refman/8.0/en/fixed-point-types.html

The precision represents the number of significant digits that are stored for values, and the scale represents the number of digits that can be stored following the decimal point.

Exactly! It should be 'digits' rather than 'bits'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/sql-infra Indicates that the Issue or PR belongs to the area of sql-infra and sql-metadata.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants