You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
How would you like to improve it?
The documentation speaks of default precision like this:
Note
As in MySQL, the FLOAT data type stores approximate values. For values such as currency, it is recommended to use the DECIMAL type instead. In TiDB, the default precision of the FLOAT data type is 8 bits, but in MySQL, the default precision is 6 bits. For example, assuming that you insert 123456789 and 1.23456789 into columns of the FLOAT type in both TiDB and MySQL, when you query the corresponding values in MySQL, you get 123457000 and 1.23457, while in TiDB, you get 123456790 and 1.2345679
Here I think "precision" mean number of decimal digits (base-10) the value is rounded to and it is not 6 or 8 bits, but rather 6 or 8 digits (in base 10).
(I intentionally left out the deeper technical details of the protocol and how it is stored internally).
The text was updated successfully, but these errors were encountered:
The precision represents the number of significant digits that are stored for values, and the scale represents the number of digits that can be stored following the decimal point.
Exactly! It should be 'digits' rather than 'bits'.
Error Report
Please answer the following questions before submitting your issue. Thanks!
What is the URL/path of the document related to this issue?
https://docs.pingcap.com/tidb/stable/data-type-numeric
How would you like to improve it?
The documentation speaks of default precision like this:
Note
As in MySQL, the FLOAT data type stores approximate values. For values such as currency, it is recommended to use the DECIMAL type instead. In TiDB, the default precision of the FLOAT data type is 8 bits, but in MySQL, the default precision is 6 bits. For example, assuming that you insert 123456789 and 1.23456789 into columns of the FLOAT type in both TiDB and MySQL, when you query the corresponding values in MySQL, you get 123457000 and 1.23457, while in TiDB, you get 123456790 and 1.2345679
Here I think "precision" mean number of decimal digits (base-10) the value is rounded to and it is not 6 or 8 bits, but rather 6 or 8 digits (in base 10).
(I intentionally left out the deeper technical details of the protocol and how it is stored internally).
The text was updated successfully, but these errors were encountered: