Web lists-archives.com

Re: mysql float data type




On Tue, 16 Dec 2014, Hartmut Holzgraefe wrote:
On 16.12.2014 15:16, xiangdongzou wrote:

 Can anyone tell me why 531808.11 has been changed to 531808.12 ?

typical decimal->binary->decimal conversion/rounding error.

never used DECIMAL nor intend to, but the issue is typical of precision issues among float (32-bit) and double (64-bit) in ANY programming language. Google for "IEEE floating point"

Some (most) users are unaware that a 32-bit real (REAL*4 for oldtimer Fortran users like myself) have about 7 digits of precision, while 64-bit (doubles, double precision, REAL*8) get to about 16.

So if a quantity needs high precision (typically this occurs for angular quantities where arcseconds are important), use double.

--
------------------------------------------------------------------------
Lucio Chiappetti - INAF/IASF - via Bassini 15 - I-20133 Milano (Italy)
For more info : http://www.iasf-milano.inaf.it/~lucio/personal.html
------------------------------------------------------------------------
Do not like Firefox >=29 ?  Get Pale Moon !  http://www.palemoon.org

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:    http://lists.mysql.com/mysql