Error

users
#1

blockquote {padding-left: 1ex; margin: 0px 0px 0px 0.8ex; border-left: #cccccc 1px solid;} p {margin: 0px;padding: 0px;}
Check =2^53 it shows 9007199254740990

#2

This is not an error!

You are using a finite computing machine - one in which each number in a cell is stored in a memory location of finite length. So there is always a limit to how large an integer can be stored and to how great a precision a floating-point value can be stored. Your computer is probably using "double-precision binary floating-point format", in which numbers are stored in eight-byte (sixty-four-bit) words. This allows around fifty-three significant binary digits, the rest being taken up by the sign and exponent. This level of precision corresponds to between fifteen and sixteen significant decimal digits, so integers can be stored precisely only up to this length of number, above which values can be stored, but only to around fifteen significant digits.

2^53 is one followed by fifty-three zeroes, of course - so is just too long to be stored accurately. You are very unlikely to need numbers of this magnitude as final results, and if you come across this problem, you can usually find a simple way to reorganise your calculation to avoid the problem.

Note that the (biased) exponent is stored in an eleven-bit field, so a separate problem occurs if you try to calculate with numbers above about 2^1023.

For a fuller explanation, see http://en.wikipedia.org/wiki/Double-precision_floating-point_format .

I trust this helps.

Brian Barker

#3

skiper skippers wrote

blockquote {padding-left: 1ex; margin: 0px 0px 0px 0.8ex; border-left:
#cccccc 1px solid;} p {margin: 0px;padding: 0px;}
Check =2^53 it shows 9007199254740990

FYI MS Excel returns the same value.
If you need accuracy then use Gnumeric or R

For those who can't spot the error, the result should be 9007199254740992 :slight_smile:

#4

Hi Brian,

While I agree with most of your post, a slight typo or something crept
in here, where you confuse 10^53 with 2^53

#5

I wasn't clear - but not in the way you thought! I meant that 2^53 is represented *in binary* (as it will be in the computer) as one followed by fifty-three zeroes, of course. So it's the value that is just too large to be stored accurately in double precision. (As so often happens, I spotted that omission just after I had sent the message!)

Brian Barker

#6

Of course! I don't think binary!

Keith

#7

Hi :slight_smile:
There are only 10 types of people in the world

Those who think in binary,
and those who don't
(err and a bit of a grey area where logic gates might or might not have
been triggered so i guess that makes 11 types of people)

Sorry! I know it's not a good joke so it's great to have the opportunity
to retell it!
Thanks and regards from
Tom :slight_smile: