Calc error?

users
#1

I'm running LO on OpenSuse and I wonder if anyone alse can confirm this
error on calc.

Multiplying 2048.28*.146 yields correctly 295.54488, but when rounding
down to 2 decimal places, 295.55 is displayed (instead of
295.54)(either using ROUND(2048.28*.146,2) or with field format.

My LO Version:
Version: 6.0.5.2
Build ID: 00m0(Build:2)
CPU threads: 2; OS: Linux 4.12; UI render: default; VCL: gtk3;
Locale: en-GB (en_GB.UTF-8); Calc: group

Cheers
Harvey

#2

Hi Harvey,

Multiplying 2048.28*.146 yields correctly 295.54488, but when rounding
down to 2 decimal places, 295.55 is displayed (instead of
295.54)(either using ROUND(2048.28*.146,2) or with field format.

Please have a look at the values you have written. Gives 299.04888 here
on Calc and pocket-calculator ...

Regards

Robert

#3

I'm running LO on OpenSuse and I wonder if anyone else can confirm this error on calc. Multiplying 2048.28*.146 yields correctly 295.54488, ...

Er, that wouldn't be correct with my maths. Do you perhaps mean 2024.28 * 0.146 instead?

... but when rounding down to 2 decimal places, 295.55 is displayed (instead of 295.54) (either using ROUND(2048.28*.146,2) or with field format.

That would indeed be odd. It's what you would get if you repeatedly rounded the value to fewer digits one by one
295.54488 -> 295.5449 -> 295.545 -> 295.55
- instead of doing it in one go. So that's not the way to do it, of course.

There is one way that you can create this result, using options at Tools | Options... | LibreOffice Calc | Calculate. If you have "Limit decimals for general number format" ticked and set to "3 Decimal places", your original result will be displayed - correctly - as 295.545. If you then round *that* value to two fractional places, you will properly get 295.55. The situation can be complicated if you also have "Precision as shown" ticked, which causes subsequent calculations to be based on what you see in a cell instead of the underlying, unrounded value.

What happens if you ensure that both of these options are *not* ticked?

I trust this helps.

Brian Barker

#4

Oh, sorry. My haste. Should have been 2024.28*.146

Cheers
Harvey

#5

Your numbers don't seem to match up on a bigger scale.
Unfortunately, ths list reformats text, so headings will not directly match with
the columns.

Using the values in message gave in message the result is 299.04888 as
shown, and the 3 digit of 8 causes it to round the 4 up to a 5, which is
correct.
Num 1Num 2Num 1 * Num 2Rounded
2048.280.146299.04888299.05

Here I recalculated the 2nd number to get the result you
show instead of the 0.146 value. Again Rounding is
correct.

Num 1Num 2 CalcYour Value??Rounded
2048.280.144291210186107295.5488295.55

Same as above, but recalculated the 1st number??

Num 1 CalcNum 2Your Value??Rounded
2024.306849315070.146295.5488295.55

#6

Dear Michael, I have to apologise for providing the wrong example. It
should have been 2024.28*.146(I was distracted while writing, and
forgot to check!!)
CheersHarvey

#7

You are right the example should have been 2024.28 * 0.146. Sorry about
that and for the trouble caused!

However, I do not have 'Limit decimals for general number format' set
to anything. (i.e. No tick)
Cheers
Harvey

#8

=With the updated first number I get the 295.54 as it should be.</span>
Third position is a 4, it doesn't change the 2nd position.

Num 1
Num 2
Num 1 * Num 2
Rounded
2024.28
0.146
295.54488
295.54

Are the values you show for the Num 1 and Num 2 entered link that or are
that what is displayed on screen.The actual numbers could throw off the
calcs.

If the 0.146 number is display like that, but actually contains
0.146001936490999 it would change the value.

#9

Good-oh! And what about "Precision as shown"?

Brian Barker

#10

Harvey Nimmo <harvey@nimmo.de> writes:

I'm running LO on OpenSuse and I wonder if anyone alse can confirm this
error on calc.

Multiplying 2048.28*.146 yields correctly 295.54488, but when rounding
down to 2 decimal places, 295.55 is displayed (instead of
295.54)(either using ROUND(2048.28*.146,2) or with field format.

2048.28*.146 = 299.04888
not 295.54488

#11

I think you have put your finger on it, Michael.

The 2024.28 figure is calculated but displayed to only 2 decimal
places. It is, to 4 decimal places, 2024.2829 which results in
295.5452888. So, thanks for the enlightenment!!

Cheers
Harvey

#12

'Precision as shown' is also not set (No tick)

Harvey

#13

Sorry Piet, for causing you trouble. My mistake. See my
previous replies.

Cheers
Harvey

#14

Hi Harvey,
>
> Multiplying 2048.28*.146 yields correctly 295.54488, but when rounding
> down to 2 decimal places, 295.55 is displayed (instead of
> 295.54)(either using ROUND(2048.28*.146,2) or with field format.

Sorry, I can't reproduce this at all (using the correct values given in

another mail):
=2024.28*0.146
Cell format set to 2 decimals ⇨ 295,54
=ROUND(2024.28*0.146;2 ⇨ The same result.

Manjaro Linux, LibreOffice 6.1.3.2

Kind regards

Johnny Rosenberg