Metrohm tiamo 1.1 Manual - Manual (Page 49)

Formula editor 21
2.47 gets 2.5
2.53 gets 2.5
2.55 gets 2.6
-2.35 gets -2.4
-2.45 gets -2.4
Statisti cs
The following formulae are used for calculation of the arithmetic mean value
as well as the absolute and relative standard deviation for results R:
Mean value
Absolute standard addition
Relative standard addition (in %)
The statistical calculations in the software have been implemented in such
a way that they are as revisable as possible for the user. This is why the in-
dividual values used in the statistics are in full precision form.
It is not the number of decimal places which is decisive for the accuracy of
the calculations, but rather the number of significant digits of the decimal
numbers displayed. As a result of the binary 64-bit number format imple-
mented on the basis of the IEEE 754 standard, the decimal numbers which
are produced have 15 reliable significant decimal digits.
You can influence the number of significant digits by selecting the unit and
the number of decimal places. As the results unit to be set sometimes con-
tains both the prefix "milli" and the physical unit itself, the number of signifi-
cant digits is altered accordingly in such a case by three digits.
Examples
The displayed result
1234.56789158763 mg/L has 15 reliable digits. It is to
be rounded to three decimal places according to the rounding method
given above:
1234.568 mg/L (7 significant digits, 3 of which are decimal places)
With the unit " g/L " the same result 1.23456789158763 g/L is also rounded to
three decimal places:
1.235 g/L (4 significant digits, 3 of which are decimal places)
The number of significant digits has now been reduced by three to four dig-
its by omitting the prefix "milli".
Note
The losses of accuracy by rounding described above in the range of the maxi-
mum reliable digits are only of theoretical relevance. Most of the time they are
lower by several orders of magnitude than – as an example – the uncertainties
resulting from weighing out the sample.