Metrohm tiamo 2.4 Manual - Manual (Page 48)

2.4 Formula editor
￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭
32 ￭￭￭￭￭￭￭￭
tiamo 2.4
Statistics
The mean value as well as the absolute and relative standard deviation of
results R are calculated using the following formulas:
Mean value
Absolute standard
deviation
Relative standard devi-
ation (in %)
The calculation of the mean value and the standard deviation requires a
multitude of calculation operations, which are carried out with full accu-
racy. The input data (results) and the output data (mean value, standard
deviation), however, are then rounded off to the accuracy specified by the
user.
It is not the number of decimal places which is decisive for the accuracy of
the calculations, but rather the number of significant digits of the decimal
numbers displayed. As a result of the binary 64-bit numerical format
implemented on the basis of the IEEE 754 standard, the resulting decimal
numbers have 15 reliable significant decimal places.
You can influence the number of significant digits by selecting the unit
and the number of decimal places. As the result unit to be set sometimes
contains the prefix "milli" as well as the actual physical unit, the number of
significant places changes accordingly by three places during such a con-
version.
Example
The displayed result of 1,234.56789158763 mg/L has 15 reliable digits.
It should be rounded off to three decimal places according to the above
rounding-off process:
1,234.568 mg/L (7 significant places, 3 of them decimal places)
The unit "g/L" means that the same result 1.23456789158763 g/L is
also rounded off to three decimal places:
1.235 g/L (4 significant places, 3 of them decimal places)