Metrohm tiamo 2.2 (ProcessLab) - Manual (Page 43)

￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭ 2 General program functions
tiamo 2.2 (ProcessLab) ￭￭￭￭￭￭￭￭ 27
-2.38 yields -2.4
-2.45 yields -2.5
Statistics
The mean value, absolute and relative standard deviation of results R are
calculated using the following formula:
Mean value
Absolute standard
deviation
Relative standard devi-
ation (in %)
The statistical calculations of the software have been implemented so that
they can be checked to as great a degree as possible by the user. The indi-
vidual values are therefore incorporated in the statistics with full accuracy.
It is not the number of decimal places which is decisive for the accuracy of
the calculations, but rather the number of significant digits of the decimal
numbers displayed. The binary 64 bit numerical format implemented on the
basis of the IEEE 754 standard means that the decimal numerals resulting
from this have 15 certain significant decimal places.
You can influence the number of significant digits by selecting the unit and
the number of decimal places. As the result unit to be set sometimes contains
the prefix "Milli" as well as the actual physical unit, the number of significant
places changes accordingly by three places during such a conversion.
Example
The result displayed 1234.56789158763 mg/L has 15 certain places. It
should be rounded off to three decimal places according to the above
rounding procedure:
1234.568 mg/L (7 significant places, 3 of them decimal places)
The unit "g/L" means that the same result 1.23456789158763 g/L is also
rounded off to three decimal places:
1.235 g/L (4 significant places, 3 of them decimal places)