Metrohm 915 KF Ti-Touch - Manual (Page 368)

31.8 Arithmetic algorithms in the Ti-Touch
￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭
354 ￭￭￭￭￭￭￭￭
915 KF Ti-Touch
Explanations
The individual values are incorporated in the statistics in the selected Preci-
sion (Round, Truncate or Full precision).
7 significant places are yielded when the 32 bit numerical format is
applied for the floating point number in decimal presentation. For the 64
bit numerical format, there are 15 significant places.
The accuracy can be controlled by the selection of the prefix of the unit
(milli, micro) and the number of decimal places.
Example (64-bit numerical format):
The result displayed, 1234.56789158763 mg/L, has 15 significant pla-
ces. It should be rounded off to three decimal places according to the
above rounding-off process:
￭ 1234.568 mg/L.
If the same result is expressed in "g/L" (1.23456789158763 g/L), and is
also rounded off to three decimal place, this yields
￭ 1.235 g/L.
I.e. you obtain the lowest losses in accuracy with rounding when you
select the application and the numerical format in such a way that the
numbers displayed have as many places before the decimal point as possi-
ble.
A complete recalculation of the statistics using a pocket calculator or PC
calculation programs may exhibit deviations. This can be explained by the
different binary numerical formats used by these computers.
NOTE
The above losses of accuracy by rounding off in the range of significant
places are only relevant theoretically. Most of the time they are lower
by several orders of magnitude than – as an example – the uncertainties
resulting from weighing out the sample.