Metrohm 915 KF Ti-Touch - Manual (Page 367)

￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭￭
31 Appendix
915 KF Ti-Touch
￭￭￭￭￭￭￭￭ 353
31.8 Arithmetic algorithms in the Ti-Touch
Some of the algorithms and numerical formats used in the device soft-
ware are explained in this chapter.
Numerical format
All floating point calculations use the IEEE 754 (1985) numerical format,
either in "single precision" (32 Bit) or "double precision" (64 Bit).
Rounding off process
Measured values and results are rounded to the defined number of deci-
mal places (commercial rounding, in accordance with the US Pharmaco-
peia USP). If the digit at the first dropped decimal place is 1, 2, 3 or 4,
then it will be rounded off; if this digit is 5, 6, 7, 8 or 9, then it will be
rounded up. Negative digits will be rounded in accordance with their
amount, i.e. away from zero.
Examples:
2.33 yields 2.3
2.35 yields 2.4
2.47 yields 2.5
–2.38 yields –2.4
–2.45 yields –2.5
Statistics
The arithmetic mean value and the absolute and relative standard devia-
tions of results R, endpoints EP and variable C are calculated:
You can statistically evaluate a maximum of nine results (1 ≤ k ≤ 9) calcu-
lated in a determination. A statistical series can contain a maximum of 20
determinations (1
≤ n ≤ 20).
The following convention applies to the subsequent formulas:
1 ≤ n ≤ 20 and 1 ≤ k ≤ 9.
Mean value:
Absolute standard deviation:
Relative standard deviation (in %):