Limitations on precision of data – HP 33s User Manual
Page 153
Advertising
Statistical
Operations
11–9
x
0 20 40 60 80
8.50
7.50
6.50
5.50
4.50
r = 0.9880
m = 0.0387
b = 4.8560
(70, y)
y
X
What if 70 kg of nitrogen fertilizer were applied to the rice field? Predict the grain
yield based on the above statistics.
Keys:
Display:
Description:
70
)
_
Enters hypothetical x–value.
|
{
¸
ˆ
}
º
ˆ
¸
ˆ
T P E
)
The predicted yield in tons per
hectare.
Limitations on Precision of Data
Since the calculator uses finite precision (12 to 15 digits), it follows that there are
limitations to calculations due to rounding. Here are two examples:
Advertising