Limitations on precision of data – HP 32SII User Manual

Page 158

Advertising
background image

11–10 Statistical Operations

File name 32sii-Manual-E-0424
Printed Date : 2003/4/24 Size : 17.7 x 25.2 cm

x

0 20 40 60 80

8.50

7.50

6.50

5.50

4.50

r = 0.9880

m = 0.0387

b = 4.8560

(70, y)

y

X

What if 70 kg of nitrogen fertilizer were applied to the rice field

?

Predict the

grain yield based on the above statistics.

Keys: Display: Description:

70

_



Enters hypothetical x–value.

{

,

{

¸

ˆ

}

) 

The predicted yield in tons per
hectare.

Limitations on Precision of Data

Since the calculator uses finite precision (12 to 15 digits), it follows that there
are limitations to calculations due to rounding. Here are two examples:

Advertising