Rockwell Automation 2706-MB1 MESSAGEVIEW 421 SOFTWARE MANUAL - Manual (Page 89)

4–41 Planning an Application
Publication 2706-817
The logic controller stores values only as integers. Digits to the right
of the decimal are rounded off. However the terminal can scale
decimal values to or from integer values.
In this illustration, the entered value of 7.5 is rounded up to 8. When
this value is displayed or used in a mathematical formula, the result
may be incorrect.
Entered Value = 7.5 Displayed Value = 8
Controller Value = 8
(rounded)
If decimal precision is needed, select appropriate values for scale and
offset.
Example:
Use scaling to convert a decimal value from degrees Fahrenheit to
degrees Celsius.
Scale
Rounded
4964.44
4964
Offset = 32
Scale = .018
4964
121.36
Entered Value
121.35
Displayed Value
Controller
Scale
50.5
51
Offset = 32
Scale = 1.8
51
122.9
Entered Value
123.8
Displayed Value
Controller
Fixed Decimal
Decimal Position = 2
Field Width = 6
Keypad Controlled Decimal
Decimal Position = Operator Entered
Field Width = 6
123.8
Rounded
The decimal point is positioned to provide a display appropriate to the scaled value.
Controller Value =
Entered Value – Offset
Scale
Displayed Value = Controller Value x Scale +
Offset
Equations for Scaling Example
121.36 – 32
0.018
= 4964.44 = 4964
(4964 x 0.018) + 32 = 121.35
Stored Value =
Displayed Value =
Fixed Decimal
Stored Value =
Displayed Value =
122.9 – 32
1.8
= 50.5 = 51
(51 x 1.8) + 32 = 123.8
Stored Value =
Displayed Value =
Keypad Controlled Decimal
The terminal stores the full precision value and the controller stores
the rounded value.
Note: The value stored in the terminal may be different than the
value sent to the controller because of rounding that occurs during
pre-scaling.
Precision and Rounding