Type conversion and accuracy – BrightSign BrightScript 3.0 Reference Manual User Manual

Page 14

Advertising
background image

9

Division (/) operates using the same rules as above, except that it can never be carried out at the Integer level: When both
operators are Integers, the operation and return will be carried out with Floats.

Comparison operations (e.g. <, >, =) will convert the numbers to the same type before they are compared. The less
precise type will always be converted to the more precise type.

Type Conversion and Accuracy

When a Float or Double number is converted to the Integer type, it is rounded down: The largest integer that is not greater
than the number is used. This also happens when the

INT function is called on a number.


When a Double number is converted to the Float type, it is 4/5 rounded: The least significant digit is rounded up if the
fractional part is >=5 (otherwise, it is left unchanged).

When a Float number is converted to the Double type, only the seven most significant digits will be accurate.

Advertising