.NET apparently handles calculations differently depending on whether or not they are implicit or explicit. For example if you were to run this statement:
If( 3 < (0.6F / 0.2F ))
you would expect .net to do the float calculation, then either cast the integer to a float, or round the float to an integer and perform the calculation. Apparently it does not as the statement above always evaluates to true, when in reality it is false since the integer 3 is the same value as the float 3.0. My guess is when performing the division there is some sort of math error in the .net engine which results in something like 3.0000000000000000000001 for 0.6F / 0.2F
The solution to this issue is to explicitly force trimming like this:
Int f = (int)(0.6F / 0.2F)
If( 3 < f )
The above statement casts the float calculation to an integer effectively truncating the extra floating data and allowing a correct comparison. A better solution would probably be to use Math.Round just in case .net decides to evaluate the calculation to just under 3 rather than just over it.
No comments:
Post a Comment