OCONV behavior - Bug? - feature? (OpenInsight 32-Bit)
At 21 NOV 2004 08:15:04PM Paxton Scott wrote:
Greetings:
While converting a REVG app to OI 7.01, I had some stray numbers in a report. Upon investigation, I came across a variation in the OCONV handling. Here is my test code:
COMPILE SUBROUTINE testnum(dummy)
num1=(1/4991)*100
num2=.02360649
NUM3=.123
NUM4=1.23
NUM5=12.34
stuff='
stuff=oconv(num1,'MD0, =oconv(num2,'MD0, =oconv(num3,'MD0, =oconv(num4,'MD0, =oconv(num5,'MD0, ARCS, Inc.
At 22 NOV 2004 03:12PM Steve Smith wrote:
Well you need to code around the maths precision / exponentiation.
Multiply all your numbers by 10,000,000,000 and recheck the comparisons.
Problem solved.
The product is best crunching numbers between 0 and 10^44 or thereabouts. It's been like this for a long long time.
Steve
At 23 NOV 2004 11:17AM Paxton Scott wrote:
Thanks, Steve.
Wanted to be sure there wasn't some 'trick' to avoid going through this old code and finding the places where this could happen and coding around it….Oh well, client pays….
Thanks for your response, I enjoy reading your stuff.
Paxton