## Monday, November 23, 2009

### Amateur Hour at the CRU

Here’s an interesting excerpt from the CRU documents, from a file called “HARRY_READ_ME.txt”. The author appears to be someone trying to analyze/debug their code.

17. Inserted debug statements into anomdtb.f90, discovered that
a sum-of-squared variable is becoming very, very negative! Key
output from the debug statements:

OpEn=   16.00, OpTotSq=    4142182.00, OpTot= 7126.00
DataA val =       93, OpTotSq=       8649.00
DataA val =      172, OpTotSq=      38233.00
DataA val =      950, OpTotSq=     940733.00
DataA val =      797, OpTotSq=    1575942.00
DataA val =      293, OpTotSq=    1661791.00
DataA val =       83, OpTotSq=    1668680.00
DataA val =      860, OpTotSq=    2408280.00
DataA val =      222, OpTotSq=    2457564.00
DataA val =      452, OpTotSq=    2661868.00
DataA val =      561, OpTotSq=    2976589.00
DataA val =    49920, OpTotSq=-1799984256.00
DataA val =      547, OpTotSq=-1799684992.00
DataA val =      672, OpTotSq=-1799233408.00
DataA val =      710, OpTotSq=-1798729344.00
DataA val =      211, OpTotSq=-1798684800.00
DataA val =      403, OpTotSq=-1798522368.00
OpEn=   16.00, OpTotSq=-1798522368.00, OpTot=56946.00
forrtl: error (75): floating point exception
IOT trap (core dumped)

..so the data value is unbfeasibly large, but why does the
sum-of-squares parameter OpTotSq go negative?!!

The author here is apparently a high school student, or a McDonald’s reject. The problem is apparently a sum of squares, i.e. x*x + y*y + z*z + a*a, etc. And the person here is stunned that the values go negative. The answer is that the algorithm is obviously using 32-bit signed integers, and he’s experiencing overflow. The result at the point of overflow is 2,494,982,989. The maximum 32-bit signed integer is 2,147,483,647.

Here’s another blogger’s take, with an example of how this could happen, using 4-bit arithmetic for simplicity.

I’m a software developer. I’d hope that any person we’d hire out of college would know this, but the person here is clueless.

And we’re supposed to trust these people’s code and their math. We’re supposed to trust the entire future of the world to it.