Why Computers Fail Second Grade Math
Transcript
Made automatically from the audio, so it may contain mistakes. Click a line to play from there.
We just finished our battles with multi-server split brain scenarios, but today we're diving into a problem that is much closer to home. Why a single CPU core can fail second grade math.
Yeah, it really is the ultimate betrayal of trust when you first discover it. Because you eye this incredibly powerful mathematically perfect machine, right?
Right, exactly.
But then you realize the very foundation of its logic, like the way it handles the most basic numbers in the universe is sitting on this precarious structural rounding error.
It is wild. And I want you, the listener, to try something if you're sitting at a computer right now. Imagine opening up just a completely blank basic code terminal.
No external libraries, no weird frameworks. Nothing.
Just the raw computational power of your machine, tapping directly into the language interpreter, and you type in a simple addition problem. 0.1 plus 0.2. Hit enter. You would expect the answer to be 0.3.
Because it's second grade math.
Right. But it's not. The terminal spits back 0.3, followed by 15 zeros. And then, for some reason, a random number 4 sitting way out at the very end. 0.300000004.
I mean, it is a total right of passage for every new programmer. Oh, absolutely. You stare at that trailing four in the screen and you genuinely think, did I just break the computer? Like, did I type it wrong?
Yeah. Is my keyboard broken? And it raises this fundamental, almost philosophical question for anyone building software. Why did the world's smartest chip designers build this bizarre, slightly broken behavior into our processors on purpose?
And they really did do it on purpose.
Which is crazy. So this deep dive is designed to really deconstruct this deceptive primitive of floating point math. We're gonna explore how hardware engineers manage to pack astronomical scales into these microscopic digital spaces.
And why this specific design choice has literally crashed rockets.
Yes. Crash rockets, erase millions of dollars from stock markets. And most importantly, how you can prevent these exact disasters in your own code.
Because if you just blindly trust the math operators on your keyboard, you're you know you're walking into a trap that has been waiting there in the silicon since the 1980s.
Okay, so to understand why basic edition fails, we first have to understand the physical constraints of how computers store numbers and the mathematical reality of base two fractions.
Yeah, let's start there. Because we are so used to counting in base ten, right? We have ten fingers, it feels completely natural to us.
But computers count in base two.
They only have two fingers, zero and one. And I know you listening already know basic binary if you've been following along this season. But how does that actually cause zero point one to break? So to understand why zero point one is such a massive nightmare for computer, it really helps to look at a similar nightmare we have in our own base ten system.
Okay, what do you mean?
Well, in everyday math, you cannot cleanly write out the fraction one-third is a decimal. You write zero point three three three three, and you just have to keep writing threes forever.
Right, until you run out of paper.
Exactly. The repeating decibel. It never actually resolves. You just draw a little line over the three and call it a day, pretending it's finished.
Yeah, we just kind of accept it and move on.
Right. And the mathematical reason it never resolves is because of prime factors. The base of our counting system is ten. The prime factors of ten are two and five.
Okay. Tracing that.
So any fraction whose denominator only uses twos and fives can be written out perfectly. One half is zero point five. One quarter is zero point two five. One fifth is zero point two.
And one tenth is zero point one.
All perfectly clean. Completely clean. Because the denominators are built from those prime factors of ten. But the denominator for one third is three. Three is not a prime factor of ten.
Okay.
Therefore, it mathematically cannot fit neatly into a base ten grid. It literally forces an infinitely repeating sequence.
Oh, I see where this is going. If we take that exact same logic and apply it to a computer's memory, which operates purely in base two, the only prime factor available is two.
Exactly. In base two, the only fractions that resolve cleanly are the ones where the denominator is a pure power of two.
Like one half, one quarter, one eighth.
One sixteenth, yeah. Those are beautifully exact in a computer. You can add one half and one quarter together all day long, and the computer will never ever make a mistake.
But what happens when you try to ask a computer to store one tenth like our zero point one?
Well, the denominator is ten. That is not a power of two.
Right.
So in binary, zero point one behaves exactly the same way that one third behaves in base ten. It becomes an infinitely repeating binary decimal.
It just goes on forever.
Forever. If you were to write out the pure mathematical binary fraction for one tenth, it looks like zero point zero zero zero one one zero zero one one, zero zero one one, and that zero zero one one pattern just repeats into infinity.
But a computer obviously doesn't have infinite memory. It has a very strict, finite physical budget. It has to cut that infinite sequence off somewhere. Like it has to take a machete to the math.
That's a great way to put it. And that physical budget is where the magic and honestly the danger happens. Almost all modern hardware uses a standard called IEEE 754 double precision floating point format.
Which sounds incredibly dense, but let's break it down.
Yeah. I want you to imagine this format physically. Like close your eyes and picture a row of 64 tiny physical light switches.
Okay, 64 switches.
That is your entire budget. With just those 64 switches flipping on or off, you have to be able to represent any number in the universe.
Which is insane. From the microscopic width of a single proton all the way to the like the distance between galaxies. 64 switches is nothing. How on earth do they divide that up to capture that scale?
It's essentially scientific notation, just baked directly into the silicon. You take the first switch on the far left. That is your science switch. Up for positive, down for negative. Simple enough.
Okay, one switch down, 63 to go.
Okay. Next you have a block of 11 switches. These control what we call the exponent.
Okay, the exponent.
Think of this block like the massive zoom dial on a microscope or a telescope. It physically shifts the decimal point left or right across the number.
So zooming in and out.
Exactly. If you want to look at the mass of a planet, you crank the exponent dial way up, moving the decimal point far to the right. If you want to measure the diameter of an atom, you dial it way down, shifting the decimal far to the left.
So the exponent just places you in the right ballpark of the universe. It doesn't give you the detail.
Right. The exponent is the neighborhood. The actual details of the number, the significant digits, that is the remaining 52 switches.
Okay, the last 52.
This block is called the significant, or sometimes you'll hear it called the mantissa. If the exponent is the zoom dial on your telescope, the significant is the actual precision of the image you see. It's the incredibly fine markings on your ruler.
The markings on the ruler. Got it.
And by a very clever hardware trick, they actually squeeze a little bit more out of it. Because binary scientific notation always starts with a one, just like standard scientific notation never starts with a zero before the decimal, the hardware designers decided not to even bother storing that leading one.
Wait, they just leave it out entirely.
Yeah, it's implicit. So those 52 physical switches actually give you 53 bits of precision.
That's a slick trick. But I want to stop and look at an everyday illusion here, because this is the part that trips up so many developers.
Oh, the printing illusion.
Yes. If the computer is physically storing 0.1 as this messy, infinitely repeating fraction that gets brutally chopped off when it runs out of those 52 significant switches. Why does it look so clean to us?
Right, like when you test it.
Exactly. If I type print 0.1 in Python or JavaScript, it prints out 0.1 on the screen. It doesn't print out a massive chopped-off string of garbage.
It is absolutely an illusion. And it's an illusion of display.
So what is actually sitting in the switches when I type 0.1?
Mathematically, the closest value those 53 bits of precision can get to true 0.1 is roughly 0.1, followed by 16 zeros, and then 5, 5511, 15, and it goes on.
That is awful.
It is. That is the true machine value. It is slightly larger than actual 0.1. But language designers know that if every time you typed a basic decimal, the console spit back that massive string of garbage digits, developers would lose their minds.
We would just stop programming.
The ecosystem would be unusable.
Completely unusable.
So modern languages inherently round the display output to make it human readable.
So it's covering its tracks.
Yeah, the language interpreter looks at that messy machine value in the switches and essentially says, Well, what is the absolute shortest, most mathematically sound decimal string that would map back to this exact binary pattern if I typed it in?
And the answer to that question is just 0.1.
Exactly.
And so it prints 0.1 to the terminal.
It's lying to us to protect our feelings.
It really creates this deceptive sense of exactness. We look at the screen and think, perfectly precise 0.1. But underneath it's a jagged, truncated approximation.
A very jagged approximation.
And that brings us back to our cold open hook. When you add 0.1 and 0.2.
You aren't adding the pure mathematical concepts of a tenth and a fifth. The machine cannot see those pure concepts. You are adding two slightly inaccurate, truncated binary approximations together. Right. And when you combine them, those microscopic errors at the very end of the significant, they bump into each other. They compound.
They stack up.
And the resulting error becomes just large enough that it crosses the threshold of the language's display rounding illusion. The language interpreter can no longer confidently hide it.
So the tiny little floating point error leaks out onto your screen as that trailing four.
Exactly. The curtain falls, and you see the ugly math.
Which is mind-blowing. But honestly, it makes me wonder if this architecture is so inherently leaky, why did we design the standard this way? It feels like we built the entire foundation of computing on a system that fails basic arithmetic.
It does feel that way.
So why did William Cahan and the IEEE committee in the 1980s architect this specific bit budget?
Because they weren't designing a system for perfect financial accounting. They were designing a system to manage the extreme chaos of scientific and engineering computations.
Okay, managing chaos.
Right. They were building a primitive that could handle wild extremes from calculating the trajectory of a galaxy to modeling the decay of a subatomic particle without instantly crashing the software.
Let's explore that. What does this chaos management actually look like in the design? For example, I know Cahan spent a lot of time figuring out what happens when numbers get incredibly close to zero. Like they aren't evenly spaced on a ruler.
Not at all. The numbers get closer and closer together the closer you get to zero, and further apart as you get larger. But there is a hard limit.
Okay, a limit to how small they can get?
Yeah. There is a smallest possible normal number you can represent when your exponent dial is turned all the way down to its lowest setting. Historically, before this IEEE standard, if you did a calculation that resulted in a number smaller than that limit, the hardware would just throw its hands up and flush the number directly to absolute zero.
It would just snap to zero.
Like falling off a cliff.
Literally abrupt underflow.
And that caused massive mathematical nightmares. Because a tiny microscopic number is not zero.
It's just very small.
Exactly. So Cahan designed a concept called subnormals. It's a way of letting the system use those 52 significant switches to represent numbers even smaller than the normal limit by essentially giving up some precision.
Oh, a trade-off.
Yes. It allows for gradual underflow. Instead of falling off a cliff to zero, the numbers gently step down, losing a little precision with each step until they finally hit true zero.
I see why that matters for logic. Like if you are writing a standard if statement, say if x equals y, and the computer evaluates that as true, you logically guarantee that x minus y equals zero will also be true.
Right. That's fundamental logic.
But without subnormals, if you have two incredibly tiny distinct numbers that are both smaller than the normal limit, a subtraction between them might suddenly snap to zero simply because the result is too small to fit the exponent budget.
Exactly. The system would tell you x minus y is zero, even though x and y were completely different numbers. It would break the algebraic fabric of programming.
Which is terrifying. So subnormals prevent that from happening. It's preserving the logic over the pure precision.
Exactly.
I get that. But then there's a feature that feels completely illogical at first glance. Signed zeros. Cahan explicitly designed positive zero and negative zero. Why on earth do we need two different zeros? Zero is the absence of value.
If you are counting apples, sure. Zero apples is zero apples.
Right.
But if you were doing complex calculus or analyzing branch cuts in complex planes, zero isn't just an absence, it's a boundary. A boundary. Okay. Okay. In limits, the direction you approach zero from is critical. If you divide a positive number by positive zero, you're modeling a limit approaching positive infinity. But if you divide by negative zero, it approaches negative infinity.
So the sign on the zero acts as like a breadcrumb.
A breadcrumb, exactly.
It's keeping a ghost of the history of the calculation, telling the software which side of the axis it just came from before the number collapsed to zero.
Precisely. And speaking of weird states, we have to talk about the ultimate failsafe Cahan built. NAN, not a number.
Oh, this one I encounter all the time. You divide zero by zero or infinity by infinity, and instead of throwing a fatal error and crashing the program immediately, it just returns nan.
Yep, and A N.
But why is that better than a crash? Shouldn't a program fail loudly if it does impossible math.
Imagine you wrote a complex algorithm like a zero finder. Its job is to blindly search across a massive, bumpy, multidimensional mathematical graph, trying to find exactly where a curve crosses the x-axis.
Okay, so it's probing different coordinates, guessing and checking millions of points, like a little robot wandering a rocky landscape. Right.
And occasionally that robot might accidentally probe a forbidden area. It might try to calculate a coordinate that results in zero divided by zero.
Just a bad guess on the map.
Yeah. In older pre IEEE systems, the moment that happened, the hardware would trigger a fatal interrupt and self destruct the entire application. Your robot basically explodes because it stepped on a mathematical landmine.
Which is terrible because maybe it was just a bad guess. The actual answer the robot was looking for was safely over to the left, but the whole program died before it could get there.
Exactly. You want the robot to say, oops, the math of this specific coordinate is undefined. Tag it and keep searching elsewhere. So Cahan designed NANS. If a calculation results in an undefined state, it doesn't crash. It silently generates a NAN tag. And NANS are designed to safely propagate.
Meaning they don't break the next step?
Right. Any math you do to a NAN just equals another NAN. NAN plus five is NAN. It allows the software to finish its massive loop of calculations, and at the very end, you can look at the data set and say, okay, these three points were NAND, we can ignore those, but here's the valid answer we were looking for.
It prevents catastrophic halts.
That's actually really elegant.
But there is one more design choice in IEEE 754 that just feels like absolute witchcraft to me. The lexicographic ordering.
Oh man. It is arguably one of the most elegant hardware hacks in computer science history.
Okay, let's unpack this slowly because it took me a minute. We established that a floating point number is composed of a sign bit, eleven exponent bits, and 52 significant bits. Right. If you take that entire 64-bit chunk of memory, and you completely ignore the fact that it's a floating point number. Like if you just interpret those raw bits as a basic sign magnitude integer, their numerical sorting order stays exactly the same. How's that even possible?
Cahan and the committee explicitly arranged the layout of the bits, putting the sign first, then the exponent, then the significant, so that it perfectly mirrored how integers scale.
But why go through all that trouble? Why arrange them that way?
Think about the 1980s hardware they were designing for. Floating point math circuits were incredibly slow and super expensive. Integer math circuits, on the other hand, were fast and cheap.
Okay.
So if a database or a scientific program needed to sort an array of a million floating point numbers from smallest to largest, doing that using floating point comparison operations would take forever.
Oh, I see it.
Yeah. Because of this lexicographic layout, the hardware can take those millions of floats, feed them directly into the blazing fast integer sorting circuits, and sort them perfectly without having to decode a single float.
The bit structure itself was perfectly aligned to make hardware sorting practically free. That is a genius. They were bouncing so many incredible constraints. They really gave us a tool of unbelievable power and efficiency.
They did. But and this is the massive pivot for this deep dive, it is still a tool of approximation.
Right, the machete.
And when you let those microscopic errors accumulate over time in the real world, things go terribly, terribly wrong.
Which brings us to the historic post-mortems. We've established the theory of the bit budget. Now let's look at the blood on the floor when truncation turns deadly. Let's start with a classic financial disaster, the Vancouver Stock Exchange in 1982.
Yeah, this is a perfect example of how truncation bias destroys data over volume.
Okay, so in January 1982, the Vancouver Stock Exchange created a new index. They initialized the index value at a clean, even 1,000 points. And throughout the trading day, as stocks were bought and sold, the system updated the index. It updated about 3,000 times a day.
3,000 tiny additions and subtractions.
Right. But whoever programmed the system made a seemingly microscopic mistake. When they calculated the new index value, the floating point math naturally generated a number with a long string of decimal places.
As it does.
But instead of properly rounding the number to three decimal places using a balanced rounding algorithm, the system simply truncated it. It used a floor function. It took a machete and just chopped off all the digits after the third decimal place.
So if the true calculated value was 1,000 point five two four eight, they didn't round it up to 1,000 point five two five. They just chopped the eight, leaving 1,000.524.
Exactly. Now to a lay person, that seems utterly harmless. You are losing fractions of a fraction of a penny on an index value that's in the thousands. But what happens when you repeat that specific mistake three thousand times a day?
The sheer volume of the truncation creates a directional bias. Because they were exclusively chopping off the trailing digits, the error was entirely one-sided. Every single mistake made the number smaller. It never ever made it larger.
It's just shaving a tiny layer off every single time.
Exactly. This microscopic loss repeated thousands of times daily caused the index to silently bleed out.
I read that the traders on the floor were completely baffled. The overall market felt robust, stocks were going up, but the index was just sinking into a depression. It bled about 25 points a month.
25 points a month.
Yeah. When the exchange finally discovered the floor rounding bug and corrected it over a weekend in November 1983, the index jumped from a depressed 524 points all the way back up to its true value of 1,098.
Nearly half of the index's value had vanished purely into the void of a floating point truncation loop.
That is staggering.
It's a stark reminder that in finance, rounding errors do not average out if you only ever truncate. They form a directional bias that acts like a parasite.
But as bad as losing fake points on a stock indexes, the consequences of floating point drift can be far more severe. The Vancouver incident was accumulation over volume. The 1991 Daron Patriot Missile Incident was accumulation over time.
And it was deadly.
Yes. Let's unpack this one carefully because it's incredibly tragic. This is the story of Alpha Battery during Operation Desert Storm in 1991. Alpha Battery was tasked with protecting a military barracks in Daron, Saudi Arabia from incoming Iraqi scud missiles.
Right. And the Patriot Missile System was originally designed in the 1970s. It was a marvel of its time. But its weapons control computer used a 24-bit register for its calculations, not the 64-bit double precision we enjoy today.
24 bits. That is a very tight physical budget for a significant.
Extremely tight.
So the system relied on its radar to detect airborne objects. But a scud missile is flying at Mach 5. It's moving so fast that you can't just look at where it is. You have to predict exactly where it is going to be in the next fraction of a second, so the interceptor missile can actually hit it.
Right. They used a range gate.
Yeah, a range gate. Basically a digital box in the sky where it expected the target to appear next.
And to calculate the position of that range gate, the algorithm had to multiply the scud's known velocity by the amount of time that had passed since the last radar detection. Velocity times time equals distance. It's standard physics.
But here is where the deceptively broken primitive rears its head. The system measured time using an internal hardware clock that ticked in tenths of a second. Okay. The clock counted these ticks as a pure exact integer. One tick, two ticks, three ticks. But to do the physics math against the velocity, the software had to convert that integer into a real floating point number. It had to multiply the integer tick count by one tenth.
And as we established earlier with the base two illusion, one tenth, zero point one is an infinitely repeating fraction in binary. It mathematically cannot be stored exactly.
Right. And because they were using a 24-bit register, that infinitely repeating fraction had to be truncated relatively early. Every single tenth of a second, when the system calculated the time, it lost a microscopic sliver of precision. It was off by roughly 0.000000095 seconds.
A tiny tiny fraction of a fraction of a second. Now, if you just turned the Patriot system on, tracked a target, and fired a missile within a few minutes, that error is entirely negligible. It wouldn't miss by an inch. But in Desert Storm, alpha battery was deployed in a static defensive posture. And we're told to leave the system on and guard the sky indefinitely.
Which brings us to a crucial concept ULP drift.
Right. ULP drift. ULP stands for unit in the last place.
Wait, what exactly is a unit in the last place? Are we talking about the final switch in the bit budget?
Exactly. Remember the 52 switches for the significant we talked about?
Yeah.
The ULP is the physical value represented by that very last switch, the least significant bit, at whatever exponent scale you are currently using. As your numbers get bigger, the exponent dial turns up, and the value of that single final switch actually gets larger.
Oh. So how does ULP drift apply to the Patriots clock?
Well, the longer the system ran, the larger the integer counting the tenths of a second became. It grew into the millions. And when you multiply a very large integer by a slightly truncated decimal, you amplify the truncation. The ULP drift scales up.
The error physically grows. Israeli forces actually noticed this exact problem weeks earlier. They observed that after the Patriot system had been running for eight consecutive hours, the accumulation of that tiny truncated 0.1 caused a 20% shift in the range gate's tracking window. 20% is huge. Yeah. They warned the US command that rebooting the system, which flushes the memory and resets the clock integer back to zero, was absolutely necessary. But rebooting takes 60 to 90 seconds. And in an active war zone, turning off your only radar defense for 90 seconds is terrifying.
So Alpha battery was left running. Not for eight hours, for over 100 consecutive hours.
Over one hundred hours of relentless ULP drift. The internal time calculation became so incredibly skewed that the predicted location of the target shifted by more than half a second.
And for a scud missile, traveling at Mach 5. Half a second of error is roughly 700 meters in the sky.
700 meters. The radar saw the scud coming. It calculated the range gate where it expected the scud to be next, but the math was so far off that the range gate was essentially looking at an entirely empty patch of sky.
The system looked at the empty box, looked at the radar blip outside the box, and just assumed the radar blip was a false positive, a ghost. Because it didn't align with the mathematical prediction. It ignored the target completely.
The scud slipped completely through the defense system. It struck the barracks in Duran, killing 28 Americans and wounding 98 others. It is one of the most chilling examples in history of why we cannot ignore the physical reality of how our computers store numbers.
It really is.
And the gut-wrenching footnote to this story. A software patch that fixed the 24-bit truncation error by utilizing higher precision math arrived in Duran on a floppy disk one day too late.
It is devastating.
Oh, the Ariane 5 rocket.
Yes. Flight 501, June 4th, 1996. It's maiden voyage.
Right.
The Ariane 5 was a massive leap forward in European spaceflight capability. But developing space software is incredibly expensive and time consuming. So the engineering team made a very common, generally sound decision. They reused a piece of software from the older, highly successful Arian 4 rocket.
Specifically the software for the inertial reference system, which calculates the rocket's position, speed, and horizontal velocity.
Right. Code reuse makes sense. If it worked perfectly on Arian 4 for a decade, why wouldn't it work on Arian 5?
Because of the flight envelope and the primitive types they chose. The software generated a value representing horizontal velocity as a 64-bit floating point number.
Okay.
But then the software attempted to convert or downcast that massive 64-bit float into a 16-bit signed integer for a specific diagnostic subsystem.
The 16-bit signed integer has a very strict ceiling. It can only hold a maximum value of 32,767. Anything larger physically cannot fit in the bits.
Exactly. Now the Arian 4 was a relatively slow rocket. Its horizontal velocity during the critical phase of flight never came close to exceeding that 32,767 integer limit. So the downcast conversion always worked perfectly.
But the Arian 5 was a much more powerful, much faster rocket.
Way faster. So very early in the flight, the horizontal velocity value blew past 32,767.
And when the system took that large 64-bit float and tried to aggressively cram it into the tiny 16-bit integer box, the hardware basically rebelled. It caused an arithmetic overflow exception. The software crashed.
Yep.
Because this was a critical aerospace system, it immediately failed over to the backup system. But the backup system was running the exact same software. So it crashed a fraction of a second later for the exact same reason.
Both computers die. The rocket is flying blind.
The rocket's main computer interpreted the diagnostic crash data as actual flight data. Thought the rocket was veering wildly off course, and triggered a massive engine gimbal over correction.
And the aerodynamic forces literally ripped the rocket apart 37 seconds after liftoff. A $500 million dollar fireworks display purely because of a mismatched numeric primitive and a failure to handle the physical reality of overflow.
It is terrifying.
And you know, I think a lot of developers sit here and think, well, I don't program rockets and I don't program air defense systems. I build web apps, I build enterprise SAS. I'm safe from this.
But they aren't.
This exact same numerical sabotage happens in modern web development every single day. Let's look at Twitter style Snowflake IDs. Yeah.
Snowflake IDs are 64-bit integers. They're used to generate massive, globally unique IDs for database records, like a tweet ID or a Discord message ID or a user profile.
Right. They're incredibly useful for distributed back-end systems because they include a timestamp and a machine ID. So multiple servers can generate them independently without overlapping.
Exactly. So the back-end database, like Postgres, handles 64-bit integers perfectly. The server backend, maybe written in Go or Rust, handles them perfectly.
But eventually that ID has to be sent to the front-end user via a JSON API. And the web runs on JavaScript.
And historically, JavaScript only had one number type for everything. The 64-bit IEEE 754 float. It did not have true 64-bit integers.
So let's debug this live. I am a front-end developer staring at my browser console. I click delete on a user profile with an ID ending in 82, but the API request goes out and my backend deletes the user ending in 80 instead.
Just silently deletes the wrong one.
Yes. How does the browser silently change a string of characters in a JSON payload into the wrong number without throwing a single error?
It comes back to the bit budget. The Snowflake ID is a 64-bit integer. But JavaScript parses all numbers as a 64-bit float. Remember the breakdown. You have one sign bit, 11 exponent bits, and only 53 bits of precision in the significant.
You are trying to stuff 64 bits of raw integer data into a bucket that only has 53 bits of precision.
Exactly. The maximum safe integer you can accurately represent in a standard float before you start losing the ability to dis to distinguish between odd and even numbers is two to the power of 53 minus 1, which is roughly 9 quadrillion.
Now, 9 quadrillion sounds huge.
It does. Until you realize that Snowflake IDs embed a high resolution timestamp in the upper bits. Modern Snowflake IDs are routinely much larger than 9 quadrillion.
Okay, so when the back end sends this massive ID to the front end via JSON, the browser's JSON parser reads the number, it hits that 53-bit precision wall.
And the trailing digits of the integer literally fall off the edge of the significant ruler.
So what does the browser do when it runs out of bits?
It rounds it. It rounds it to the nearest even representable number on the floating point grid. So your ID ending in 82 and a completely different database record with an ID ending in 81, both get silently parsed and rounded to ending in 80 by the browser.
The data corruption is entirely silent.
The front end fetches the wrong user or updates the wrong record because the browser crushed the primary key into a float. It is the Vancouver stock exchange all over again. Just shifted from money to primary keys.
It absolutely is.
And this perfectly illustrates why you have to understand the physical constraints of the types you are using. Okay. So we've seen what happens when truncation accumulates over time, and when large numbers get crushed into smaller boxes.
Right.
But truncation is just one trap. Let's explore what happens when you try to smash two numbers of completely different scales together. Like, do they break differently?
This brings us to part four.
The void of non-associativity and edge case disasters. Because when you try to add vastly different numbers, floating point math completely destroys a fundamental rule of mathematics that we all learn in elementary school, the associative property of addition.
Right, in pure abstract math, if you have three numbers, A, B, and C, it doesn't matter how you group them. Parentheses around A plus B plus C is mathematically identical to A plus parentheses around B plus C.
Right.
If I have a bucket of water and I pour in a cup and then add a single drop, it shouldn't matter what order I do it in. The total amount of water is exactly the same.
Exactly.
But how does the computer drop the ball here? I'm trying to picture how non-associativity actually works mechanically.
Let's use an audio analogy. Imagine you are trying to record the loudest heavy metal rock concert in history. The sound wave is massive. Let's call that number one times ten to the twentieth power. One E20. A one with 20 zeros.
That is a very loud concert.
Very loud. Now, at the very back of the stadium, someone drops a pin. That tiny sound is zero point one.
Okay, a massive number and a tiny number.
To record the rock concert without blowing out your speakers, you have to turn the microphone sensitivity, the gain, way down. In floating point math, this is the equivalent of the computer aligning the exponent dials.
Oh, to add two numbers together, the computer must shift the exponent of the tiny number to match the exponent of the massive number. So their decimal points line up?
Yes. It has to slide the significant ruler of the pin drop to match the rock concert. But because you turn the microphone gain down so drastically to handle the concert, the signal of the pin drop is too weak to even move the microphone membrane. Wow. When the computer shifts the tiny numbers exponent to align with one e20, the 0.1 gets pushed entirely off the edge of the 53-bit significant ruler. It literally falls into the void.
The computer just drops it entirely to keep the exponents aligned?
Exactly. So if you type parentheses 0.1 plus 1 e to the 20th minus 1 e to the 20th, the computer first evaluates the parentheses. It tries to add the pin drop to the rock concert. It aligns the exponents, the pin drop falls off the edge, and the result is just one e to the twentieth.
The pin is gone.
Then it subtracts one e to the twentieth, a massive number minus itself. The result is exactly zero.
But if we change the grouping. So a plus b, then plus c equals zero, but a plus b plus c equals zero point one. The associative property is completely broken.
The order of operations changes the physical result because of how the bit budget shifts the rulers.
And this isn't just fun math trivia. This specific behavior breaks determinism in modern artificial intelligence. It breaks large language model inference.
Yeah, this is a huge deal right now.
Wait, how does it break an LLM? I thought LLMs were just predicting the next word based on weights.
They are, but under the hood, predicting that next word requires billions of matrix multiplications and additions happening simultaneously on GPUs. Now, if you set an LLM's temperature parameter to zero, you are telling it to be completely deterministic.
Like don't be creative, just pick the most probable word. You expect that if you give it the exact same prompt, it should give you the exact same answer, word for word every single time.
Right. But it doesn't always do that.
Why not?
Because GPU architectures process math in parallel to be incredibly fast, depending on the batch size of the request or how the GPU decides to thread the workload on that specific millisecond. The physical order in which those billions of floating point numbers are added together might change slightly.
Oh. And because floating point addition is non-associative, changing the order of additions changes which tiny fractions get dropped off the edge of the ruler.
Exactly. The microscopic rounding errors shift. And in a deep neural network, those tiny differences propagate. They amplify layer by layer through hundreds of layers until the final probability distribution of the next token actually changes.
So the LLM might pick a completely different word simply because of rounding dust.
Yes. And changing one word alters the context for the entire rest of the generated essay. A zero temperature LLM can become non deterministic purely because of hardware level floating point non-associativity.
That is insane. It's the butterfly effect, but the butterfly is a dropped bit on a GPU register. And this scale mismatch problem where large numbers swallow small numbers, it shows up in simulations and video games too. Let's talk about the expanding grid and the classic odometer bug.
Oh, the odometer bug is a fantastic C programming exercise. It usually leaves students scratching their heads, convinced the compiler is broken.
The setup is simple. You write a program to simulate a car's odometer. You declare a 32-bit float called meters. You write a for loop that runs 100 million times. Okay. Inside the loop, you add 0.01 to meters. So you are adding 10 centimeters a tick. At the end of 100 million ticks, you should have exactly one million meters or 1000 kilometers.
But you don't.
You don't. If you run this code, the odometer counts up normally for a while, but then it completely stops. It gets permanently stuck at 262.144 kilometers. You can let the loop run for another billion times and it will never tick past 262.144. It is frozen. Why?
It is because of the exponent dial and how it stretches the floating point grid. As the meters variable grows larger, the exponent aisle has to click up to accommodate the larger scale.
Right.
And every time the exponent clicks up a power of two, the physical gap between representable numbers doubles. The floating point grid is not evenly spaced like a standard ruler. Close to zero, the numbers are microscopic. The grid lines are packed incredibly tight. But as you get larger, as you get larger the gaps between the valid numbers stretch out.
Like walking on stepping stones that get further and further apart the further you walk from the center.
Precisely. And 2624 is exactly two to the eighteenth power. When the odometer reaches that exact number, the gap between that number and the very next possible number the 32 bit flow can physically represent becomes larger than 0.01.
Oh wow.
So when you try to add 0.01, the system aligns the exponents, the 0.01 falls completely off the 24-bit significant ruler, and it adds nothing. The number just swallows the tiny addition forever.
It is literally impossible for a 32-bit float to add 10 centimeters once you get 262 kilometers away from zero. It just drops it. And this brings us to one of the most famous examples of this expanding grid the deep space kraken in Kerbal Space Program.
I love this bug. It is a legendary piece of gaming history and a perfect visualization of the expanding grid.
Yeah Kerbal Space Program is this brilliant astrophysics and space flight simulator but in the early versions of the game players notice something terrifying. When they built a spaceship and flew it really really far away from the origin point of the solar system the physics engine would start to glitch. Right. The ship would vibrate violently parts would clip into each other and eventually the entire spaceship would just tear itself apart in the void of deep space. The gaming community named this invisible monster the deep space kraken.
And the Kraken was entirely a floating point artifact.
Right. The 3D game engine used 32-bit floats to track the geospatial coordinates of the ship's parts in space. When the ship was near the home planet, close to the 000 origin of the universe, the floating point grid was incredibly telt.
So the physics engine could calculate the exact position of every bolt, wing, and thruster with millimeter precision.
But as they flew billions of virtual meters out into deep space.
The exponent dial cracked up.
The gaps between representable coordinates became massive. The grid stretched out to the point where the distance between valid coordinates was feet or even meters wide. Suddenly the physics engine couldn't place the ship's parts exactly where they were supposed to be based on velocity. It had to round their positions to the nearest available grid point. One part of the ship would get snapped to a coordinate a foot to the left, while an adjacent part got snapped a foot to the right.
And those rounding gaps literally ripped the virtual metal apart. The math itself became jagged and violent.
So how did they fix it?
To fix it, the developers had to implement a system called Kraken Spain. They essentially stopped moving the ship through the universe. Instead, they constantly re-centered the universe around the ship.
Oh.
They kept the ship perpetually at the zero, zero, zero origin where the mathematical grid is tightly packed and safe, and they move the rest of the universe around it instead.
That is a phenomenal hack.
It is.
But it illustrates the danger of the expanding grid perfectly. Now, what happens when you do the opposite? What happens when you subtract two numbers that are massive but almost identical?
Catastrophic cancellation.
It sounds like a heavy metal band, but it is an absolute nightmare for algorithms that calculate variance, standard deviation, or quadratic equations.
Explain how it works.
Let's use the classic quadratic formula. B squared minus 4AC.
Imagine B squared evaluates to a massive number, and 4AC evaluates to a massive number that is almost identical to it. Because they are both incredibly huge, they require a high exponent.
Which means their trailing digits have been heavily rounded off or truncated by the bit budget to fit into the significant.
Right. They are approximations. The leading digits are accurate, but the trailing digits are contaminated by rounding errors.
Okay, so you have two massive approximations.
Now you subtract them. Let's simplify and say the numbers are 11.2 and 11.1, but scaled up massively. When you subtract them, the accurate leading digits, the 11, completely cancel each other out to zero. And what is left over?
The point one. But wait, the leading digits were the accurate ones. The trailing digits were the ones contaminated by the rounding errors.
Exactly. The accurate mathematical information is entirely annihilated by the subtraction. The only thing left behind in the result is the microscopic rounding error, which is now magnified and sitting at the very front of the new number.
The error becomes the entire answer.
The calculation is catastrophically canceled. You can end up with a negative variance, which is mathematically impossible, purely because the true answer was swallowed by the subtraction, leaving only the toxic rounding dust behind.
Toxic rounding dust. I love that. So we have built this massive house of horrors. We have truncation bleeding stock markets, ULP drift, crashing missiles, overflow conversions blowing up rockets, JSON parsers corrupting primary keys, non-associativity breaking deterministic AI, and the Kraken tearing apart spaceships. It is a bleak landscape.
It is a total minefield.
Which means we have to give you, the listener, the playbook. If you are writing this code, how do you protect yourself? Armed with the knowledge of how and why these primitives fail, what exact rules should you bake into your mind and into your AI prompts to build bulletproof software? Because this brings us to part five and what we call the vibe coder trap.
Yeah, the vibe coder trap is incredibly real right now. With AI coding assistants like Copilot and Claude so prevalent, many developers just prompt build me a shopping cart or write a SQL schema for a financial ledger.
And the AI, unless strictly instructed otherwise, will reflexively reach for the most common numeric primitive it knows, the binary float.
Right. The AI spits out a SQL schema with price float.
You write a quick unit test checking if a five dollar item adds to the cart, it passes, the vibes are good, you approve the PR, and you push to production.
And then a week later, you try to split a $10 tax bill three ways, and your accounting software leaks pennies.
Or worse, a customer pays off their loan, they have a zero balance, but the database says they still owe.0001 cents. So the automated system locks their account for nonpayment.
Which leads us to rule one of the playbook. Never ever use binary floats for money or financial ledgers. Period.
Never.
Financial systems require base 10 exactness. They require the math to behave exactly the way human accountants expect it to behave, without toxic rounding bust.
So what do you use instead?
You use exact, arbitrary precision signed decimal types. In Java, this is big decimal. In Python, it's the decimal module. In SQL, it is the numeric or decimal column type.
Because these types do not use base two hardware floats.
Right. They store the numbers essentially as arrays of base 10 digits, keeping the math perfectly aligned with human currency. Or if you want high performance, you use integer minor units. You store ten dollars and fifty cents as the integer 1050 cents. This is the pattern Stripe uses.
But Stripe's integer approach has a caveat you have to be careful with, right?
Yes. You must respect zero decimal currencies. The Japanese yen doesn't have cents. So a hundred yen is just the integer 100.
If you try to divide it by 100, assuming it's cents, you instantly corrupt the ledger.
Exactly. But whether you use integers or big decimals, the rule's the same. No hardware floats for money.
Okay, so rule one is use big decimal or integers, but that leads directly into rule two, which is a massive trap that catches senior developers all the time. The string initialization trap.
Oh, this one hurts.
Let's trace this live. If I am coding in Python or Java, I know I shouldn't use floats. So I type big decimal point one. Am I safe?
Absolutely not. You are already dead.
Why? I'm using the safe type. The class is literally called big decimal.
Because of the order of operations in the language compiler. Before the big decimal constructor even receives the number, the language parser looks at your code. It sees the float literal point one sitting there inside the parentheses.
Oh no.
It immediately converts that literal into the IEEE 754 hardware float approximation. It converts it to 0.1000000000000001555.
Oh man.
And that toxic, jagged approximation is what actually gets passed into the big decimal constructor. You just took an incredibly precise, mathematically perfect base 10 volt, and you locked a corrupted, infinitely repeating base two fraction inside it. The big decimal will faithfully preserve the exactness of the error forever.
So how do you avoid it?
You must initialize exact decimals using strings. You write big decimal point one wrapped in quotes.
Because when you pass a string, the hardware float parser is bypassed entirely. Exactly. Exactly.
The big decimal library parses the text characters, zero, point, and one, and constructs the exact base ten representation safely.
Quotes save lives. Rule two, always string initialize your decimals. Okay, rule three. Let's talk about rounding. If you are forced to round a number, say you are calculating interest and you have a half cent, how do you do it? Grade school taught us that if it's 0.5, you always round up.
And grade school lied to you, at least for software engineering at scale.
Really?
If you always round up on.5, you introduce a massive statistical bias. Think about a huge data set of financial transactions. If you have a perfectly even distribution of numbers, rounding up at 0.5 means you are systematically adding slightly more value to the ledger than you are removing.
Over millions of rows, that bias accumulates into phantom money.
Exactly.
So what's the solution?
Rule three is bankers rounding. In code, this is usually called round half even.
How does that work mechanically?
If the number is exactly equidistant, like 2.5, you round to the nearest even neighbor. So 2.5 rounds down to 2, but 3.5 rounds up to 4.
So half the time it rounds up and half the time it rounds down, depending entirely on the preceding digit?
Exactly. It statistically minimizes cumulative rounding errors over a sequence of calculations. It balances the bias. And if you are physically allocating pennies, like splitting a $10 charge three ways, you don't just divide by three and round.
What do you do?
You use Martin Fowler's remainder preserving allocation pattern. You give $3.33 to all three buckets, calculate the remaining one cent, and deliberately distribute that remaining penny to the first bucket. No value is mathematically created or destroyed.
That is robust engineering. Rule four. What if you are forced to sum a massive array of hardware floats? Say you were doing scientific data processing, and you have to add thousands of numbers, and you know the bit budget is going to cause truncation and non-associative drift on every single addition.
Instead of writing a standard four loop and keeping a running total, which forces a rounding event on every single loop iteration, compounding the error exponentially, you rely on specialized math functions based on cahan summation.
Like what?
In Python, this is math.f sum.
How is F sum different from just writing sum?
Math.f sum uses extended precision under the hood. As it iterates through the array, it actually tracks all of the lost digits that get truncated during the intermediate steps.
Right, it saves them?
It keeps a separate mathematical ledger of the errors. And it only applies a single rounding operation at the very end of the entire summation.
It saves the rounding for the very end. That is brilliant. It completely bypasses the catastrophic cancellation and the non-associative drift we talked about.
It is much slower than a standard hardware sum, but it guarantees absolute accuracy even when large magnitude inputs are canceling each other out.
Which brings us to our final rule. Rule 5. Knowing when not to overengineer. Because we just spent the last hour terrifying everyone about hardware floats. It would be very easy to leave this deep dive and decide I'm gonna use big decimal for literally everything. I'm putting it in my 3D game engine, I'm putting it in my machine learning models.
Please do not do that. It will utterly destroy your throughput.
Why?
Because big decimal is software emulated math. It has to allocate memory objects, parse strings, and do math using arrays of integers. It is magnitudes slower than a hardware float.
And hardware floats are fast.
IEEE 754 floats are baked directly into the physical silicon of the CPU and the GPU. They execute in a single clock cycle. If you force software decimals into geospatial distance calculations or 3D physics engines, your application will grind to an absolute halt.
So for the deep space kraken, the solution wasn't big decimal.
No. The solution was simply switching from a 32-bit float to a 64-bit double. That pushes the expanding grid, the rounding gaps, billions of miles further out into virtual space.
It gives you 53 bits of precision instead of 24.
Right. It utilizes the blazing speed of the 64 bit hardware registers while providing enough precision that the truncation errors are smaller than a millimeter.
And in machine learning, we are actively going the opposite direction.
We are. To train massive LLMs faster, we are intentionally throwing away precision. We are using B float 16, 16-bit floats.
Because in ML training, the neural network is incredibly resilient to microscopic noise. We don't need accounting precision. We need massive parallel throughput to calculate gradients.
Exactly. The noise from the truncation actually acts as a form of regularizing dropout.
The error literally becomes a feature.
It's about knowing your domain. If you are doing continuous physics, graphics, or ML, you use hardware floats. You accept the jagged grid, and you manage the errors using relative tolerances, checking if numbers are close enough rather than exactly equal.
But if you are doing discrete financial ledgers or JSON IDs, you strictly use strings, integer minor units, and software decimals.
Spot on.
Okay, let's pull all of this together. We've gone from the microscopic architecture of 64 tiny silicon switches all the way up to exploding rockets, silently shifting range gates, and nondeterministic AIs. What does this all mean for you, the developer listening to this?
It means that every time you declare a variable type in your code, you are making a physical compromise. The physical universe outside your window might be continuous, infinitely smooth and analog, but our digital reality is built on a finite grid of bits. And because of floating point math, that grid isn't evenly spaced.
The squares on the grid get exponentially larger and more distorted the further you travel from zero. It's like a funhouse mirror of mathematics.
Every single piece of software you use, every game you play, every financial transaction you make, is constantly making microscopic, high-speed compromises to hide this jagged, pixelated reality from you. The illusion of a smooth, mathematically perfect digital world is maintained only by the deliberate managed chaos of William Kahan's bit budget.
And the next time you open a terminal and see a point zero zero zero zero zero zero zero zero zero zero zero zero zero four four pop up on your screen, you'll know that you aren't looking at a broken computer. You are looking at the seams of the digital universe briefly tearing open, revealing the gears of the machine underneath.
You stop trusting the machine blindly, and you start engineering alongside its physical reality.
That is beautifully said. The digital world is finite, but the problems we solve with it are boundless. Next time, we're moving from the math that breaks our logic to the text that breaks our screens. Join us for season eight, episode eight, where we tackle the absolute nightmare of string encoding and Unicode. Until then, keep an eye on your pennies and watch out for the deep space kraken.