Why Single Threads Have Race Conditions
Transcript
Made automatically from the audio, so it may contain mistakes. Click a line to play from there.
I want you to picture something for a second. Think about the last time you bought a digital gift card online.
Oh, sure.
You get the email, right? You click the little redeem button, and your account balance just ticks up by 50 bucks.
It feels incredibly simple.
Yeah, exactly. Underneath the hood, when you look at that back end logic, there is this comforting expectation of precision. You expect it to look like a clean engineering diagram.
Right, like step one, step two.
Exactly. The application asks the database, you know, what is the balance? It gets a number, it adds 50, and it saves the number.
It's sequential.
It's binary.
It makes sense. It is deeply comforting to think of it that way. Yeah. I mean, we naturally crave that kind of visible step-by-step logic.
We really do.
We want to believe our software executes exactly the way we read it on the screen. Top to bottom, one clean action after another.
But then you step into the world of concurrency.
Oh boy.
And that comforting diagram completely disintegrates. You are suddenly looking at a logical landscape that is honestly just terrifying.
It really is the absolute definition of diagnostic muddy waters.
Muddy waters is a good way to put it. We're talking about a world where the fundamental rules of time and sequence seem to break down right in front of your eyes.
Because when you introduce concurrent operations, you know, you are no longer just managing data. You are managing the physics of time itself.
And that brings us directly to the mission of today's deep dive. Now, in our last session together, you'll remember we established exactly why servers fundamentally have no shared wall clock now.
Right. The distributed illusion of time across networks.
Exactly. But today we are taking that concept and shrinking it way down. We're gonna discover that even inside a single process on a single CPU thread, overlapping tasks turn a simple check the balance, then update the balance operation into a total wallet draining nightmare.
Yeah, and we're gonna explore this through the lens of some of the most foundational computer science papers ever written. The heavy hitters. Oh yeah. We are talking about the pioneers, people like Edsker Dijkstra, Rob Pike, and Russ Cox.
And we aren't just gonna look at their theories in a vacuum, right?
No, absolutely not. We are gonna map their insights directly onto modern vulnerability databases.
Because we want to see exactly how time and state are violently colliding in the applications you are building and using every single day.
Well, I want to let's actually start immediately by attacking what might be the single most expensive myth in modern web development.
I hear this all the time from developers actually.
I bet you do. It goes exactly like this. My back end runs on a node event loop or it runs on Python's async I.O.
Right.
Therefore, my environment is strictly single threaded. And because it's single threaded, I physically cannot have race conditions.
It is a massively pervasive myth. And arguably, you know, one of the most dangerous assumptions a developer can make today.
The idea that a single thread protects you from concurrency bugs is just a fundamental misunderstanding of what concurrency actually is.
Let me paint a vivid picture for you of just how dangerous this assumption is. I want you to imagine a security researcher or a malicious actor targeting an e-commerce platform's wallet system.
Okay, setting the scene.
The back end developer for this platform believed the single thread myth. They assumed, hey, only one thing can happen at a time in my application memory, so I don't need to aggressively synchronize my database calls.
Which leaves the door wide open for something we call a single packet technique.
Exactly. So the attacker doesn't just click redeem 20 times really fast on their browser.
Right, because that would send 20 separate network requests over a few seconds, which the server could probably handle sequentially.
Yeah, the server would just process them one by one. Instead, the attacker carefully packs 20 completely separate gift card redemption payloads into one single highly compressed HTTP2 network packet.
Ooh, nasty.
Right. They send this single packet over the wire. Because those 20 requests arrive in the exact same TCP transmission, they hit the server's application layer at the precise same millisecond.
And this is where the mechanics of the event loop take over and where the developer's assumptions just completely fall apart.
Walk us through it.
So the server's event loop receives this massive network payload. It unpacks it, sees 20 requests, and starts processing them. It iterates through the first request, says, okay, I need to check the balance, and fires a query off to the database over the network.
But, and this is the crucial part, because it is a modern non-blocking event loop, that single thread does not sit there and wait for the database to reply.
Not at all. Waiting is wasted CPU time. The instant it sends the first query over the network to the database, the thread says, I'm paused on this task. What's next?
It grabs the second gift card request. Right.
It fires off a second balance query to the database, then the third, then the fourth, all the way up to 20.
So before the database has even had a chance to process the first query and send a response back across the network, the application has successfully fired off 20 separate what is the balance questions.
Yeah. And then the database starts answering.
It replies to the first query, the balance is zero. It replies to the second, the balance is zero.
It tells the application 20 times that the user has a starting balance of zero.
And the application happily receives those 20 answers. For each one. The application logic says, Great, the balance is zero, the gift card is worth 50. Let's update the database to 50.
It fires off 20 completely independent update commands.
The server reads the current balance 20 times, sees that the user hasn't exceeded any limits 20 times, and proceeds to credit the account 20 times.
Before the very first database update actually commits and changes the state of the data, the application is authorized 20 times the actual value of the gift card.
The single thread didn't protect them at all.
Not even a little bit. Exactly.
It is entirely about the illusion of time inside the processor itself and how our modern software paradigms attempt to manage the concept of waiting.
The event loop is a brilliant invention, right? It was designed to handle massive scale and high concurrency without the crushing memory overhead of spawning a new operating system thread for every single user.
But utilizing it safely requires a very specific, almost philosophical understanding of what it actually means to juggle tasks.
Let's unpack that juggling concept, actually.
Because if a single threaded event loop can drop the ball so catastrophically, we have to precisely define what juggling actually means in the context of computer science.
Which brings us right to Rob Pike, one of the co-creators of the Go programming language, and his classic distinction between concurrency and parallelism.
Rob Pike articulated this better than almost anyone.
He really did. He stated very clearly, that concurrency is the composition of independently executing processes. It is a way of structuring a program. It is about dealing with lots of things at once.
While parallelism, on the other hand, is the simultaneous execution of computations. It is about physically doing lots of things at the exact same time. Right.
I've always loved Pike's own juggling analogy for this, so let's build on it for you.
Yeah, let's do the circus analogy.
Imagine a circus. Concurrency is the act of structuring the juggling performance. It's having a precise plan for how one single juggler is going to manage keeping five bowling pins in the air.
Because the juggler can only physically touch two pins at any given millisecond, right? One in each hand.
Exactly. But by perfectly timing the throws and the catches, by interleaving the tasks, they manage five pins at once. That is concurrency, dealing with many things even with limited resources.
While parallelism is just a matter of resources. Parallelism is hiring five different jugglers, putting them on five different stages, and having them each toss a single pin up and down simultaneously.
You are physically executing multiple actions at the exact same moment.
So when we talk about a single-threaded event loop like what powers Node or Python's async.
You have exactly one juggler.
That single main thread runs in a continuous loop. It picks up a task, say a user's web request, and it executes the code until it hits an I.O. boundary.
And an I.O. boundary is any moment where the code has to talk to something outside of the CPU, reading a file from the hard drive, making a network call to an API, or querying a database.
When it hits that boundary, the thread cannot physically proceed until the external system answers.
In older synchronous models, the thread would just freeze and wait, completely wasting the CPU's potential.
But in cooperative multitasking, the thread essentially puts that specific pin in the air and says, I will come back to this when the database answers.
It cooperatively yields control back to the central event loop, which then picks up the next task.
But there's a massive vulnerability in the word cooperative because it relies on the tasks willingly giving up control.
Right. What if you hand this single juggler a 100-pound boulder?
Say a massive CPU-bound task like calculating a complex cryptographic hash or parsing a gigantic JSON file synchronously.
That is where cooperative multitasking breaks down. If the task doesn't contain an I.O. boundary, it never yields. The juggler is forced to hold that hundred-pound boulder and stare at it while they process the math.
And because they only have two hands, they can't catch anything else.
Which is exactly why environments like Node have to cheat a little bit.
Right, they have that hidden worker pool.
Yeah, Node has that single event loop, but behind the scenes, it relies on a C library called libUV, which maintains a hidden worker pool of separate threads.
Yeah, but whenever Node encounters one of those heavy boulders like a file system read or a cryptographic function, it secretly passes the boulder off to a background worker thread.
That way the main juggler keeps their hands free to manage the network requests.
And the orchestration of that main event loop is incredibly intricate, isn't it?
It operates in distinct phases. It checks a phase for pending timers, then it moves to a phase to pull the operating system for new network I.O. Then it handles deferred callbacks and so on. It is a highly engineered merry-go-round.
But even that merry-go-round has traps. If you've worked in Node, you might know about process.next tick. Oh, yeah.
The dangerous cue.
It's a special cue that tells the event loop, execute this callback immediately after the current operation finishes, before moving to the next phase.
And if you accidentally write a recursive function that keeps scheduling things in the next tick queue, you create a black hole.
The event loop gets stuck in that queue forever. The polling phase never runs, no new network requests are answered, and your server is effectively dead, even though the CPU is spinning at 100%.
It highlights how fragile the illusion of simultaneous execution is when you only have one thread.
Okay, I have a pushback here though. Let's go back to the gift card heist for a second. Sure. If I am the developer and I only have one juggler, and they can physically only execute one line of code at a time in that main thread, how on earth do the states collide? If the code is literally processing one instruction after another, how does the race condition actually manifest?
The collision doesn't happen while the code is running. The collision happens in the spaces between.
The spaces between.
It happens in the waiting. Let's go back to your juggler. When the juggler throws a pin high into the air, which in your code is typing the word await before a database query, they're intentionally looking away.
Right, to keep the server responsive.
But here's the critical flaw. While they are looking away, the state of the world continues to evolve. When they finally look back up to catch that pin, when the database query returns and the code resumes, the universe is not the same as when they threw it.
Oh wow.
While they were busy handling other requests, another user might have mutated the database. But the juggler's local memory, the variables they had saved right before they hit a wait, represent a ghost.
A ghost.
They represent the world as it used to be.
So the code sequence isn't broken by another thread physically running over it and corrupting the memory. It's broken by the thread itself volunteering to go into a coma and then waking up blindly, assuming nothing in the external world has changed.
Precisely. You check the balance, you go into a coma, someone else changes the balance, you wake up, and you authorize a transaction based on the ghost balance.
That is wild.
Now, to truly appreciate this, we have to contrast it with the older paradigm preemptive operating system threads.
This is where the OS acts like a dictator, right?
Yes. In a preemptive threaded model, the operating system's kernel is the absolute boss. You spawn multiple threads, and the OS kernel decides who gets to run on the CPU course.
And the kernel does not wait for you to cooperatively yield.
No, it operates on a timer. When your thread's time slices up, the kernel violently pauses your thread right in the middle of whatever it was doing.
It saves the exact state of your CPU registers, swaps out your memory context, and forcefully hands the core to another thread.
Which introduces a completely different, much more aggressive flavor of chaos. Because the kernel could pause your thread literally right between reading a variable and writing to it.
Which sounds awful. And this violent preemption is the entire reason why languages like Python have historically relied on the infamous global interpreter lock or the GIL.
Exactly. Python's internal memory management, particularly how it counts references to know when to garbage collect objects, was never designed to survive being violently paused and modified by multiple OS threads simultaneously.
So the JIL is basically a massive blunt instrument. It's a rule that says, I don't care if you spawned 50 OS threads, and I don't care if you have a 64 core processor, only one thread is allowed to hold the lock and actually execute Python bytecodes at any given microscopic fraction of a second.
It acts as a universal shield for the interpreter. It prevents the internal memory from corrupting, but the cost is astronomical. It completely destroys the ability to perform true parallel CPU bound work within a single Python process.
If you have heavy math to do, the GIL forces those 50 threads to wait in a single file line.
But the Python ecosystem is finally trying to kill the Jill, aren't they?
Yeah, I've been following the discussions around PEP 703. It's a massive architectural shift to make the GIL optional.
It is perhaps the most ambitious engineering effort in Python's history. The core challenge they face is this. If you remove the GIL, how do you prevent the garbage collector from corrupting when multiple threads look at the same object?
Because in a standard multi-threaded C or REST environment, every time a thread wants to interact with a shared object, it has to use specialized heavy-duty atomic CPU instructions to safely increment a reference counter.
And doing that for every single object, every single time it's touched in a highly dynamic language like Python would slow the language to an absolute crawl. It would be unusable.
Right. So the architects of PEP 703 are employing a brilliant compromise called biased reference counting.
They observed a very human truth about software. The vast majority of objects created by a thread are only ever used by that specific thread.
Like temporary variables inside a function. Exactly. And never leave that little bubble.
Exactly. So biased reference counting biases the performance cost. When a thread creates an object, it becomes the owner. The owning thread is allowed to use blazing fast, standard non-atomic instructions to manage a private local reference count.
It acts as if it's completely single-threaded.
But if a different thread suddenly tries to access that object, the system detects it. The second thread is forced to use the slow, expensive atomic instructions on a separate shared reference counter.
That is a phenomenal mental model.
It's like having a private messy ledger for the owner where they can scribble down notes instantly. And a heavily audited, a heavily audited public ledger for anyone else who wants to audit the object.
Most of the time nobody asks for the public ledger, so the system flies.
If you look across the computing landscape, different paradigms solve this in wild ways.
Like Go with its Go routines.
Yeah, Go completely abandons a one-to-one mapping with OS threads. A Go routine is a tiny, incredibly lightweight virtual thread managed by the Go runtime.
The runtime might multiplex a hundred thousand Go routines onto just four heavy OS threads, seamlessly moving them around when they block on I.O.
Or look at Erlang, which powers massive telecom switches and chat apps like WhatsApp.
Erlang just says shared memory is the root of all evil.
It uses an actor model. Every process is an isolated actor with its own private memory. They literally cannot touch each other's variables.
If actor A wants actor B to do something, it has to send a distinct message through a mailbox, just like sending an email. No shared memory means no traditional data races.
And then of course you have Rust. Rust takes the approach of proving safety before the program is even allowed to run.
Right. Rust uses a strict compiler and a borrow checker to enforce ownership rules. It uses special markers called the send and sync traits to track exactly which data is allowed to cross thread boundaries.
The famous fearless concurrency. If you write code in Rust that contains a data race, the compiler literally screams at you and refuses to turn the code into an executable binary. You physically cannot run the bug.
It's a triumph of static analysis. But here is where we have to take a very dark, very sobering turn in our deep dive.
Oh boy.
All of these paradigms, the event loops, the GIL, the Go routines, the Erlang actors, the Rust borrow checker, they all operate in the realm of software. Right. They manage the intent of the code. They manage what we tell the computer we want it to do. But when you strip away the compiler, when you strip away the operating system, and you look at the raw silicon underneath. Okay, we need to spend some serious time here. Because when I first read Russ Cox's work on hardware memory models in preparation for this, it genuinely broke my brain.
It has that effect on people.
As developers, we have this fundamental assumption that the CPU executes our code in the exact sequential order that we wrote it. Step one, then step two, then step three. But you're saying the silicon doesn't actually do that.
Not even close. Modern processors, especially ARM architectures like the Apple Silicon chips in your laptop or power architectures used in big data centers, aggressively disobey the sequential order of your code.
They reorder your read and write instructions on the fly constantly just to squeeze out a few extra fractions of a nanosecond of performance.
Yes.
But why? Why would the hardware intentionally scramble the execution order?
Because of the physics of memory latency. The CPU core is incredibly fast. It operates on billionths of a second. But main memory, the RAM stick sitting a few inches away on the motherboard, is comparatively agonizingly slow.
For a CPU waiting for data to travel to RAM and back is like you ordering a pizza and waiting three years for delivery.
Exactly. And the CPU refuses to wait.
So how does it cheat?
It uses something called a CPU store buffer. When your code tells the processor core to write a value to a variable, the core does not stop and wait for that electrical signal to travel all the way out to main memory.
Instead, the core writes the new value into a tiny hyperfast local cue sitting right next to the execution unit.
This is the store buffer. Once the value is in the buffer, the core immediately moves on to execute the next line of your code. It assumes the write is done. The store buffer will eventually, quietly, flush that data out to main memory in the background.
Okay, let's build a really solid analogy for this. Imagine you live with a roommate and you share a refrigerator in the kitchen.
Okay. The refrigerator represents our main computer memory, the RAM.
Right. A CPU store buffer is like you writing a sticky note that says we are completely out of milk. But instead of walking to the kitchen, you just shove the sticky note in your pocket.
That is a brilliant way to visualize it.
From your perspective, the perspective of that single CPU core, you wrote the fact that there is no milk. If someone asks you, you check your pocket and say, no milk.
But until you actually physically walk into the kitchen and stick that note on the fridge door, until the store buffer flushes to main memory, your roommate has absolutely no idea.
If your roommate walks into the kitchen and looks at the fridge right now, they still think you have milk. The states are completely out of sync.
And this hardware illusion creates mind-bending paradoxes in concurrent code. Russ Cox illustrates this with a classic computer science puzzle. Imagine two CPU processors, processor A and Processor B, running at the exact same time. Okay. Processor A's job is to write the number one to a variable called X. Processor B's job is to write the number one to a variable called Y.
Simple enough.
No, immediately after writing, processor A reads the value of Y.
Mm-hmm.
And processor B reads the value of X. Logically, as a human reading this code, you assume a sequential timeline. Because both processors just wrote a one, you would deduce that at least one of them, if not both, must read a one when they check the other variable.
Right. It's basic deduction. If processor A reads a zero for Y, it implies that processor B hasn't executed its write yet.
Correct.
But if processor B also reads a zero for X, it implies processor A hasn't written yet. They logically cannot both have not written yet. If they are both currently at the step where they are reading, it violates the concept of linear time.
It violates linear logic, yes. But physically, down on the silicon, they can both read zero. And they do. Because processor A executes its right to X, but that write just goes into its local pocket, its store buffer.
And processor B executes its right to Y, and that goes into its local store buffer.
Then they both execute their read instructions. Where do they read from? Main memory. Since neither store buffer has flushed to the fridge yet, they both read the old values. They both read zero.
The hardware has essentially time traveled and allowed the read instructions to logically happen before the write instructions in the shared global timeline.
That is genuinely terrifying for anyone trying to write safe code.
Think about what this means for a basic naive Boolean flag. I see developers do this all the time. They spawn a background task and they have a shared variable called is finished. Right. The background task does a bunch of math, saves the result to a variable, and then sets is finished to true. The main thread is just sitting there in a while loop waiting for is finished to become true.
In a world with store buffers and instruction reordering, that naive Boolean flag is a ticking time bomb. The background core might set is finished to true in its store buffer, but it hasn't flushed a memory.
So the main thread just spins forever, frozen, waiting for a signal that never arrives.
Or even worse, because the CPU aggressively reorders instructions, the hardware might decide to flush the is finished equals true signal to main memory before it flushes the actual math results.
Oh my god. So the main thread sees that it's finished, grabs the result variable, but the result variable is still holding garbage memory because the write instructions arrived out of order.
Exactly. This chaos is why the entire computing industry had to come together and agree on a foundational truce. We call it sequential consistency for data race-free programs, or SCDRF.
It is essentially a complex peace treaty negotiated between compiler writers, hardware manufacturers, and application developers.
What are the terms of the treaty?
The hardware manufacturers say, look, we are going to keep lying to you. We are going to keep using store buffers and reordering instructions because it makes our chips incredibly fast, and that's what sells.
But they offer a concession. The treaty states that if you, the programmer, explicitly use proper synchronization primitives in your code, things like memory barriers or special atomic compare and swap instructions, the hardware promises to temporarily stop lying.
It promises to pause its optimizations, flush the store buffers to main memory, and pretend that it executes those specific operations in a strict sequential order.
Going back to the roommate analogy, a memory barrier is a specific command that forces you to empty your pockets. It says, stop whatever you are doing, do not read or write anything else.
Take the sticky note, walk into the kitchen, put it on the fridge, and wait for everyone to see it before you take another step.
Yes. A memory barrier is a literal physical instruction to the silicon. But if you fail to use those barriers, if you try to be clever and write lock-free code without a deep understanding of atomics, all bets are completely off.
The C11 and Java memory models have to grapple with this extensively, right? I was reading about how without strict mathematical rules defining what the compiler and hardware are allowed to do, you can end up with something called out of thin air values. This sounds like bad magic.
It is the darkest corner of compiler optimization. An out-of-thin air value is a paradox, where an aggressive compiler trying to make your code run faster accidentally creates a self fulfilling prophecy.
How does that even happen?
Imagine the compiler looks at a complex branch of concurrent code. It uses heuristics to guess, you know what, based on this logic, I bet this variable target is eventually going to be set to the number 42.
Okay, compilers do speculative execution all the time.
Right. So to speed things up, the compiler speculatively executes a branch of code, assuming target is 42. But here's the paradox. That very optimization, that speculative branch of code, ends up being the exact code path that actually sets the variable to 42 in another thread.
Wait, really?
Yes. The compiler guessed the value, used the guess to alter the execution, and the altered execution made the guess come true. A value of 42, which logically should never have existed based on the starting state of the program, magically materializes completely out of thin air.
That breaks causality. And the wild part is that no language can completely escape this physics. Not even JavaScript, which developers love to think of as this cozy single-threaded sandbox in the browser.
Exactly. For a long time, JavaScript developers never had to worry about memory models. But then the web evolved. We introduced web workers to do background processing.
And more importantly, we wanted to compile massive high-performance C video games into the browser using WebAssembly. To do that efficiently, WebAssembly needed a way to share writable memory across different web workers simultaneously.
Enter the shared array buffer.
Yes. The moment JavaScript introduced the shared array buffer, it suddenly had shared, writable memory across threads. It invited the vampires into the house.
The JavaScript committee had to formally adopt a complex hardware-aware memory model. They had to introduce the Atomics Global Object, which gives JavaScript developers explicit access to those CPU memory barriers and compare and swap instructions.
It proved once and for all that no language, no matter how high level or abstracted, can escape the fundamental physical reality of the CPU.
So because the silicon is actively trying to trick us with store buffers and because compilers are trying to guess the future, computer scientists have spent the last 50 years designing logical locks. We try to force synchronization.
We force the threads to stop, wait, and coordinate. But forcing things to wait introduces an entirely new catastrophic category of software failure.
Indeed. The moment you introduce locks, you exit the frying pan of data races and you jump directly into the fire of deadlocks and live locks.
And to truly understand the anatomy of a deadlock, we have to go back to the 1960s and look at Edsker Dijkstra. He was one of the very first computer scientists to formally tackle what he called the mutual exclusion problem.
The famous P and V semaphores. I remember learning about these in university and thinking they sounded like secret codes.
Dykstra was Dutch, so the letters actually stand for Dutch words. He realized that to protect a critical section of code like updating a shared bank balance, we needed operations that were mathematically indivisible that could not be interrupted.
He proposed the concept of a semaphore, which is essentially just an integer variable.
But a magic integer.
Right. The P operation, from the Dutch word probarin, meaning to test or try, attempts to decrement the semaphore. If the semaphore drops below zero, it means the resource is locked and the thread is put to sleep by the operating system.
And the V operation from verhogan, meaning to increment, increases the semaphore and signals the operating system to wake up one of the sleeping threads. Because the hardware guarantees these P and V operations are strictly atomic, we can use them to build complex locks.
But locks are incredibly dangerous if you don't orchestrate them perfectly. Which brings us to the work of Edward Kaufman in 1971. He identified the four deadlock conditions.
I want to spend some serious time breaking these down because these four conditions from 1971 are the exact structural reasons why massive production databases suddenly freeze and crash today.
They are the foundational laws of gridlock. Kaufman proved mathematically that for a system to enter a state of true, unrecoverable deadlock, four specific conditions must all be present at the exact same moment.
If you can break even a single one of these conditions, a deadlock physically cannot happen.
Okay, let's build the trap step by step. Condition one. Mutual exclusion.
This is the most basic requirement.
It simply means that a resource can only be held by one thread at a time. It demands exclusive control.
If you are locking a specific row in a Postgres database to update a user's billing tier, no other transaction can touch that row. It is mutually exclusive.
Condition two.
Hold and wait.
This is where the greed comes in. A thread has successfully acquired lock A. It is holding it tight. But to finish its job, it also needs lock B.
So it requests lock B. The lock B is currently being used by someone else. The thread sits down, refuses to release lock A, and just waits. It holds, and it waits.
Condition three. No preemption.
This is a rule about authority. It means the operating system or the database manager is not allowed to forcefully play referee. It cannot step in and violently yank Lock A away from the waiting thread.
The resource can only be released voluntarily by the thread that holds it.
And condition four, the final nail that seals the coffin. Circular weight.
This is the architectural death spiral. There exists a literal closed chain of threads, each waiting for the next. Thread one holds lock A, but is waiting for lock B. Thread two holds lock B, but is waiting for lock C. Thread three holds lock C, but is waiting for lock A.
You have a perfect circle of dependency. Everyone is holding something, everyone is waiting for something, nobody is willing to let go, and nobody has the authority to force them to let go.
The entire system is paralyzed forever.
Dykstra illustrated this beautifully with his classic dining philosopher's problem. Let's walk through it because it perfectly visualizes the Kaufman conditions.
Imagine five brilliant philosophers sitting around a circular dinner table. In the center of the table is a massive bowl of spaghetti. Between each philosopher, resting on the table is a single fork.
So there are five philosophers and five forks.
And here's the catch. Spaghetti is difficult to eat. To successfully eat, a philosopher requires two forks. The one immediately to their left and the one immediately to the right.
Right. So if a philosopher gets hungry, they reach to their left and grab a fork. That represents mutual exclusion. Only one person can hold that fork.
If they grab the left fork, they hold on to it while they reach for the right fork. That is hold and wait.
The forks cannot be stolen out of their hands by force. That is no preemption.
Now, what happens if the universe aligns terribly, and all five philosophers decide they are hungry at the exact same millisecond?
They all reach to their left simultaneously. Every single philosopher successfully grabs their left fork. Now every philosopher reaches to their right to get the second fork they need to eat.
But every single right fork is already firmly in the left hand of their neighbor.
That is the circular weight.
And because of the four conditions, they are trapped. They all sit there holding one fork, refusing to put it down, staring at the spaghetti until they literally starve to death at the table. That is a deadlock.
Dykster's formal analysis of how to escape this trap was brilliant, and it forms the basis of how we design operating systems today. The most common and robust solution is to attack the fourth condition, break the circular weight by implementing a strict lock hierarchy.
How does a hierarchy fix the table?
You assign a unique number to every single fork on the table, from one to five, and you enforce a strict global rule. A philosopher must always attempt to pick up the lowest numbered fork available to them first.
Okay, let's play that out. Philosophers one through four are going to reach for their left fork first, because fork one is lower than fork two, fork two is lower than fork three, and so on.
But the fifth philosopher is sitting between fork five on their left and fork one on their right.
Exactly. Because of the hierarchy rule, the fifth philosopher is forbidden from reaching left for fork five. They must reach right for fork one.
But fork one was already grabbed by the first philosopher. So the fifth philosopher's hand is empty. They are blocked before they even grab a single fork.
Which means fork five is left sitting cleanly on the table. The fourth philosopher can grab it, eat their spaghetti, put both forks down, and the whole system unblocks. The circular dependency is shattered just by enforcing an ordering rule.
It is a beautiful mathematical solution.
But solving deadlocks doesn't mean you've solved concurrency. Because even if you avoid a hard freeze, you can fall victim to something called a live lock.
The live lock.
This is the awkward hallway dance.
It is exactly the awkward hallway dance. Imagine two people walking in opposite directions down a narrow corridor. They suddenly find themselves face to face, blocking each other.
Both are incredibly polite. So person A steps to their left to make room, but at the exact same millisecond, person B steps to their right.
Now they are blocking each other again.
Right. So they both immediately try to correct it. Person A steps right, person B steps left, blocked again. They are actively moving. They are changing state.
The CPU is running at 100% capacity, consuming massive amounts of electricity, but the system is making absolutely zero forward progress. They will dance in the hallway forever.
And then there is priority inversion. This one always sends a chill down my spine because when you read the postmortems, this specific bug has caused catastrophic physical world disasters. It's not just a database freezing, it's machines failing in reality.
Since we know from episode four that time coordination is critical for hardware, how does priority inversion actually break a physical system?
Priority inversion is a catastrophic scheduling failure. In real-time operating systems, threads are assigned strict priorities. You have a high priority thread that absolutely must run instantly, say a thread monitoring a hardware temperature sensor to prevent an explosion.
And you have a low priority thread doing something trivial, like writing a debug log to a hard drive.
Okay, so the low priority thread is running and it grabs a lock on a shared memory bus. Then the operating system pauses it because it's low priority.
Right. Then an emergency happens. The high priority temperature thread wakes up. It urgently needs to access that same shared memory bus to shut off the heater.
But it hits the lock held by the paused low priority thread. So the high priority thread is forced to block and wait.
Which means the most critical system in the machine is now entirely at the mercy of the least critical system.
It gets worse. Imagine a third thread, a medium priority thread handling network traffic, wakes up. Because its priority is medium, the operating system says, you outrank the low priority thread.
Oh no.
The OS allows the medium thread to run indefinitely. The low priority thread never gets CPU time to finish its log and release the lock, which means the high priority temperature thread is permanently starved, the machine explodes.
I know this exact scenario effectively paralyzed the Mars Pathfinder rover on the surface of Mars in 1997. They had to beam a patch across space to fix the lock hierarchy.
But when we talk about actual physical harm to human beings from concurrency bugs, there is a medical case from the 1980s that always comes up in engineering ethics classes.
Yes. You are referring to the Theric 25. It is a tragic, monumental and deeply sobering case study in software engineering, heavily documented in a famous postmortem by Dr. Nancy Levison.
Between 1985 and 1987, the Theric 25, which was a medical radiation therapy machine, massively overdosed six patients, leading to severe radiation poisoning and multiple fatalities.
When I first heard about this, I assumed it was a hardware failure. A broken radiation shield or a jammed physical turntable.
Lethal failures were entirely due to software race conditions.
How does a race condition fire a lethal dose of radiation?
The machine had two modes.
Okay.
A low power electron beam mode and a massive high power X-ray mode that required a heavy physical metal shield to be rotated into place.
The operators controlled this via a computer terminal.
The software was poorly designed with concurrent tasks that shared memory, but lacked proper synchronization locks.
Let me guess. The operator types something, the system starts configuring, and then the operator changes their mind.
Exactly. The operators became incredibly fast at typing the configuration sequences. If an operator quickly typed the command for the high power X-ray mode, realized they made a mistake, hit the up arrow to edit the command, and rapidly changed it to the low power electron mode. The concurrent software threads tripped over each other.
The UI thread and the hardware control thread fell out of sync.
Yes. The user interface thread processed the edit and displayed on the screen that the machine was safely in low power mode. But the background hardware thread had already read the initial high power command, and due to a race condition and how the variables were updated, missed the edit.
The operator hit fire, trusting the screen. The machine fired the massive unshielded X-ray beam, delivering a lethal overdose.
It is horrifying. It proved definitively that life critical software race conditions can be fatal. It highlighted a desperate industry wide need for rigorous state management and the concept of transactions. Which leads us perfectly to the monumental work of Jim Gray.
Jim Gray was a visionary. In 1985 he published a seminal paper on software transactions. Prior to Gray, the industry heavily focused on hardware fault tolerance, having multiple redundant hard drives or physical data paths.
Gray took that concept of fault tolerance and applied it directly to software state.
He formalized the concept of ACID transactions, right? Atomicity, consistency, isolation, durability.
He did. He realized that trying to manually manage individual locks and semaphores for every single variable in a complex system was a fool's errand. Humans are too error prone.
Gray proposed moving away from that and instead grouping operations together into a single transaction.
The database engine guarantees that either all the updates apply successfully as one atomic unit or if anything goes wrong the entire transaction rolls back, leaving no trace it ever started.
And in that very same 1985 paper, Jim Gray also popularized one of my favorite terms in all of computer science the Heisenbug.
Named after the Heisenberg uncertainty principle in quantum mechanics a Heisenbug is a software bug that disappears or alters its behavior the moment you try to observe it.
This is the absolute bane of every developer's existence you have a race condition that crashes your program once a week in production. So you go into the code and you add a simple print statement like made it to step three to see what the variables are doing.
And suddenly, the bug never happens again. You deploy the print statement, and the system is perfectly stable. You remove the print statement and it immediately crashes again.
Because with concurrency, timing is the only thing that matters. The microscopic delay introduced by the CPU having to execute the instructions for that print statement, writing to the standard output buffer, changes the execution timing of that specific thread by a few nanoseconds.
That tiny delay is just enough to prevent it from colliding with the background thread. The bug hides entirely because you looked at it. But today we aren't flying blind in the 1980s anymore. We have incredible automated tools to hunt down these Heisenbugs. I use Go's data race detector all the time. You just type go test-race in your terminal and the compiler magically watches for races. How does that actually work under the hood?
Tools like the Go race detector and particularly Clang's thread sanitizer are masterpieces of dynamic analysis. When you compile your code with the race flag, the compiler actually injects extra tracking code around every single memory read and write instruction in your program.
As your code runs, this tracking machinery monitors every memory address, keeping a history of which thread touched what, and whether a proper synchronization lock was held at the time. But if a Heizen bug relies on a one in a million timing collision, how does the test environment reliably trigger it? Do they just run the test a million times?
Here is where it gets incredibly clever. Thread sanitizer uses a technique called adaptive delay. It artificially and purposefully manipulates the flow of time within your application.
Wait, it slows the code down on purpose?
Highly selectively. The thread sanitizer watches your code execute. When it identifies a high value synchronization point like a relaxed atomic operation or a thread attempting to grab a mutex lock, it injects chaos.
It might artificially insert a random delay loop, forcing the thread to spin uselessly for a few milliseconds. Or it might aggressively call into the operating system and force the current thread to yield its CPU time slice right at the most dangerous possible moment.
That is brilliant.
It is.
Massive network latency spikes artificially.
And because it is all governed by a single random seed, if a complex concurrency bug crashes the cluster, they can replay that exact seed and watch the bug happen again. Deterministically, 100% of the time, down to the nanosecond.
Okay, I have to push back here. I am looking at the timeline. Jim Gray figured out the math for software transactions in 1985. Nancy Levison warned the entire industry about the fatal consequences of race conditions in the late 80s. Right. And today we have compilers that can artificially manipulate time and databases running inside deterministic matrix simulations to catch these bugs. So why, in the year 2026, are modern web developers still pushing wallet draining race conditions into production every single day.
That is the core question of our era. And the answer lies in the double-edged sword of abstraction. The mathematical tools exist, yes, but modern web frameworks and especially the explosion of AI code generation tools actively abstract the danger away from the developer.
We are so heavily insulated from the raw metal of the CPU that we forget the physics apply to us, which leads us directly into a phenomenon we call the vibe coder trap.
The vibe coder trap. I absolutely love this term. It describes perfectly what happens when you just prompt an AI agent, you get a slick block of code that looks structurally sound, it passes your basic happy path unit tests, and you ship it to production.
You are coding on vibes. You aren't engineering state. You are just trusting the aesthetic of the code. And in doing so, you are actively auto completing edge case disasters.
Exactly. AI models are incredibly powerful, but we have to understand how they generate code. Let's walk through the exact four architectural traps that AI coding tools and frankly exhausted human developers auto-complete every single day.
Okay, lay them out for us.
The first trap is perhaps the most classic vulnerability in computing history. CWE367, the time of check, time of use race condition, commonly known as talk to.
Talk to.
This is the exact vulnerability that enabled the gift card heist from our opening story. Without reading raw code, walk us through the anatomy of how an AI confidently writes a talk to trap.
Let's say you ask an AI to write a function to withdraw money from an account. The AI will almost always structure it in three logical but fatal steps. First, it tells the database, hey, go grab the current balance for this user.
It pulls that number, let's say it's $100, out of the secure database and loads it into the application's local volatile memory.
Step one.
Let's check the state.
Step two.
It performs the business logic in the application. It asks, is the balance in my memory greater than or equal to the $50 that he wants to withdraw? Since 100 is greater than 50, the check passes.
Step two.
Validate the state.
Step three is where the trap snaps shut. The AI needs to do something asynchronous. Maybe it needs to call out to the Stripe API to process a fee. Or maybe it just needs to send the final update command back to the database. Whatever it is, the code uses an await keyword.
And right there, at the await, the event loop thread throws the pin in the air, looks away, and goes into a coma.
Yes. It voluntarily yields control. And in that microscopic window of vulnerability, while the first request is sleeping, a second concurrent withdrawal request for the same user arrives.
The event loop picks it up.
It executes step one. Grab the balance from the database. The database still says 100, because the first request hasn't updated it yet. The second request loads 100 into its memory. It passes the check.
And then both requests wake up from their await comas.
And here is the disaster. Both requests execute the final step. They both confidently tell the database, update the user's balance to $50. Or worse, they tell the database, subtract $50 from whatever the balance is now.
The user just withdrew $100 total. But because the world changed while the threads were sleeping, the system completely loses track of the math. The time you check the state is completely disconnected from the time you used the state.
It is the defining flaw of application level state management. Now let's look at trap number two, resource exhaustion, officially tracked as CWE362. This trap isn't about corrupting data, it's a self-inflicted denial of service attack on your own infrastructure.
AI tools absolutely love doing this when you ask them to process lists of data. If you ask an AI to process an array of incoming webhooks, it will almost always write code that maps over the array and wraps the whole thing in a concurrent batching function like promise.all in JavaScript or asyncio.gather in Python.
Which is perfectly fine and actually quite efficient if your array has 10 items in it. The event loop fires off 10 concurrent database queries, the database handles it effortlessly, and you move on.
But what if the incoming webhook payload doesn't have 10 items? What if a malicious user, or just a really busy day, sends a payload with 5,000 items?
The AI's code blindly accepts it. The event loop hits that promise.all and instantly, simultaneously fires off 5,000 independent network queries to your Postgres database. There is absolutely zero concurrency cap.
And that is a structural catastrophe. Your node environment might instantly exhaust its internal libUV worker pool trying to manage the crypto for the network connections.
Or more likely you instantly exhaust the maximum connection limit on your Postgres SQL database. The database goes into protective shock, starts dropping connections, and crashes. Your application runs out of memory. You have effectively DDoSed your own company because the AI assumed infinite resources.
Trap number three is something that catches experienced developers off guard, especially when moving between different back-end frameworks. It's the trap of module-level global variables. This is insidious in stateless server environments like node or Python.
It stems from a misunderstanding of how the server loads code into memory. In a truly stateless web architecture, every single user request is supposed to be a completely isolated bubble.
But if you prompt an AI to create a utility file to manage, say user authentication context, the AI might define a variable at the very top of the file outside of any specific function.
Something like setting a global current user variable and then exporting functions that read and write to it.
Exactly. And because of how node and Python module caching works, that file is only loaded and evaluated exactly once when the server starts up. That global variable represents a single physical location in the server's RAM. And it is shared across every single request hitting that node process.
So user A logs in, the event loop handles the request, calls the utility function, and overwrites that global memory location with Alice's profile data. Then Alice's request hits a database await and goes to sleep.
While Alice is sleeping, user B logs in. The event loop handles Bob's request. It calls the exact same utility function, and it overwrites that exact same global memory location with Bob's profile data.
Alice's request wakes up from the database, resumes execution, reads the global variable to render the HTML dashboard, and accidentally sends Bob's private financial data directly to Alice's screen.
It is a catastrophic headline-making data breach, and it is caused entirely by an AI or a vibe coder failing to understand that in a concurrent process, global module scope is essentially public property.
And finally, trap number four, because we can't let back end developers take all the heat. Front-end race conditions are everywhere, and React use effect hooks are notorious for this.
Front-end concurrency is arguably messier because you are dealing with the ultimate chaotic input, human fingers. Imagine a user interface with a search bar. The user wants to search for shoes. They type S, then H, then O, then E, then S. They hit enter. Then they immediately change their mind, highlight the word, delete it, and type shirts.
Okay, so based on typical React patterns, the UI fires off a network fetch request to the back end API for the search term shoes. And then, a second later, it fires off a completely separate network fetch for shirts.
Now, introduce network latency. Let's say the database query for shoes requires a heavy, unindexed table scan. It takes five agonizing seconds for the backend to crunch the data and send the response over the wire. But the query for shirts happens to hit a warm cache. It returns in just one second.
So the timeline crosses. The user searched for shirts last. The shirts data arrives at the browser almost immediately. The React component renders the list of shirts on the screen, the user is happy. But four seconds later, that slow, heavy shoes query finally finishes its journey across the internet and arrives at the browser.
And because the AI that wrote the React component didn't manage the concurrency, the component blindly accepts the late data. It takes the shoes results and overwrites the screen. The user who explicitly searched for shirts and was looking at shirts, suddenly watches their screen flicker and populate with shoes. The stale old request violently overwrote the fresh new request.
This happens because the vibe-coded component was missing a critical piece of browser infrastructure, the abort controller. Without wiring up a signal to explicitly tell the browser, hey, I started a new search for shirts. Please instantly cancel and ignore that old in-flight request for shoes. The UI is at the mercy of network routing speeds.
Exactly. This raises an incredibly important question. Why does AI struggle with this so profoundly? These large language models have ingested almost every line of code on GitHub. They have read the documentation. Shouldn't they know better?
It has to be a training data bias, right?
It is. AI models predict the next most likely token based on the statistical weight of their training data. And what if their training data consist of? It is heavily skewed towards introductory tutorials, simple blog posts, and highly upboated but context-free Stack Overflow answers.
And tutorial code is fundamentally optimized for readability and brevity, not for production grade, highly concurrent environments. If someone writes a blog post on how to update a database row, they aren't going to include a massive, complex lock hierarchy or distributed transaction logic because it detracts from the core lesson of just showing the SQL syntax.
Precisely. The AI learns the simplified, inherently racy, bare bones version of the code as the absolute ground truth. It actively mimics the naivety of a junior developer writing a tutorial.
So just knowing that these four traps exist isn't enough. We can't just cross our fingers and hope we spot a missing abort controller or a TOCTU flaw in a massive pull request. We need an ironclad structural system for navigating these dangers, especially when we are generating code with AI or writing it ourselves at 2 a.m. Which brings us to the final act of this deep dive the judicious AI prompting playbook.
Yes. If you are using an AI agent like Claude, or if you are building system prompts for an internal coding tool, you cannot rely on the AI's intuition. You must codify the rules of concurrency explicitly in a Claude.md or agents.md file. You have to force the AI to respect the physics of time.
Rule one in the playbook, and this is the most critical SQL atomicity. We have to explicitly instruct the AI to treat every single await keyword in the application code as a cliff. You have to assume that the entire world will change underneath you the moment you type await. Therefore, you cannot pull state up into the application to check it. You must push the check and mutate logic down into the database engine itself.
Exactly. You forbid the AI from writing the three-step TOCTU trap. You tell it, do not read the balance, check it in node, and write it back. Instead, you mandate that it writes a single atomic SQL statement. You tell Postgres, hey, deduct $50 from this account, but only if you can verify in the exact same breath that the balance is $50 or more. And if you succeed, return the new balance to me.
That is architecturally beautiful. Because the Postgres engine guarantees automicity for that single row update. Deep down in the database, it grabs an exclusive lock on that specific row, evaluates the condition, applies the math, and releases the lock, all in one microscopic, indivisible operation. If our second concurrent request tries to hit it, it cues up behind the lock. When it gets its turn, the condition fails, the database refuses to make the update, and your application safely knows the user didn't have enough funds. No overdrawing, no ghosts.
But to rely on that, you have to deeply understand your database's isolation levels. Since we already covered how Postgres handles the default read committed isolation in episode two of this season, you know that under read committed, the database will just automatically reevaluate the updated rows if a concurrent transaction modifies them while you were waiting for the lock.
Right. It's forgiving.
But if we need incredibly strict consistency across multiple tables, we might bump that isolation level up to repeatable read. And that fundamentally changes how our application logic has to behave, doesn't it?
It changes it drastically.
Repeatable read is a much stricter mathematical guarantee. It promises that your transaction will see a perfectly consistent snapshot of the entire database from the exact millisecond your transaction began. However, if two concurrent repeatable read transactions try to modify the exact same row, Postgres won't play nice and wait to reevaluate. It will violently abort one of the transactions and throw a hard serialization error.
I have seen that exact error flooding logs before. Error could not serialize access due to concurrent update. It panics developers who aren't used to it.
If you instruct your AI to use repeatable read, or if you design a system that requires it, you must also explicitly instruct the AI to wrap those database calls in an application level retry loop. The database is doing its job. It is protecting your state by rejecting the collision. It is now the application's responsibility to catch that specific error, back off for a few milliseconds, and try the entire transaction again from scratch.
Rule two in the playbook. External API boundaries. This is a massive one, and I see it crash startups all the time. The rule is simple. Never ever hold a local database transaction open while you are waiting for an external network poll to finish.
This is the ultimate architectural sin. If you begin a Postgres transaction, lock a user's row, and then call out to the Stripe API to process their credit card.
And the internet is notoriously unreliable.
Exactly. If Stripe's API is having a slow day and takes five seconds to respond, or if the network connection drops entirely, your database lock is held open. Within seconds, hundreds of other requests for that user or for related tables pile up in the database waiting for that lock to release. Your connection pool maxes out, and your entire database cluster grinds to a complete catastrophic halt. All because you were waiting on a third party server you don't even control.
So how do we fix it? We can't just not charge the credit card. We have to use atomic phases and recovery points. I really love the method outlined by Brander on his engineering blog for this.
Yes, Brander's pattern is essential reading. He describes modeling your complex API workflows as a directed acyclic graph or a DAG. And instead of one massive transaction, you break the work into strictly atomic, forward moving phases.
Walk us through a DAG implementation for a payment.
Phase one. You generate a unique item potency key, you open a quick local database transaction, insert a record saying, I am about to attempt to charge this credit card, and you commit that transaction immediately. You are now at a safe recovery point.
Crucially, the database lock is released. The database is free to handle other traffic.
Exactly. Then phase two. You make the network call to Stripe. This is a foreign state mutation. It is entirely out of your control. You sit there and await the response without holding any local locks. When Stripe finally returns the success, you enter phase three. You begin a new quick local transaction, record the Stripe success against your initial record, and finalize the order.
And because it's a directed acyclic graph, the state machine only ever moves forward. It never loops back on itself. If the node process completely crashes right in the middle of phase two while waiting for Stripe, your system isn't corrupted. Your local database has a permanent record that it intended to call Stripe, but it never recorded a success. A background sweeper job can wake up five minutes later, look at that record, and safely query Stripe to reconcile the state. Okay, rounding out the playbook. Rule three, fan out caps. We have to explicitly forbid the AI from writing raw concurrent batching without a limit. Tell the AI to use a utility library or implement a semaphore pattern to aggressively cap concurrent tasks. Say, never process more than 10 webhooks simultaneously. Rule four for UIs, mandate that an abort signal is wired to absolutely every single network fetch request. So stale data is actively assassinated by the browser before it can corrupt the screen.
Which brings us to the final and perhaps most philosophical rule of the entire playbook. Yag me. You aren't gonna need it. And we must apply this specifically to in-process memory locks.
Yes. If an AI ever tries to write a custom lock-free atomic class for you, or if it tries to sprinkle memory mutexes across your node or Python server to solve a race condition, you reject the code immediately.
An in-memory mutex only protects the memory within that one specific operating system process running on that one specific piece of hardware. It works fine on your laptop during local testing. But the second your web traffic increases and your DevOps team scales your server horizontally to run on two or three different cloud containers behind a load balancer, your in-memory lock becomes actively harmful garbage.
Because container A's mutex lock has absolutely zero visibility into container B's RAM. They don't share a brain container A locks its local variable container B locks its local variable and they both happily and concurrently execute the exact same race condition against your centralized database.
You must rely on the ACID properties of the database to manage your state. Cannot rely on the volatile, isolated memory of a stateless web server to protect distributed data. If you can solve a concurrency issue by simply pushing a where clause down into the SQL query, you do not need to invent a custom lock in your application code.
So we have covered a massive amount of ground today. If I had to distill this entire 60-minute deep dive into a mantra, concurrency is not a bug to be patched. It is not an edge case. It is a fundamental property of physics and time in computing. The hardware will actively lie to you with store buffers to gain speed. You cannot ignore it. You have to architect your entire system around it from the very first line of code.
You must assume the entire world has changed while you are falling. Design atomic phases, push all state mutations down to the database engine, and never ever hold a local lock across a network boundary.
We have successfully solved concurrency inside a single process, and we have safely delegated our state management to our single robust database engine. But what happens when your startup goes viral? What happens when one database node physically cannot handle the traffic anymore and you are forced to split your database across multiple physical machines?
That is where the local illusions end and the real distributed nightmare begins.
Exactly. Next time we are teeing up the massive finale of ARC 2 in this series. Episode S08E06, distributed consensus and split brain. We are going to explore the terrifying reality of what happens when a network practition suddenly slices your multi-node database cluster in half, and both halves genuinely believe they are the boss. I cannot wait to tear into that. But before we sign off, I want to leave you, the listener, with a final, slightly provocative thought to mull over. We have spent an hour talking about how our CPU hardware, our operating systems, and our compilers are all actively working to break sequential consistency, just to give us the illusion of speed. We spend half our careers writing locks to fight back against that speed. Is it possible that the ultimate future of programming is moving away from trying to control time and state entirely?
It is a fascinating question. If you look at pure functional programming architectures, or massive immutable event logs, you see a completely different philosophy of computing taking hold.
Right. Think about it. Instead of locking a row in a database to mutate it, which, as we've seen, causes all this massive friction and deadlock danger. What if we simply never mutate anything? What if we just append new immutable facts to a permanent history log? Fact, user added $50. Fact, user spent $10. You calculate the current state just by replaying the history of facts. If you never overwrite data, you physically never have a data race.
You completely sidestep Stikes for his mutual exclusion problem. There is no critical section to lock, because there is no shared mutable state to protect. The data is eternal.
Maybe a pending facts instead of mutating state is the only way we truly escape the muddy waters and get our clean X-ray machine back. Thank you so much for joining us on this deep dive. Go check your awaits, tear out those useless memory mutexes, write some atomic sequel, and we will see you in the distributed split brain chaos next time.