Web Architecture Overview
Transcript
Made automatically from the video's sound, so it may contain mistakes. Click a line to play from there.
Right, let's get straight into it. Welcome to episode zero of Web Architecture and Infrastructure Components for Zoomy Labs. In this explainer, we're going to construct an end-to-end architectural map, literally tracing a high concurrency HTTPS request from a client socket directly to a persistent database and all the way back.
And since this is designed strictly for software engineers, DevOps practitioners and solution architects, we are completely bypassing the beginner analogies today. We're focusing purely on exact OSI layers, latency budgets, and strict failure boundaries.
But before we start architecting our ideal distributed system, we really have to face the catastrophic reality of scale. We need to ask, what exactly happens when a single box web server hits its breaking point?
Just imagine trying to process 500 plus transactions per second, whilst absorbing volumetric attacks, regional latency spikes, and you know, the inevitable deploy downtime. Without distributed infrastructure, a single server simply collapses under the sheer mathematical impossibility of that load.
And what that collapse looks like is a classic cascading failure, something deeply documented in Google's site reliability engineering practices. It all kicks off with CPU exhaustion. In languages like Java, this triggers what's known as a garbage collection death spiral.
Basically, less CPU means slower requests, which consume more RAM, which triggers more garbage collection, which in turn leaves you with even less CPU. This leads straight to thread starvation and dropped RPC deadlines. When clients don't get their responses, they trigger aggressive retry storms, amplifying the load until the entire server just enters a total crash loop.
So, to actually solve that failure, our agenda today will systematically build out our architecture. We'll start by looking at scale failure under load, then we'll trace the end-to-end request lifecycle, clearly define our component boundaries and contracts, and finally wrap up with macro infrastructure trade-offs.
Let's dive right into section one: the end-to-end request lifecycle, tracing the hop by hop journey. We're going to map out exactly how a single user request traverses our distributed infrastructure.
The journey actually begins way out at the edge network. Before a single packet of application data is even exchanged, a recursive resolver uses Anycast BGP to route the client to the nearest CDN point of presence. It's at this edge server where TLS 1.3 termination happens, handling the cryptographic handshake, which is immediately followed by an HTTP cache lookup to serve up static assets with absolute minimal latency.
Now, AnyCast DNS is absolutely brilliant because it essentially eliminates the single point of failure for resolution, whilst guaranteeing the lowest possible latency to the edge. Rather than relying on one specific geographic server, Anycast allows multiple global authoritative name servers to share the exact same IP address.
The network itself naturally routes the user's query to the closest available node, purely based on the shortest network round trip time. But what if the CDN cache misses? Well, the request has to travel deeper into our architecture.
First, it passes through the web application firewall, or WAF, for layer 7 inspection, stripping out malicious payloads like cross-site scripting or SQL injections. From there, it hits the Origin Shield reverse proxy to consolidate traffic before it finally arrives at our data center's network load balancer.
Here's a crucial detail for hyperscale systems. To avoid the physical limitations of layer two direct server response, where your load balancers and backends have to physically sit in the exact same local broadcast domain.
Modern hyperscale balancers use GRE encapsulation. By taking the forwarded packet and encapsulating it into another IP packet, they can route traffic to compute backends anywhere at all, completely independent of the broadcast domain.
It's an absolute game changer for horizontal scalability. Finally, the load balancer hands the request off to a containerized compute pod. But crucially, the application doesn't just hammer the database right away. It first checks an in-memory side cache, something like Redis, to see if the data's already there.
If it's a cache miss, the compute pod queries the primary or replica persistent database to fulfil the request, and then updates the cache on its way back out. Component boundaries and contracts, disambiguating the infrastructure stack. With our end-to-end map in place, we really need to zoom in and establish the strict architectural contracts of these components, because engineers frequently conflate them.
Let's take the reverse proxy, for example. While a CDN is distributed globally to reduce latency for the end user, the strict contract of a reverse proxy is to sit right at the origin boundary to protect your back-end resources.
It acts as a necessary intermediary to handle cryptographic offloading, hide origin IP addresses from direct exposure, and strictly prevent any direct client-to-server communication. Then there's load balancing, where the distinction is strictly grounded in the OSI model.
Layer 4 network load balancers operate at the transport layer, so IP, TCP, and UDP. They use consistent hashing for blazing fast, stateless routing with incredibly low latency. Layer 7 application load balancers, on the other hand, inspect HTTP headers and terminate TLS.
Because they actually read the payload, they incur a latency penalty. But that penalty buys you highly intelligent, content-aware routing decisions. And when it comes to caching, your architectural boundary literally dictates how you write your code.
An inline cache, like Varnish, provides a uniform API. The application is completely unaware of the cache's existence. It just asks for data. The downside, it introduces a hard availability dependency.
A side cache, like Redis, is just a generic object store. This forces your application code to explicitly check the cache before hitting the data source, manually handle the misses, and update the cache afterwards. Alright, let's step back for our final section.
Macro infrastructure trade-offs. Balancing systemic tensions. Because let's be real, distributed systems are never about perfect solutions. They're entirely about deliberate, quantitative engineering compromises.
Have a look at this number. 1.1 seconds. That might not sound like a lot, but it is the time it takes to process a request when a thread takes a mere 100 milliseconds to execute, but his stock sitting in a queue 10 times the size of the thread pool.
This perfectly highlights why queue management is so critical. If you just rely on deep FIFO cues during a traffic burst, you will effortlessly blow past your client's RPC deadlines. You'll end up processing work that the client has already completely abandoned, which is a massive waste of your back-end compute.
These architectural factors reveal the inherent systemic tensions in infrastructure design. It's a balancing act. You simply cannot have deep layer 7 WAF inspection without paying a compute and latency tax, just as you can't have multi AZ redundancy without paying for cross AZ data transfer and dealing with the tail latency of synchronous replication.
And of course, pushing data out to edge caching brilliantly protects your origin load, but it inherently sacrifices data freshness. We also have to talk about caching, because it is incredibly dangerous if misused. AWS engineers frequently warn against what they call cache addiction.
This happens when the cache inadvertently goes from being a helpful addition to an absolutely critical part of the service's ability to even operate. If your back end database has been scaled way down because the cache is doing all the heavy lifting, a cold start or a cash fleet failure will result in a massive thundering herd that instantly browns out your database.
So, how do we prevent that exact cascading failure we opened with? Well, you've got to engineer resiliency tactics right into the platform. You must enforce request coalescing to ensure that only one pending request goes to the back end for an uncached resource.
You have to inject jitter, which is randomized exponential backoff, into your client retries, so network ripples don't synchronize into massive spikes. You need soft versus hard TTLs to serve stale data during outages for graceful degradation.
And you absolutely must ensure strict RPC deadline propagation across every single layer of the stack. It is the ultimate test of your infrastructure's resilience. Go dig into your metrics, review those queue limits, and ensure your system is built to fail gracefully.
Thanks for joining me on this end to end architectural journey.