Skip to content
masterclassBuddyTech Buzz5 fresh blogs Β· 5:00 AM IST

The Deploy That Dropped Zero Requests, Until a Node Drained Mid-Rollout

Learn to combine readiness probes, PodDisruptionBudgets and rollout tuning so deploys and node maintenance never overlap into an outage.

MCB Editorial

Tech Buzz Daily Desk Β· 10 min read Β·

New story
Narrated by Study Buddy Narration
Voiceover Engine β€’ Interactive Audio Sync
0:0010:00

Learning Objectives & Key Takeaways

0 / 4 Completed
Architecture Ready
100%
What you will learn
  1. A readiness probe only stops traffic to a bad pod, it never restarts the container for you.
  2. RollingUpdate defaults to 25 percent maxUnavailable and maxSurge, which can be too loose for small replica counts.
  3. A PodDisruptionBudget protects pods only from voluntary disruptions like node drains, not from your own bad rollout.
  4. minReadySeconds catches pods that report Ready too early but crash seconds later under real load.
  5. kubectl rollout status is the only honest signal that your new version is actually serving traffic.

The 2 PM maintenance window nobody flagged as risky

Our platform team scheduled routine node maintenance for a Tuesday afternoon, cordoning and draining three nodes in a six node cluster. At the exact same time, a different team pushed a routine image bump to their checkout service Deployment. Nobody coordinated because nobody thought a node drain and an application rollout were related events worth coordinating.

Within ninety seconds, checkout pods were disappearing faster than new ones could become Ready. The rollout's own replacement pods were landing on the same nodes being drained, and the eviction controller was removing old pods with no PodDisruptionBudget to slow it down. Checkout returned 503s for just under four minutes during a flash sale.

In the rest of this story
  1. Readiness probes: the gate that decides who gets traffic
  2. Tuning RollingUpdate so maxUnavailable actually means something
  3. PodDisruptionBudgets protect against the disruption you forgot about
  4. A complete manifest you can apply and watch end to end
  5. When a rollout goes wrong, pause before you panic
1 code example Β· 10 min read in full
Member-only story

Checking your membership…

Members read every Tech Buzz story in full, with code examples, audio and the discussion, plus every course, quiz and interview set in the Learning Dashboard.

0

Written by

MCB Editorial

Tech Buzz Daily Desk. Contributing deep-dive engineering architectures, cloud paradigms, and system design insights to Master Class Buddy.