Your Kubernetes Project Will Succeed or Fail Before the First Deployment. Here’s Why.
What running Kubernetes actually looks like past the initial setup
Kubernetes documentation is not the problem. There is plenty of it, and most of it is accurate. The gap is between getting a cluster running and keeping it running eighteen months later, when the engineers who built it have moved on, the team that inherited it never got a proper handoff, and every production incident starts with forty minutes of figuring out how the system was originally designed.
Getting a cluster running on EKS or GKE takes days. That part is well-supported. The problems come later and they are operational, not technical: resource contention between workloads that nobody planned for, probe misconfigurations causing cascading restarts under load, node upgrades that have never been run because the cluster has never needed one.
The most common situation we walk into at a US mid-market company: the cluster was built by one or two engineers who understood Kubernetes well, those engineers left or moved to other work, and the team now responsible for it has no documented picture of how it was designed or where it breaks. The architecture knowledge left with the people who had it. Every incident is a discovery exercise.
This is not a Kubernetes problem specifically. It’s an operational handoff problem that Kubernetes makes worse, because the failure surface is large. A single misconfigured resource — a namespace without quotas, a workload without a disruption budget, a liveness probe threshold that’s too tight — can take down workloads across the cluster in ways that take significant time to trace back to their source.
Two things most adoption plans get wrong
The first is the diagnostic model. A VM failing is straightforward: the process died, the disk filled, the network dropped. A pod failing in Kubernetes can fail for a dozen distinct reasons — image pull errors, resource limit breaches, liveness probe misconfiguration, node pressure, eviction policies triggered under load, a malformed ConfigMap that only surfaces at startup. Each failure mode has its own diagnostic path and its own remediation. Engineers who haven’t built that mental model before going to production spend their first months learning it under pressure, during incidents, with users affected.
The second is ongoing maintenance. Adoption plans account for deployment benefits. They rarely account for what the cluster requires to stay healthy:
– Node upgrades, which need to be sequenced carefully to avoid evicting workloads during the process
– Certificate rotation for node and control plane certificates, which managed services like EKS, GKE, and AKS handle automatically. The risk area is custom certificates — webhook TLS, ingress certificates, anything managed outside the cluster by cert-manager or similar tools — which require active monitoring and a tested rotation process
– etcd backup and restore on self-managed clusters, where most teams discover they have no tested restore procedure until they actually need one. On managed services this is handled by the provider, but on self-managed clusters it is the team’s responsibility and consistently underprepared
– RBAC management as headcount and application count grow, which becomes a real ongoing task if not designed for from the start
– Network policy maintenance in multi-team environments, where one misconfigured policy can affect workloads across namespaces
In a US mid-market company where the platform team is two or three people, all of this competes with everything else on their plate. The teams that manage it well made explicit decisions about platform ownership before the cluster existed. The teams that struggle made a deployment decision without a corresponding operations decision.
The one question your adoption plan needs to answer: who owns the cluster operationally — covering upgrades, incident response at off-hours, capacity management, and RBAC as the team grows? A Kubernetes adoption plan without a clear answer to that question is incomplete.
How we scope Kubernetes engagements with mid-market clients
Before recommending Kubernetes to a client, we ask about the team and the organization, not the workloads.
Platform team size today, and a realistic picture of what it looks like in eighteen months. Existing Kubernetes experience on the team. The specific problems driving the adoption conversation, and whether Kubernetes actually addresses them. Greenfield infrastructure or a migration from a system carrying live traffic.
When the answers point to a small team with limited Kubernetes experience and a handful of services, we recommend starting with a managed service — EKS, GKE, or AKS — rather than anything self-managed. The operational gap between a managed control plane and a self-managed one is significant, and for most mid-market workloads the tradeoffs are worth making. The team gets to focus on workload management, networking, and observability instead of also managing control plane availability and etcd.
We also push for observability investment before any production workload goes in. A cluster without metrics collection, log aggregation, and alerting configured from day one will generate incidents the team can’t diagnose efficiently. Which specific tools matter less than when you set them up — before you need them, not after the first incident makes the gap obvious.
The third pre-production requirement we push on: resource requests and limits on every workload. Teams skip this because nothing breaks in a lightly loaded cluster without it. Under real load, a cluster without properly configured requests and limits develops resource contention that affects every tenant and resists attribution to any specific workload.
Four configurations that separate stable clusters from fragile ones
Across production Kubernetes engagements at US companies ranging from 200 to several thousand employees, four practices show up consistently in the clusters that hold up.
Namespace-per-team with enforced resource quotas. Two teams sharing a cluster can get away without namespace boundaries. Eight teams cannot. A workload that consumes all available cluster memory will evict workloads in other namespaces without warning, and the team whose workload caused it won’t know it happened. Setting up namespace boundaries and resource quotas before the second team joins the cluster is far cheaper than retrofitting them after the first contention incident.
GitOps for cluster state. Using ArgoCD or Flux to manage both application deployments and cluster configuration means every change to the cluster is version-controlled, reviewed, and reversible. The failure mode it prevents is configuration drift — the gap between what the cluster is actually doing and what anyone believes it’s doing, which opens up through manual changes during incidents, undocumented fixes, and one-off modifications nobody wrote down. Drift doesn’t announce itself. It shows up as unexplained behavior months later.
Probe discipline across all workloads. Misconfigured liveness and readiness probes cause more production incidents in Kubernetes environments than any other single configuration issue. The pattern that works: readiness probes that verify the service is ready to handle traffic, not just that the process started; liveness probes with conservative thresholds, meaning three or more consecutive failures before triggering a restart; startup probes for services with variable initialization time, to prevent the liveness probe from killing a legitimately slow-starting service. None of this is obscure. It is absent or misconfigured in the majority of clusters we inherit from other teams.
PodDisruptionBudgets on every production workload. A PodDisruptionBudget tells Kubernetes the minimum number of replicas that must stay available during a voluntary disruption, such as draining a node for an upgrade. Without one, a rolling node upgrade can evict every replica of a workload simultaneously, causing a full outage. With one, the upgrade process respects the availability requirement and sequences evictions accordingly. This is one of the highest-impact, lowest-effort configurations in a production cluster. It is missing from most clusters we encounter.
On autoscaling: HPA, VPA, and KEDA HPA is typically the first thing teams want to configure. We add it last: after resource requests and limits are set on all workloads, after the observability stack is running, and after we have baseline load data. HPA without those inputs scales erratically. Two things worth knowing alongside HPA: VPA (Vertical Pod Autoscaler) automatically adjusts CPU and memory requests based on actual usage, which is often more relevant than horizontal scaling for stateful workloads or services with variable resource profiles. KEDA (Kubernetes Event-Driven Autoscaling) extends HPA to scale on external metrics — queue depth, Kafka consumer lag, database connections — and is worth evaluating if your scaling triggers aren’t HTTP traffic. |
When Kubernetes is the wrong answer for your company
For a US engineering team with no hard requirement for workload isolation between teams, no need for custom networking or ingress control, and no stateful workloads that need fine-grained scheduling, the operational investment Kubernetes requires may not pay off for two years or more. The number of services is a weak signal. The stronger signals are: do multiple teams need to share infrastructure without stepping on each other, do you need pod-level network policies, and does your team have the capacity to own a platform on top of shipping product. If the answer to all three is no, ECS Fargate, AWS App Runner, and Azure Container Apps deliver the core deployment and scalability benefits without the overhead.
The useful question to ask before committing: what specific problem is Kubernetes solving for your engineering team right now? If the answer is complex workloads with different scaling profiles, multiple teams that need isolation from each other, and an organization prepared to staff and own the platform, Kubernetes is the right tool. If the answer is cleaner deployments and some horizontal scaling, there are paths to that outcome that don’t require a dedicated platform investment.
The US mid-market companies that get the most value from Kubernetes share a few characteristics: they had explicit platform ownership before the cluster was built, they set up observability before the first production workload went in, and they treated the adoption decision as an operational commitment, not a technical upgrade. The companies that struggle made the technology decision without making the organizational one alongside it.
The cluster is the easy part. The operating model is the hard part. And the operating model question has to be answered before the cluster exists, not after the first incident that exposes the gap.
Other container orchestration paths worth knowing
Kubernetes is not the only way to run containerized workloads in production. Depending on your team size, traffic patterns, and how much platform ownership you want to take on, several alternatives cover the same core needs with meaningfully different operational profiles.
Amazon ECS with Fargate. ECS is AWS’s native container orchestration service. With Fargate, you define the task — the container, the CPU, the memory — and AWS handles the underlying infrastructure. No nodes to manage, no control plane, no cluster upgrade windows. Autoscaling is handled through Application Auto Scaling policies tied to CloudWatch metrics. ECS integrates directly with IAM, ALB, and AWS service discovery, which makes it operationally simpler for teams already running their stack on AWS. The tradeoff is portability: ECS is AWS-specific, and workloads built around its primitives don’t move easily to other environments.
AWS App Runner. App Runner is a further step toward fully managed. You point it at a container image or a source repository, define the CPU and memory, and App Runner handles deployment, scaling, load balancing, and TLS. There is no cluster concept at all. It is best suited for stateless HTTP services with predictable interfaces — internal APIs, web backends, lightweight microservices. Teams running early-stage products or internal tooling often find App Runner sufficient for a year or two before the workload complexity justifies more control.
Azure Container Apps. Microsoft’s answer to the managed container platform space, built on top of Kubernetes but exposing none of its complexity to the operator. Container Apps handles scaling, including scale-to-zero for workloads that don’t need to run continuously, and integrates with KEDA for event-driven scaling from sources like Azure Service Bus, Kafka, or HTTP traffic. For teams already in the Azure ecosystem, it offers a credible path to running microservices and background jobs without platform engineering investment. It also supports Dapr natively, which is useful for distributed application patterns like pub/sub, service invocation, and state management.
Google Cloud Run. Cloud Run runs containers on demand, scaling from zero to handle requests and back to zero when idle. Billing is per request and per CPU/memory usage during execution, which makes it cost-efficient for workloads with irregular or spiky traffic. It supports both HTTP and gRPC services, and integrates with Cloud Pub/Sub and Eventarc for event-driven patterns. Teams building APIs or processing pipelines on GCP often use Cloud Run before the scale or complexity of their workloads justifies GKE.
Nomad by HashiCorp. Nomad is a general-purpose workload orchestrator that handles containers, VMs, and standalone binaries in a single scheduler. The core architecture is simpler than Kubernetes — fewer moving parts, a shorter learning curve, and more predictable failure modes. In production you still run server nodes and client nodes, and you’ll want Consul for service discovery and Vault for secrets, so it’s not operationally trivial at scale. The advantage over Kubernetes is that the complexity ceiling is lower: you get a lot of the scheduling and orchestration capability without the full Kubernetes operational surface. Companies that run mixed workloads, containerized services alongside non-containerized jobs and batch processes, often find Nomad a better fit. Nomad has a smaller community than Kubernetes and fewer managed hosting options, which is a real consideration if self-managed infrastructure is not something your team wants to own.
Red Hat OpenShift. OpenShift is Kubernetes with a significant operational layer on top: built-in CI/CD via Tekton, a developer console, integrated image registry, stricter security defaults out of the box, and Red Hat’s support contract. It runs on-premises or on AWS, Azure, and GCP. For US enterprises in regulated industries where the procurement and support relationship with a vendor matters — financial services, healthcare, government contractors — OpenShift is a serious option. The tradeoff is cost and opinion: OpenShift makes specific choices about how things work, and working against those choices is expensive. Teams that need flexibility in their toolchain often find it constraining.
Docker Swarm. Swarm is Docker’s built-in orchestration layer and has been in maintenance mode since around 2019. It still runs, and some teams still use it for small internal workloads. We mention it because it comes up, not because we recommend it for new deployments in 2026. If you’re running Swarm today, the question is not whether to move away from it but when.
How to choose If you’re on AWS and want the least operational overhead, ECS Fargate or App Runner cover most mid-market use cases. If you’re on Azure, Container Apps is worth evaluating before committing to AKS. If you have mixed workload types — containers, batch jobs, legacy services — Nomad is worth a serious look. If you’re in a regulated industry and need a vendor support relationship, OpenShift is the serious alternative to vanilla Kubernetes. Kubernetes makes sense when you need fine-grained control over scheduling, networking, and resource isolation across multiple teams, and when you have the platform capacity to run it properly. |
Two things most adoption plans get wrong
If your organization runs on VMware
One path that doesn’t show up in most Kubernetes comparisons: vSphere Kubernetes Service (VKS), which is Kubernetes built directly into VMware Cloud Foundation. If your organization is already on VCF, or is evaluating it now — particularly relevant given Broadcom’s move of over 90% of its top VMware customers to VCF subscriptions — VKS is worth understanding before you make a separate Kubernetes platform decision.
VKS runs inside VCF’s management layer. Your vSphere administrators manage VMs and containers through the same control plane they already know, without standing up a separate Kubernetes cluster or learning an entirely different operational model. For organizations that have invested in VMware tooling and want to adopt containers without fragmenting their operations team, this is a more practical path than running a parallel Kubernetes environment.
The catch is application readiness. Most organizations evaluating VCF migration have application portfolios that were never built for containers — monolithic apps tightly coupled to VM-level dependencies that will fail if you try to lift them straight onto VKS. The infrastructure migration is the straightforward part. Getting the applications ready to actually benefit from running on Kubernetes inside VCF is where most engagements stall.
Our approach with these clients: application portfolio assessment before any infrastructure work begins. We classify each application for containerization, lift-and-shift, or phased re-architecture, map the inter-system dependencies that cause migration failures when they go unresolved, and sequence the migration around application readiness rather than arbitrary timelines. The refactoring happens in parallel with production — existing systems stay live while we build and validate containerized versions in a staging VCF environment.
The healthcare provider we worked with had 40+ clinical applications across 100+ sites. Three critical inter-system dependencies would have caused outages during migration if we hadn’t caught them in the assessment phase. After re-factoring the priority clinical applications into containerized microservices on VKS, new clinic deployment time dropped from weeks to hours.
Working through a VCF migration or evaluating VKS If you’re mapping out a VMware Cloud Foundation migration and want to understand what application re-factoring actually involves before committing to a timeline, we’re happy to walk through your specific portfolio. The assessment is where the real decisions get made. |
If you’re evaluating Kubernetes for your engineering organization, running into operational friction after adoption, or working through a VCF migration — we’re happy to work through the specifics with you.
Bohdan Shuliar
Share:
