Ivan CernjaKubernetes Orchestration: How It Works and When You Need It
Container orchestration solves real problems at scale, but most teams pay for complexity they don't need
Container orchestration is the automation layer that manages how containers are deployed, scaled, networked, and recovered across a cluster of machines. If you're running a handful of containers on a single server, you don't need it. Once you're running dozens or hundreds of containers across multiple nodes, doing this manually stops being viable.
Kubernetes (K8s) is the dominant container orchestration platform. It was designed at Google, open-sourced in 2014, and is now maintained by the CNCF. Most large-scale production systems that use containers run on Kubernetes in some form. But dominance doesn't mean it's the right choice for every team, and alternatives like managed containers (Fargate, Cloud Run) and infrastructure-from-code platforms like Encore have matured enough to handle many of the same workloads.
What container orchestration actually does
A container orchestrator takes responsibility for a set of problems that emerge when you run containers in production:
- Scheduling: deciding which machine runs which container, based on available CPU, memory, and other constraints
- Scaling: adding or removing container instances in response to traffic or resource usage
- Networking: giving containers addressable identities so they can find and communicate with each other
- Health checking and recovery: detecting when a container has crashed or become unresponsive, then restarting or replacing it
- Rolling deployments: updating containers to a new version without downtime, with the ability to roll back if something breaks
Without orchestration, you'd handle each of these manually or with a patchwork of scripts. Orchestration platforms bundle them into a single control plane.
How Kubernetes handles orchestration
Kubernetes organizes work around a few core concepts. A cluster consists of a control plane (which makes scheduling decisions) and a set of worker nodes (the machines that actually run your workloads). The smallest deployable unit is a Pod, which wraps one or more containers that share networking and storage.
You describe your desired state in YAML manifests: how many replicas of a service should run, what resources they need, how they should be exposed to traffic. Kubernetes continuously compares the actual state of the cluster to this desired state and takes action to close any gap. If a pod crashes, Kubernetes reschedules it. If you update a deployment, it rolls out new pods incrementally.
This declarative model is Kubernetes' core strength. You tell it what you want, not how to get there. The system figures out placement, failover, and scaling on its own.
On top of this foundation, Kubernetes has a large ecosystem of add-ons: service meshes for traffic management, custom resource definitions (CRDs) for extending the API, operators for managing stateful workloads, and ingress controllers for routing external traffic. The flexibility is real, but so is the surface area you need to understand.
When orchestration makes sense
Kubernetes orchestration earns its complexity in specific situations:
High scale with variable load. If your services handle traffic that spikes 10x during peak hours, auto-scaling container replicas across a pool of nodes is exactly what orchestration was designed for.
Many independently deployed services. Organizations running 20, 50, or 100+ microservices benefit from a unified control plane that handles service discovery, load balancing, and deployment rollouts across all of them.
Multi-cloud or hybrid deployments. Kubernetes provides a consistent abstraction layer whether you're running on AWS, GCP, Azure, or your own hardware. Teams that need workload portability across environments get real value from this.
Strict compliance or isolation requirements. Kubernetes' namespace and network policy features let teams enforce isolation boundaries between workloads, which matters in regulated industries.
When it doesn't
The uncomfortable truth about Kubernetes is that the majority of teams using it don't fall into any of those categories. They're running a modest number of services at moderate scale, and they've adopted Kubernetes because it felt like the professional, industry-standard choice.
The costs are concrete. Kubernetes requires dedicated expertise: someone who understands YAML manifests, Helm charts, networking policies, RBAC, and debugging across pods, services, and ingress layers. Managed offerings like EKS, GKE, and AKS reduce the operational burden but don't eliminate it. You still own the configuration, monitoring, and upgrade lifecycle. And the compute overhead of running a Kubernetes cluster (the control plane, system pods, monitoring agents) is non-trivial for smaller workloads.
For a team of 3-15 engineers shipping a product with a few backend services, the time spent managing Kubernetes infrastructure is time not spent building features. The orchestration layer becomes the thing that needs the most care and attention, rather than the application itself.
Kubernetes orchestration alternatives
The alternative isn't "go back to running everything on a single VM." Several approaches give you production-grade deployments without the operational weight of Kubernetes:
Managed container platforms like AWS Fargate and Google Cloud Run let you deploy containers without managing clusters or nodes. You push a container image, define scaling rules, and the platform handles placement and infrastructure. You lose some of Kubernetes' flexibility, but you also lose the YAML.
Platform-as-a-Service (PaaS) offerings abstract further, managing both infrastructure and deployment pipelines. The tradeoff is less control over the underlying compute.
Infrastructure from code takes a different approach entirely. Instead of writing application code and then separately configuring orchestration, networking, and infrastructure, the infrastructure is derived from the application code itself.
Encore is an open-source backend framework for TypeScript and Go that works this way. You define services, databases, pub/sub topics, and cron jobs using the framework's type-safe APIs. Encore's tooling parses your application code, understands what infrastructure is needed, and Encore Cloud provisions it in your own AWS or GCP account. You don't write Kubernetes manifests, Terraform configuration, or Dockerfiles. The infrastructure topology is a function of your application architecture, not a separate layer you manage in parallel.
This matters because for most teams, the goal was never "run Kubernetes." It was "deploy reliable backend services to the cloud." Orchestration is one way to get there. Deriving infrastructure directly from code is another, one that sidesteps the operational complexity rather than managing it.
Choosing the right approach
The decision comes down to what you're actually optimizing for. If you need fine-grained control over container placement, custom networking topologies, or multi-cloud portability, Kubernetes orchestration is purpose-built for that. If you need to ship backend services to production reliably and your team would rather spend time on application logic than infrastructure management, the alternatives have matured enough to be the better choice.
Want to jump straight to a running app? Clone this starter and deploy it to your own cloud.