Kubernetes Across Multiple Continents: Highly Available k3s and k8s in Data Centers Worldwide

Kubernetes is the standard when applications need to stay resilient and scale automatically. The obvious idea of stretching a single Kubernetes cluster across servers in the US, Europe and Asia fails because of one detail, though: etcd, the database in which Kubernetes stores its entire state. This article shows how Kubernetes across multiple continents works anyway, and in a way that tolerates the failure of an entire continent: with one cluster per region, shared delivery through GitOps and a GeoDNS service that sends users to the nearest healthy region.

We build the architecture with k3s, the lightweight, fully certified Kubernetes distribution that installs in minutes, and at the end we show what changes with classic Kubernetes set up via kubeadm. The fundamentals of availability, quorum and failover across multiple locations are covered in detail in the article Docker Swarm across three continents; this article focuses on what is different with Kubernetes.

One cluster across all continents or one cluster per region?

For Kubernetes across multiple continents, one cluster per region is the right architecture, not a single cluster spanning all continents. Each cluster runs on its own, all of them are rolled out from the same Git repository, and a GeoDNS service with health checks distributes the users. If one region fails, the others take over without any shared cluster state having to be coordinated across the ocean.

Why etcd cannot handle long distances

Kubernetes stores all of its state, every configuration and every change in etcd, a key-value store with Raft consensus. Every write needs to be acknowledged by a majority of all etcd members. By default, etcd works with a heartbeat interval of 100 milliseconds and an election timeout of one second; between Europe, North America and Asia, packet latency ranges from 80 to 250 milliseconds. You can raise these timings, but then every write in the cluster becomes slow, from scheduling a pod to saving a Secret. The k3s documentation is clear on this point: the embedded etcd is not supported in clusters spread across multiple networks, and all servers should be at the same location. Kubernetes itself is also designed for a cluster to cover multiple zones within one region, not multiple continents.

The three patterns compared

The “one cluster per region” approach has a second, often underestimated advantage: a fault in the control plane, a bad change to the cluster or an upgrade that goes wrong only ever hits one region. Users in the other regions notice nothing.

k3s or k8s?

Both are real Kubernetes with the same API, the same manifests and the same tools. The difference lies in how they are put together and how much effort they take:

For the one-cluster-per-region architecture, we recommend k3s: running three clusters is only pleasant if each one of them is simple. If you already have kubeadm experience, you can adopt the architecture unchanged; the kubeadm section further down shows the differences.

The architecture: three regions, three clusters, one entry point

Starting setup and expansion

The starting setup consists of three servers, one each in the US, Europe and Asia, each running as its own single-node cluster. If a server fails, its region fails, and the GeoDNS sends that region's users to the neighboring region. This stage is already designed to handle the failure of an entire continent. In the expansion stage, each region gets three servers at the same location; then each region can also survive the failure of one server on its own, without users having to be redirected.

Which KernelHost locations are suitable

KernelHost operates servers in the maincubes data center in Frankfurt am Main and offers virtual servers at additional locations in Europe, North America and Asia-Pacific, including three locations in the US, Canada, London, Strasbourg, Warsaw, Helsinki, Singapore, Japan, Sydney and Mumbai. The complete list is on the Server Locations page. The example in this article uses Frankfurt am Main for Europe, the US East Coast for North America and Singapore for Asia.

Guide: Kubernetes with k3s in three regions

The example uses three servers running Debian 12 or 13: k3s-eu, k3s-us and k3s-asia. Public addresses come from the documentation network 203.0.113.0/24, and the domain is example.com. Replace both with your own values.

Step 1: Provision servers in three regions

Order three servers in three regions, each with at least 2 vCPUs and 4 GB of RAM, so there is room for your application alongside k3s. Apply the basic hardening from the checklist for new root servers and assign meaningful hostnames. etcd benefits noticeably from fast SSDs, and the NVMe storage in KernelHost servers meets that need.

Step 2: Install k3s

On each of the three servers, install k3s with a single command. This turns every server into a complete Kubernetes cluster with one node:

curl -sfL https://get.k3s.io | sh -
kubectl get nodes

After about a minute, kubectl get nodes reports the node as Ready. Included are Traefik as the ingress controller, Flannel for networking and a provisioner for local volumes.

Step 3: Expand to three servers per region

If a region itself is to become highly available, place three servers at the same location. The first server starts the embedded etcd, and the other two join with a shared token. etcd requires an odd number of servers; with three servers, the region can cope with the failure of one server:

curl -sfL https://get.k3s.io | K3S_TOKEN=SECRET_TOKEN sh -s - server \
    --cluster-init \
    --tls-san=api.eu.example.com

curl -sfL https://get.k3s.io | K3S_TOKEN=SECRET_TOKEN sh -s - server \
    --server https://203.0.113.21:6443 \
    --tls-san=api.eu.example.com

All servers in a region need the same settings for network ranges and features. Between them, ports 2379 to 2380/TCP for etcd, 6443/TCP for the API, 10250/TCP for the kubelet and 8472/UDP for the Flannel network must be open; toward the outside, they stay blocked. The token is a secret: anyone who knows it can add their own servers to the cluster.

Step 4: Set up access to all three clusters

k3s stores the access credentials in /etc/rancher/k3s/k3s.yaml. Copy each cluster's file to your workstation, replace 127.0.0.1 in it with the server's address and name the contexts after the region:

kubectl config rename-context default eu
kubectl config use-context eu
kubectl --context us get nodes

The API on port 6443 is the most powerful way into the cluster. Allow it in the firewall only from your own address or a VPN, never for the entire internet. The file k3s.yaml contains an administrator certificate and needs to be kept just as carefully as a root password.

Step 5: GitOps with Flux in every cluster

To make sure all three regions run the same application in the same version, the desired state lives in a Git repository, and every cluster runs Flux, which establishes that state on its own. Each cluster therefore fetches its configuration itself; there is no central deployment server that could fail. A proven repository layout:

apps/
  web/            shared application manifests
clusters/
  eu/             settings and version for Europe
  us/             settings and version for North America
  asia/           settings and version for Asia

flux bootstrap git --url=ssh://git@git.example.com/infra/fleet.git --branch=main --path=clusters/eu

Run the same command with --path=clusters/us and --path=clusters/asia in the respective context. Because each region has its own directory, you can roll out a new version in one region first and only afterwards in the others.

Step 6: Define the application to withstand failures

Within a region, three things make sure the application survives maintenance and server failures: multiple replicas spread across different nodes, probes that detect unhealthy pods, and a disruption budget that prevents maintenance from removing all pods at the same time:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: ScheduleAnyway
          labelSelector:
            matchLabels:
              app: web
      containers:
        - name: web
          image: registry.example.com/web:1.0
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /health
              port: 8080
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            periodSeconds: 10
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web
spec:
  minAvailable: 1
  selector:
    matchLabels:
      app: web

The readiness probe takes a pod out of rotation as long as it does not respond, and the liveness probe restarts it when it hangs. In a single-node cluster, spreading across multiple nodes has no effect yet; it kicks in automatically as soon as the region has three servers.

Step 7: Ingress and certificates

The bundled Traefik handles incoming traffic. Because all three regions serve the same domain, obtain the TLS certificates with cert-manager via the DNS challenge: it works in every region, regardless of where the DNS record currently points. The HTTP challenge, by contrast, fails in every region the record is not currently pointing to. If you work without a Kubernetes ingress, you will find the basics in the article Setting up nginx as a reverse proxy.

Step 8: Set up GeoDNS with failover

A DNS service with geo-routing and health checks sends users from Europe to Frankfurt am Main, from the Americas to the US East Coast and from Asia to Singapore, and checks every 30 to 60 seconds whether each region's entry point responds. If a region fails, the service stops handing out its address. Set the TTL of the records to 60 seconds; a switchover is then usually complete after one to two minutes. If you want to manage the records from within the cluster, you can use external-dns for that.

Step 9: Roll out region by region and test a failure

Enter a new version in one region's directory first, watch that region for a few minutes and then apply the version to the others. That way, a bug that no check catches reaches one region at most. To rehearse the failure of a region, stop k3s on its server and check whether the GeoDNS removes the region from its answers and the neighboring region carries the load:

systemctl stop k3s
systemctl start k3s

In a region with three servers, also rehearse the maintenance of a single server. kubectl drain moves the pods while respecting the budget from step 6, and kubectl uncordon puts the server back into service:

kubectl drain k3s-eu-2 --ignore-daemonsets --delete-emptydir-data
kubectl uncordon k3s-eu-2

Data across regions

Kubernetes distributes pods, not data. A PersistentVolume from the bundled provisioner lives on exactly one node, and out of the box there is no shared data storage between the clusters at all. For anything involving data, the same rules therefore apply as for any cluster spanning multiple locations:

• Databases handle their own replication. A proven setup is a primary instance in one region with replicas in the others; database operators such as CloudNativePG for PostgreSQL support such replicas across cluster boundaries as well. Across continents, replication runs asynchronously, so in an emergency the last few seconds of writes may be missing. For lossless writes worldwide, there are multi-region databases such as CockroachDB or YugabyteDB.
• Files and uploads belong in S3-compatible object storage with replication to a second region.
• Sessions live in a replicated database or a replicated cache, or the application uses signed tokens.
• Backups remain mandatory, because replication spreads errors just as it spreads good data. Velero is well suited for Kubernetes objects and volumes; for the basics, see the backup strategy for servers.

Kubernetes with kubeadm instead of k3s

With kubeadm, the architecture stays the same: one cluster per region, GitOps, GeoDNS. What differs is how each cluster is built. You install a container runtime such as containerd plus kubeadm, kubelet and kubectl on all nodes, place a load balancer or a virtual address in front of the region's API, for example with kube-vip, and initialize the first control plane node:

kubeadm init --control-plane-endpoint "api.eu.example.com:6443" --upload-certs

The output contains two join commands: one with --control-plane --certificate-key for the two additional control plane nodes and one for workers. After that, you install a network plugin such as Calico or Cilium and an ingress controller, which k3s already includes. The extra effort pays off if you need to pick individual components deliberately or stay close to the upstream version.

Why KernelHost for Kubernetes across multiple continents

You will find an overview of all cloud plans and a cost comparison with the major cloud providers on the Cloud Server Hosting page.

Common mistakes and how to avoid them

• etcd stretched across continents. The result is slow writes and unstable leader elections, and with k3s and embedded etcd it is not supported. Solution: one cluster per region.
• Two servers in one region. etcd needs a majority, so two servers cannot tolerate a single failure. Solution: one or three.
• API open to the entire internet. Port 6443 is the master key to the cluster. Solution: only from your own address or via VPN.
• Central deployment server. If it fails, nothing can be rolled out anymore. Solution: Flux in every cluster, pulling from Git on its own.
• Updating all regions at once. A bug then hits every user. Solution: region by region via the directories in the repository.
• Certificates via HTTP challenge. In regions the DNS record is not currently pointing to, renewal fails. Solution: DNS challenge.
• Database on a local volume without replication. If the node fails, the data is unreachable. Solution: replication through a database operator.
• No probes and no budget. Unhealthy pods keep receiving traffic, and maintenance takes all pods offline at the same time. Solution: readiness and liveness probes plus a PodDisruptionBudget.

In short

• For Kubernetes across multiple continents, one cluster per region is the right choice; a single cluster across all continents fails because of etcd latency.
• The starting setup consists of three servers, one each in the US, Europe and Asia; even this stage survives the failure of an entire continent.
• In the expansion stage, each region gets three servers at the same location, so each region also survives the failure of individual servers.
• Flux in every cluster rolls out the application from the same Git repository, region by region.
• GeoDNS with health checks and a short TTL sends users to the nearest healthy region, and certificates come via the DNS challenge.
• Kubernetes distributes pods, not data: databases need their own replication, and backups remain mandatory.
• For this architecture, k3s is usually a better choice than kubeadm, because three simple clusters are easier to operate than three complex ones.