We learn of Kubernetes as having been created at Google to take advantage of all extra hardware and cpu laying around. That every bit of hardware available could be added to a cluster. (Well, Kubernetes was a rewrite of an internal project but, let’s not get distracted.)
Then we learn all about Kubernetes using tools which spin up a cluster on our desktop, then spin up a cluster out in the cloud, then something onprem.
Today it occurred to me, as I have been installing kubernetes using ms hyperv, vmware, xcp-ng, etc looking for the right vm solution, and noticing as Microsoft pieces are fading away, no longer needed and no longer the best fit… that, many may have missed what Google was saying. Why am I looking for a vm solution which can hotswap a vm from one vm server to another when Kubernetes already handles where to place a container?
Why not just install Kubernetes to my vm servers to create one large cluster? I can then spin up multiple Kubernetes clusters if I want to, using the same gitops methods I’ve been using. There is no need for a hyper-v layer, no need for vms. It is an enlightening thought.
Still, there may be a need for vms… maybe, we still need:
- an identity server (for user accounts)
- dns (which external-dns can update)
- certificate authority (which cert-manager can use)
- servers for non-containerized, legacy, apps
- dhcp (for client systems)
We could run one app in the “one cluster to rule them all”, a dns app, in addition to a controller which can spin up clusters. Hashicorp vault could be our certification authority. Our identity service provider could be almost anything, even Google. We could use keycloak locally and point to Google as the backend provider. The only dhcp that Kubernetes needs is built in.
Perhaps:
- Subnet for root cluster & dns, bare metal
- Subnet for legacy servers, vm based
- Subnet for client systems, dhcp
Next steps:
- Sidero to install Kubernetes to bare metal cluster https://www.sidero.dev/
- vcluster to deploy clusters within the bare metal cluster https://youtu.be/JqBjpvp268Y