3 used MSFF PCs (i5, kingston SSDs, 24GB of ram each). All running proxmox, set up as a cluster.
1x Raspberry Pi 4 8GB. Running ubuntu.
1x Vultr 2vCPU/4GB RAM instance.
I've got a small kubernetes cluster set up using Talos with 3 controlplane / 3 workers in VMs on the proxmox nodes. The vultr node is also running Talos and attached to the same cluster. Their KubeSpan feature is pretty neat, automatic full mesh wireguard between all cluster nodes.
Traffic inside the cluster flows seamlessly between all nodes, and I can even use it as sort of a proxy server using Cilium's Egress Gateway function.
Meanwhile my Pi4 is running k3s, to host a few services needed to operate the main cluster, such as the Harbor registry operating as a cache and a zigbee2mqtt instance because I have a raspbee2 for a zigbee adapter.
The main reason I'm using K3S even on the single node Pi is because I very much like using flux to manage the deployments on the servers.
Network wise, I've got a USG-3P, one of the newer compact 16 port POE switch. And a pair of UAP-AC-LITE for APs.
Maybe one day I'll get around to switching the USG for something a little more capable. And maybe capable of doing IPS/IDS on my 500M/100M internet connection. But no idea what kind of specs I'd need for that.
Would also like a NAS but... eh.. Maybe I'll just see if i can add more storage to the proxmox nodes and expand the ceph cluster or something.
I've got a small kubernetes cluster set up using Talos with 3 controlplane / 3 workers in VMs on the proxmox nodes. The vultr node is also running Talos and attached to the same cluster. Their KubeSpan feature is pretty neat, automatic full mesh wireguard between all cluster nodes.
Traffic inside the cluster flows seamlessly between all nodes, and I can even use it as sort of a proxy server using Cilium's Egress Gateway function.
Meanwhile my Pi4 is running k3s, to host a few services needed to operate the main cluster, such as the Harbor registry operating as a cache and a zigbee2mqtt instance because I have a raspbee2 for a zigbee adapter.
The main reason I'm using K3S even on the single node Pi is because I very much like using flux to manage the deployments on the servers.
Network wise, I've got a USG-3P, one of the newer compact 16 port POE switch. And a pair of UAP-AC-LITE for APs.
Maybe one day I'll get around to switching the USG for something a little more capable. And maybe capable of doing IPS/IDS on my 500M/100M internet connection. But no idea what kind of specs I'd need for that.
Would also like a NAS but... eh.. Maybe I'll just see if i can add more storage to the proxmox nodes and expand the ceph cluster or something.