Specs
Hardware
The following hardware components are used:
- 5x Raspberry Pi 4 Model B (1x 4 GB, 4x 8 GB)
- 5x Raspberry Pi PoE+ Hat
- 5x SanDisk Extreme microSDXC (1x 64 GB, 4x 128 GB)
- 1x UniFi Switch Lite 16 PoE
- 1x GeekPi 6 Layer Cluster Case
- 1x Netcup VPS 1000 G10 (6 vCores, 8 GB RAM)
- 1x Synology DiskStation DS920+ (2x 3 GB & 2x 4 GB)
Note
The fans of the official PoE hats appear to be very noisy. Therefore I adjusted their temperature thresholds to 70°C and 80°C:
Software
- Raspberry Pi OS Lite (64-bit) / Ubuntu Server 22.04 LTS
- Ansible from RedHat
- K3s originally from Rancher, now a CNCF project
- Flux originally from Weaveworks, now a CNCF project
- SOPS from Mozilla
- GitHub Actions
Networking
Physical
The Raspberry Pis are all physically connected to a UniFi PoE switch for power and data. The actual communication between all cluster nodes is happening through the Tailscale network.
Virtual
Inside the cluster the main parts of the networking setup are CoreDNS and Traefik.
The following IPs and CIDRs are configured inside K3s:
Name | IP / CIDR |
---|---|
Kubernetes Pod IPs | 10.42.0.0/16 |
Kubernetes Service IPs | 10.43.0.0/16 |
Kubernetes Cluster DNS IP | 10.43.0.10 |
DNS
Internal
All services with a UI running in my cluster have a dedicated subdomain configured. Those domains are managed by Blocky which is configured as the main DNS server in my home network. It also performs network-wide ad blocking based on some block lists.
External
Services that need to be reachable externally have dedicated public DNS records. These are configured using Terraform.
Example of a DNS Terraform resource
data "cloudflare_zone" "zone_public" {
name = "my-domain.com" # target DNS zone (1)
}
resource "cloudflare_record" "public" {
zone_id = data.cloudflare_zone.zone_public.id
type = "A"
name = "*" # record name (2)
value = "1.2.3.4" # record value (3)
}
- This block references the target DNS zone
- Record name (a.k.a subdomain) to be configured
- Record value to be configured
Dynamic
Since I don't have a static IP address allocated from my ISP, my IP address can change at any point in time. To bypass this circumstance I configured a Kubernetes CronJob
object which performs an DynDNS update for my domain at Cloudflare. The configuration can be found in the dyndns
section.
Storage
Currently I'm not using anything special as a storage solution for my cluster. The built-in local-path-provisioner
of K3s is fulfilling any PersistentVolumeClaim
objects but I'm planning to switch to Longhorn.
Hostnames
Every node has configured a hostname which follows a particular naming scheme:
As you may have guessed, the planets are taken from the Star Wars universe. Currently the following names are in use:
coruscant
kashyyyk
alderaan
dathomir
mustafar
jakku
ryloth
ilum