108 lines
3.3 KiB
Markdown
108 lines
3.3 KiB
Markdown
|
# Kubernetes on Hetzner with Terraform
|
||
|
|
|
||
|
|
Provision a Kubernetes cluster on [Hetzner](https://www.hetzner.com/cloud) using Terraform and Kubespray
|
||
|
|
|
||
|
|
## Overview
|
||
|
|
|
||
|
|
The setup looks like following
|
||
|
|
|
||
|
|
```text
|
||
|
|
Kubernetes cluster
|
||
|
|
+--------------------------+
|
||
|
|
| +--------------+ |
|
||
|
|
| | +--------------+ |
|
||
|
|
| --> | | | |
|
||
|
|
| | | Master/etcd | |
|
||
|
|
| | | node(s) | |
|
||
|
|
| +-+ | |
|
||
|
|
| +--------------+ |
|
||
|
|
| ^ |
|
||
|
|
| | |
|
||
|
|
| v |
|
||
|
|
| +--------------+ |
|
||
|
|
| | +--------------+ |
|
||
|
|
| --> | | | |
|
||
|
|
| | | Worker | |
|
||
|
|
| | | node(s) | |
|
||
|
|
| +-+ | |
|
||
|
|
| +--------------+ |
|
||
|
|
+--------------------------+
|
||
|
|
```
|
||
|
|
|
||
|
|
The nodes uses a private network for node to node communication and a public interface for all external communication.
|
||
|
|
|
||
|
|
## Requirements
|
||
|
|
|
||
|
|
* Terraform 0.14.0 or newer
|
||
|
|
|
||
|
|
## Quickstart
|
||
|
|
|
||
|
|
NOTE: Assumes you are at the root of the kubespray repo.
|
||
|
|
|
||
|
|
For authentication in your cluster you can use the environment variables.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
export HCLOUD_TOKEN=api-token
|
||
|
|
```
|
||
|
|
|
||
|
|
Copy the cluster configuration file.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
CLUSTER=my-hetzner-cluster
|
||
|
|
cp -r inventory/sample inventory/$CLUSTER
|
||
|
|
cp contrib/terraform/hetzner/default.tfvars inventory/$CLUSTER/
|
||
|
|
cd inventory/$CLUSTER
|
||
|
|
```
|
||
|
|
|
||
|
|
Edit `default.tfvars` to match your requirement.
|
||
|
|
|
||
|
|
Run Terraform to create the infrastructure.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
terraform init ../../contrib/terraform/hetzner
|
||
|
|
terraform apply --var-file default.tfvars ../../contrib/terraform/hetzner/
|
||
|
|
```
|
||
|
|
|
||
|
|
You should now have a inventory file named `inventory.ini` that you can use with kubespray.
|
||
|
|
You can use the inventory file with kubespray to set up a cluster.
|
||
|
|
|
||
|
|
It is a good idea to check that you have basic SSH connectivity to the nodes. You can do that by:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
ansible -i inventory.ini -m ping all
|
||
|
|
```
|
||
|
|
|
||
|
|
You can setup Kubernetes with kubespray using the generated inventory:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
ansible-playbook -i inventory.ini ../../cluster.yml -b -v
|
||
|
|
```
|
||
|
|
|
||
|
|
## Cloud controller
|
||
|
|
|
||
|
|
For better support with the cloud you can install the [hcloud cloud controller](https://github.com/hetznercloud/hcloud-cloud-controller-manager) and [CSI driver](https://github.com/hetznercloud/csi-driver).
|
||
|
|
|
||
|
|
Please read the instructions in both repos on how to install it.
|
||
|
|
|
||
|
|
## Teardown
|
||
|
|
|
||
|
|
You can teardown your infrastructure using the following Terraform command:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
terraform destroy --var-file default.tfvars ../../contrib/terraform/hetzner
|
||
|
|
```
|
||
|
|
|
||
|
|
## Variables
|
||
|
|
|
||
|
|
* `prefix`: Prefix to add to all resources, if set to "" don't set any prefix
|
||
|
|
* `ssh_public_keys`: List of public SSH keys to install on all machines
|
||
|
|
* `zone`: The zone where to run the cluster
|
||
|
|
* `machines`: Machines to provision. Key of this object will be used as the name of the machine
|
||
|
|
* `node_type`: The role of this node *(master|worker)*
|
||
|
|
* `size`: Size of the VM
|
||
|
|
* `image`: The image to use for the VM
|
||
|
|
* `ssh_whitelist`: List of IP ranges (CIDR) that will be allowed to ssh to the nodes
|
||
|
|
* `api_server_whitelist`: List of IP ranges (CIDR) that will be allowed to connect to the API server
|
||
|
|
* `nodeport_whitelist`: List of IP ranges (CIDR) that will be allowed to connect to the kubernetes nodes on port 30000-32767 (kubernetes nodeports)
|
||
|
|
* `ingress_whitelist`: List of IP ranges (CIDR) that will be allowed to connect to kubernetes workers on port 80 and 443
|