feat: add kubelet systemd service hardening option (#9194)
* feat: add kubelet systemd service hardening option * refactor: move variable name to kubelet_secure_addresses Co-authored-by: Cristian Calin <6627509+cristicalin@users.noreply.github.com> * docs: add diagram about kubelet_secure_addresses variable Co-authored-by: Cristian Calin <6627509+cristicalin@users.noreply.github.com>pull/9227/head
parent
220f149299
commit
acb6f243fd
|
@ -85,6 +85,13 @@ kubelet_streaming_connection_idle_timeout: "5m"
|
|||
kubelet_make_iptables_util_chains: true
|
||||
kubelet_feature_gates: ["RotateKubeletServerCertificate=true","SeccompDefault=true"]
|
||||
kubelet_seccomp_default: true
|
||||
kubelet_systemd_hardening: true
|
||||
# In case you have multiple interfaces in your
|
||||
# control plane nodes and you want to specify the right
|
||||
# IP addresses, kubelet_secure_addresses allows you
|
||||
# to specify the IP from which the kubelet
|
||||
# will receive the packets.
|
||||
kubelet_secure_addresses: "192.168.10.110 192.168.10.111 192.168.10.112"
|
||||
|
||||
# additional configurations
|
||||
kube_owner: root
|
||||
|
@ -103,6 +110,8 @@ Let's take a deep look to the resultant **kubernetes** configuration:
|
|||
* The `encryption-provider-config` provide encryption at rest. This means that the `kube-apiserver` encrypt data that is going to be stored before they reach `etcd`. So the data is completely unreadable from `etcd` (in case an attacker is able to exploit this).
|
||||
* The `rotateCertificates` in `KubeletConfiguration` is set to `true` along with `serverTLSBootstrap`. This could be used in alternative to `tlsCertFile` and `tlsPrivateKeyFile` parameters. Additionally it automatically generates certificates by itself, but you need to manually approve them or at least using an operator to do this (for more details, please take a look here: <https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/>).
|
||||
* If you are installing **kubernetes** in an AppArmor-based OS (eg. Debian/Ubuntu) you can enable the `AppArmor` feature gate uncommenting the lines with the comment `# AppArmor-based OS` on top.
|
||||
* The `kubelet_systemd_hardening`, both with `kubelet_secure_addresses` setup a minimal firewall on the system. To better understand how these variables work, here's an explanatory image:
|
||||
![kubelet hardening](img/kubelet-hardening.png)
|
||||
|
||||
Once you have the file properly filled, you can run the **Ansible** command to start the installation:
|
||||
|
||||
|
|
Binary file not shown.
After Width: | Height: | Size: 1.5 MiB |
21
docs/vars.md
21
docs/vars.md
|
@ -175,25 +175,46 @@ Stack](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/dns-stack.m
|
|||
|
||||
* *docker_options* - Commonly used to set
|
||||
``--insecure-registry=myregistry.mydomain:5000``
|
||||
|
||||
* *docker_plugins* - This list can be used to define [Docker plugins](https://docs.docker.com/engine/extend/) to install.
|
||||
|
||||
* *containerd_default_runtime* - If defined, changes the default Containerd runtime used by the Kubernetes CRI plugin.
|
||||
|
||||
* *containerd_additional_runtimes* - Sets the additional Containerd runtimes used by the Kubernetes CRI plugin.
|
||||
[Default config](https://github.com/kubernetes-sigs/kubespray/blob/master/roles/container-engine/containerd/defaults/main.yml) can be overriden in inventory vars.
|
||||
|
||||
* *http_proxy/https_proxy/no_proxy/no_proxy_exclude_workers/additional_no_proxy* - Proxy variables for deploying behind a
|
||||
proxy. Note that no_proxy defaults to all internal cluster IPs and hostnames
|
||||
that correspond to each node.
|
||||
|
||||
* *kubelet_cgroup_driver* - Allows manual override of the cgroup-driver option for Kubelet.
|
||||
By default autodetection is used to match container manager configuration.
|
||||
`systemd` is the preferred driver for `containerd` though it can have issues with `cgroups v1` and `kata-containers` in which case you may want to change to `cgroupfs`.
|
||||
|
||||
* *kubelet_rotate_certificates* - Auto rotate the kubelet client certificates by requesting new certificates
|
||||
from the kube-apiserver when the certificate expiration approaches.
|
||||
|
||||
* *kubelet_rotate_server_certificates* - Auto rotate the kubelet server certificates by requesting new certificates
|
||||
from the kube-apiserver when the certificate expiration approaches.
|
||||
**Note** that server certificates are **not** approved automatically. Approve them manually
|
||||
(`kubectl get csr`, `kubectl certificate approve`) or implement custom approving controller like
|
||||
[kubelet-rubber-stamp](https://github.com/kontena/kubelet-rubber-stamp).
|
||||
|
||||
* *kubelet_streaming_connection_idle_timeout* - Set the maximum time a streaming connection can be idle before the connection is automatically closed.
|
||||
|
||||
* *kubelet_make_iptables_util_chains* - If `true`, causes the kubelet ensures a set of `iptables` rules are present on host.
|
||||
|
||||
* *kubelet_systemd_hardening* - If `true`, provides kubelet systemd service with security features for isolation.
|
||||
|
||||
**N.B.** To enable this feature, ensure you are using the **`cgroup v2`** on your system. Check it out with command: `sudo ls -l /sys/fs/cgroup/*.slice`. If directory does not exists, enable this with the following guide: [enable cgroup v2](https://rootlesscontaine.rs/getting-started/common/cgroup2/#enabling-cgroup-v2).
|
||||
|
||||
* *kubelet_secure_addresses* - By default *kubelet_systemd_hardening* set the **control plane** `ansible_host` IPs as the `kubelet_secure_addresses`. In case you have multiple interfaces in your control plane nodes and the `kube-apiserver` is not bound to the default interface, you can override them with this variable.
|
||||
Example:
|
||||
|
||||
The **control plane** node may have 2 interfaces with the following IP addresses: `eth0:10.0.0.110`, `eth1:192.168.1.110`.
|
||||
|
||||
By default the `kubelet_secure_addresses` is set with the `10.0.0.110` the ansible control host uses `eth0` to connect to the machine. In case you want to use `eth1` as the outgoing interface on which `kube-apiserver` connects to the `kubelet`s, you should override the variable in this way: `kubelet_secure_addresses: "192.168.1.110"`.
|
||||
|
||||
* *node_labels* - Labels applied to nodes via kubelet --node-labels parameter.
|
||||
For example, labels can be set in the inventory as variables or more widely in group_vars.
|
||||
*node_labels* can only be defined as a dict:
|
||||
|
|
|
@ -22,6 +22,12 @@ kubelet_kubelet_cgroups_cgroupfs: "/system.slice/kubelet.service"
|
|||
### fail with swap on (default true)
|
||||
kubelet_fail_swap_on: true
|
||||
|
||||
# Set systemd service hardening features
|
||||
kubelet_systemd_hardening: false
|
||||
|
||||
# List of secure IPs for kubelet
|
||||
kubelet_secure_addresses: "{{ groups['kube_control_plane'] | map('extract', hostvars, ['ansible_host']) | join(' ') }}"
|
||||
|
||||
# Reserve this space for kube resources
|
||||
kube_memory_reserved: 256Mi
|
||||
kube_cpu_reserved: 100m
|
||||
|
|
|
@ -24,6 +24,11 @@ ExecStart={{ bin_dir }}/kubelet \
|
|||
$KUBELET_CLOUDPROVIDER
|
||||
Restart=always
|
||||
RestartSec=10s
|
||||
{% if kubelet_systemd_hardening %}
|
||||
# Hardening setup
|
||||
IPAddressDeny=any
|
||||
IPAddressAllow={{ kubelet_secure_addresses }}
|
||||
{% endif %}
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
Loading…
Reference in New Issue