Support Calico advertisement of MetalLB LoadBalancer IPs (#7593)
* add initial MetalLB docs * metallb allow disabling the deployment of the metallb speaker * calico>=3.18 allow using calico to advertise service loadbalancer IPs * Document the use of MetalLB and Calico * clean MetalLB docspull/7612/head
parent
afbabebfd5
commit
14cf3e138b
|
@ -219,6 +219,8 @@ See also [Network checker](docs/netcheck.md).
|
|||
|
||||
- [nginx](https://kubernetes.github.io/ingress-nginx): the NGINX Ingress Controller.
|
||||
|
||||
- [metallb](docs/metallb.md): the MetalLB bare-metal service LoadBalancer provider.
|
||||
|
||||
## Community docs and resources
|
||||
|
||||
- [kubernetes.io/docs/setup/production-environment/tools/kubespray/](https://kubernetes.io/docs/setup/production-environment/tools/kubespray/)
|
||||
|
|
|
@ -0,0 +1,81 @@
|
|||
# MetalLB
|
||||
|
||||
MetalLB hooks into your Kubernetes cluster, and provides a network load-balancer implementation.
|
||||
It allows you to create Kubernetes services of type "LoadBalancer" in clusters that don't run on a cloud provider, and thus cannot simply hook into 3rd party products to provide load-balancers.
|
||||
The default operationg mode of MetalLB is in ["Layer2"](https://metallb.universe.tf/concepts/layer2/) but it can also operate in ["BGP"](https://metallb.universe.tf/concepts/bgp/) mode.
|
||||
|
||||
## Install
|
||||
|
||||
You have to explicitly enable the MetalLB extension and set an IP address range from which to allocate LoadBalancer IPs.
|
||||
|
||||
```yaml
|
||||
metallb_enabled: true
|
||||
metallb_speaker_enabled: true
|
||||
metallb_ip_range:
|
||||
- 10.5.0.0/16
|
||||
```
|
||||
|
||||
By default only the MetalLB BGP speaker is allowed to run on control plane nodes. If you have a single node cluster or a cluster where control plane are also worker nodes you may need to enable tolerations for the MetalLB controller:
|
||||
|
||||
```yaml
|
||||
metallb_controller_tolerations:
|
||||
- key: "node-role.kubernetes.io/master"
|
||||
operator: "Equal"
|
||||
value: ""
|
||||
effect: "NoSchedule"
|
||||
- key: "node-role.kubernetes.io/control-plane"
|
||||
operator: "Equal"
|
||||
value: ""
|
||||
effect: "NoSchedule"
|
||||
```
|
||||
|
||||
## BGP Mode
|
||||
|
||||
When operating in BGP Mode MetalLB needs to have defined upstream peers:
|
||||
|
||||
```yaml
|
||||
metallb_protocol: bgp
|
||||
metallb_ip_range:
|
||||
- 10.5.0.0/16
|
||||
metallb_peers:
|
||||
- peer_address: 192.0.2.1
|
||||
peer_asn: 64512
|
||||
my_asn: 4200000000
|
||||
- peer_address: 192.0.2.2
|
||||
peer_asn: 64513
|
||||
my_asn: 4200000000
|
||||
```
|
||||
|
||||
When using calico >= 3.18 you can replace MetalLB speaker by calico Service LoadBalancer IP advertisement.
|
||||
See [calico service IPs advertisement documentation](https://docs.projectcalico.org/archive/v3.18/networking/advertise-service-ips#advertise-service-load-balancer-ip-addresses).
|
||||
In this scenarion you should disable the MetalLB speaker and configure the `calico_advertise_service_loadbalancer_ips` to match your `metallb_ip_range`
|
||||
|
||||
```yaml
|
||||
metallb_speaker_enabled: false
|
||||
metallb_ip_range:
|
||||
- 10.5.0.0/16
|
||||
calico_advertise_service_loadbalancer_ips: "{{ metallb_ip_range }}"
|
||||
```
|
||||
|
||||
If you have additional loadbalancer IP pool in `metallb_additional_address_pools`, ensure to add them to the list.
|
||||
|
||||
```yaml
|
||||
metallb_speaker_enabled: false
|
||||
metallb_ip_range:
|
||||
- 10.5.0.0/16
|
||||
metallb_additional_address_pools:
|
||||
kube_service_pool_1:
|
||||
ip_range:
|
||||
- 10.6.0.0/16
|
||||
protocol: "bgp"
|
||||
auto_assign: false
|
||||
kube_service_pool_2:
|
||||
ip_range:
|
||||
- 10.10.0.0/16
|
||||
protocol: "bgp"
|
||||
auto_assign: false
|
||||
calico_advertise_service_loadbalancer_ips:
|
||||
- 10.5.0.0/16
|
||||
- 10.6.0.0/16
|
||||
- 10.10.0.0/16
|
||||
```
|
|
@ -132,6 +132,7 @@ cert_manager_enabled: false
|
|||
|
||||
# MetalLB deployment
|
||||
metallb_enabled: false
|
||||
metallb_speaker_enabled: true
|
||||
# metallb_ip_range:
|
||||
# - "10.5.0.50-10.5.0.99"
|
||||
# metallb_speaker_nodeselector:
|
||||
|
|
|
@ -50,6 +50,11 @@
|
|||
# - x.x.x.x/24
|
||||
# - y.y.y.y/32
|
||||
|
||||
# Adveritse Service LoadBalancer IPs
|
||||
# calico_advertise_service_loadbalancer_ips:
|
||||
# - x.x.x.x/24
|
||||
# - y.y.y.y/16
|
||||
|
||||
# Choose data store type for calico: "etcd" or "kdd" (kubernetes datastore)
|
||||
# calico_datastore: "kdd"
|
||||
|
||||
|
|
|
@ -1,17 +0,0 @@
|
|||
# Deploy MetalLB into Kubespray/Kubernetes
|
||||
|
||||
MetalLB hooks into your Kubernetes cluster, and provides a network load-balancer implementation.
|
||||
In short, it allows you to create Kubernetes services of type "LoadBalancer" in clusters that
|
||||
don't run on a cloud provider, and thus cannot simply hook into paid products to provide load-balancers.
|
||||
This addon aims to automate [MetalLB in layer 2 mode](https://metallb.universe.tf/concepts/layer2/)
|
||||
or [MetalLB in BGP mode](https://metallb.universe.tf/concepts/bgp/).
|
||||
It deploys MetalLB into Kubernetes and sets up a layer 2 or BGP load-balancer.
|
||||
|
||||
## Install
|
||||
|
||||
In the default, MetalLB is not deployed into your Kubernetes cluster.
|
||||
You can override the defaults by copying the contents of roles/kubernetes-apps/metallb/defaults/main.yml
|
||||
to somewhere in inventory/mycluster/group_vars such as inventory/mycluster/groups_vars/k8s_cluster/addons.yml
|
||||
and updating metallb_enabled option to `true`.
|
||||
In addition you need to update metallb_ip_range option on the addons.yml at least for suiting your network
|
||||
environment, because MetalLB allocates external IP addresses from this metallb_ip_range option.
|
|
@ -6,6 +6,7 @@ metallb_port: "7472"
|
|||
metallb_limits_cpu: "100m"
|
||||
metallb_limits_mem: "100Mi"
|
||||
metallb_peers: []
|
||||
metallb_speaker_enabled: true
|
||||
metallb_speaker_nodeselector: {}
|
||||
metallb_controller_nodeselector: {}
|
||||
metallb_speaker_tolerations:
|
||||
|
|
|
@ -47,6 +47,7 @@ spec:
|
|||
- secret
|
||||
- emptyDir
|
||||
---
|
||||
{% if metallb_speaker_enabled %}
|
||||
apiVersion: policy/v1beta1
|
||||
kind: PodSecurityPolicy
|
||||
metadata:
|
||||
|
@ -85,6 +86,7 @@ spec:
|
|||
- configMap
|
||||
- secret
|
||||
- emptyDir
|
||||
{% endif %}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
|
@ -94,6 +96,7 @@ metadata:
|
|||
name: controller
|
||||
namespace: metallb-system
|
||||
---
|
||||
{% if metallb_speaker_enabled %}
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
|
@ -101,6 +104,7 @@ metadata:
|
|||
app: metallb
|
||||
name: speaker
|
||||
namespace: metallb-system
|
||||
{% endif %}
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
|
@ -140,6 +144,7 @@ rules:
|
|||
verbs:
|
||||
- use
|
||||
---
|
||||
{% if metallb_speaker_enabled %}
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
|
@ -172,6 +177,7 @@ rules:
|
|||
- podsecuritypolicies
|
||||
verbs:
|
||||
- use
|
||||
{% endif %}
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
|
@ -220,6 +226,7 @@ subjects:
|
|||
name: controller
|
||||
namespace: metallb-system
|
||||
---
|
||||
{% if metallb_speaker_enabled %}
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
|
@ -234,6 +241,7 @@ subjects:
|
|||
- kind: ServiceAccount
|
||||
name: speaker
|
||||
namespace: metallb-system
|
||||
{% endif %}
|
||||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
|
@ -267,6 +275,7 @@ subjects:
|
|||
- kind: ServiceAccount
|
||||
name: speaker
|
||||
---
|
||||
{% if metallb_speaker_enabled %}
|
||||
apiVersion: apps/v1
|
||||
kind: DaemonSet
|
||||
metadata:
|
||||
|
@ -353,6 +362,7 @@ spec:
|
|||
tolerations:
|
||||
{{ metallb_speaker_tolerations | to_nice_yaml(indent=2) | indent(width=8) }}
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
|
|
|
@ -29,6 +29,9 @@ global_as_num: "64512"
|
|||
# Advertise Service External IPs
|
||||
calico_advertise_service_external_ips: []
|
||||
|
||||
# Adveritse Service LoadBalancer IPs
|
||||
calico_advertise_service_loadbalancer_ips: []
|
||||
|
||||
# Limits for apps
|
||||
calico_node_memory_limit: 500M
|
||||
calico_node_cpu_limit: 300m
|
||||
|
|
|
@ -188,6 +188,12 @@
|
|||
with_items: "{{ calico_advertise_service_external_ips }}"
|
||||
run_once: yes
|
||||
|
||||
- name: Populate Service LoadBalancer IPs
|
||||
set_fact:
|
||||
_service_loadbalancer_ips: "{{ _service_loadbalancer_ips|default([]) + [ {'cidr': item} ] }}"
|
||||
with_items: "{{ calico_advertise_service_loadbalancer_ips }}"
|
||||
run_once: yes
|
||||
|
||||
- name: "Determine nodeToNodeMesh needed state"
|
||||
set_fact:
|
||||
nodeToNodeMeshEnabled: "false"
|
||||
|
@ -213,6 +219,7 @@
|
|||
{% if not calico_no_global_as_num|default(false) %}"asNumber": {{ global_as_num }},{% endif %}
|
||||
"nodeToNodeMeshEnabled": {{ nodeToNodeMeshEnabled|default('true') }} ,
|
||||
{% if calico_advertise_cluster_ips|default(false) %}"serviceClusterIPs": [{"cidr": "{{ kube_service_addresses }}" }],{% endif %}
|
||||
{% if calico_version is version('v3.18.0', '>') and calico_advertise_service_loadbalancer_ips|length > 0 %}"serviceLoadBalancerIPs": {{ _service_loadbalancer_ips }},{% endif %}
|
||||
"serviceExternalIPs": {{ _service_external_ips|default([]) }} }}
|
||||
changed_when: false
|
||||
when:
|
||||
|
|
Loading…
Reference in New Issue