Merge pull request #11633 from tico88612/feat/remove-in-tree-cloud-provider

Cleanup: remove in-tree cloud provider support
pull/11689/head
Kubernetes Prow Robot 2024-11-05 09:13:30 +00:00 committed by GitHub
commit 107c3cc6f4
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
46 changed files with 45 additions and 963 deletions

View File

@ -123,12 +123,9 @@ vagrant up
- [Fedora CoreOS bootstrap](docs/operating_systems/fcos.md)
- [openSUSE setup](docs/operating_systems/opensuse.md)
- [Downloaded artifacts](docs/advanced/downloads.md)
- [Cloud providers](docs/cloud_providers/cloud.md)
- [OpenStack](docs/cloud_providers/openstack.md)
- [AWS](docs/cloud_providers/aws.md)
- [Azure](docs/cloud_providers/azure.md)
- [vSphere](docs/cloud_providers/vsphere.md)
- [Equinix Metal](docs/cloud_providers/equinix-metal.md)
- [OpenStack](docs/cloud_controllers/openstack.md)
- [vSphere](docs/cloud_controllerss/vsphere.md)
- [Large deployments](docs/operations/large-deployments.md)
- [Adding/replacing a node](docs/operations/nodes.md)
- [Upgrades basics](docs/operations/upgrades.md)

View File

@ -620,7 +620,7 @@ Edit `inventory/$CLUSTER/group_vars/k8s_cluster/k8s_cluster.yml`:
- Set variable **kube_network_plugin** to your desired networking plugin.
- **flannel** works out-of-the-box
- **calico** requires [configuring OpenStack Neutron ports](/docs/cloud_providers/openstack.md) to allow service and pod subnets
- **calico** requires [configuring OpenStack Neutron ports](/docs/cloud_controllers/openstack.md) to allow service and pod subnets
```yml
# Choose network plugin (calico, weave or flannel)

5
docs/_sidebar.md generated
View File

@ -15,13 +15,14 @@
* [Ansible](/docs/ansible/ansible.md)
* [Ansible Collection](/docs/ansible/ansible_collection.md)
* [Vars](/docs/ansible/vars.md)
* Cloud Controllers
* [Openstack](/docs/cloud_controllers/openstack.md)
* [Vsphere](/docs/cloud_controllers/vsphere.md)
* Cloud Providers
* [Aws](/docs/cloud_providers/aws.md)
* [Azure](/docs/cloud_providers/azure.md)
* [Cloud](/docs/cloud_providers/cloud.md)
* [Equinix-metal](/docs/cloud_providers/equinix-metal.md)
* [Openstack](/docs/cloud_providers/openstack.md)
* [Vsphere](/docs/cloud_providers/vsphere.md)
* CNI
* [Calico](/docs/CNI/calico.md)
* [Cilium](/docs/CNI/cilium.md)

View File

@ -1,5 +1,7 @@
# GCP Load Balancers for type=LoadBalacer of Kubernetes Services
> **Removed**: Since v1.31 (the Kubespray counterpart is v2.27), Kubernetes no longer supports `cloud_provider`. (except external cloud provider)
Google Cloud Platform can be used for creation of Kubernetes Service Load Balancer.
This feature is able to deliver by adding parameters to `kube-controller-manager` and `kubelet`. You need specify:

View File

@ -104,8 +104,7 @@ following default cluster parameters:
* *enable_coredns_k8s_endpoint_pod_names* - If enabled, it configures endpoint_pod_names option for kubernetes plugin.
on the CoreDNS service.
* *cloud_provider* - Enable extra Kubelet option if operating inside GCE or
OpenStack (default is unset)
* *cloud_provider* - The provider for cloud services. (default is unset, Set to `external` for running with an external cloud provider)
* *kube_feature_gates* - A list of key=value pairs that describe feature gates for
alpha/experimental Kubernetes features. (defaults is `[]`).

View File

@ -1,5 +1,7 @@
# AWS
> **Removed**: Since v1.31 (the Kubespray counterpart is v2.27), Kubernetes no longer supports `cloud_provider`. (except external cloud provider)
To deploy kubespray on [AWS](https://aws.amazon.com/) uncomment the `cloud_provider` option in `group_vars/all.yml` and set it to `'aws'`. Refer to the [Kubespray Configuration](#kubespray-configuration) for customizing the provider.
Prior to creating your instances, you **must** ensure that you have created IAM roles and policies for both "kubernetes-master" and "kubernetes-node". You can find the IAM policies [here](https://github.com/kubernetes-sigs/kubespray/tree/master/contrib/aws_iam/). See the [IAM Documentation](https://aws.amazon.com/documentation/iam/) if guidance is needed on how to set these up. When you bring your instances online, associate them with the respective IAM role. Nodes that are only to be used for Etcd do not need a role.

View File

@ -1,5 +1,7 @@
# Azure
> **Removed**: Since v1.31 (the Kubespray counterpart is v2.27), Kubernetes no longer supports `cloud_provider`. (except external cloud provider)
To deploy Kubernetes on [Azure](https://azure.microsoft.com) uncomment the `cloud_provider` option in `group_vars/all/all.yml` and set it to `'azure'`.
All your instances are required to run in a resource group and a routing table has to be attached to the subnet your instances are in.

View File

@ -1,5 +1,7 @@
# Cloud providers
> **Removed**: Since v1.31 (the Kubespray counterpart is v2.27), Kubernetes no longer supports `cloud_provider`. (except external cloud provider)
## Provisioning
You can deploy instances in your cloud environment in several ways. Examples include Terraform, Ansible (ec2 and gce modules), and manual creation.

View File

@ -42,9 +42,7 @@ loadbalancer_apiserver_healthcheck_port: 8081
## There are some changes specific to the cloud providers
## for instance we need to encapsulate packets with some network plugins
## If set the possible values are either 'gce', 'aws', 'azure', 'openstack', 'vsphere', 'oci', or 'external'
## When openstack is used make sure to source in the openstack credentials
## like you would do when using openstack-client before starting the playbook.
## If set the possible values only 'external' after K8s v1.31.
# cloud_provider:
## When cloud_provider is set to 'external', you can set the cloud controller to deploy

View File

@ -140,11 +140,7 @@ kube_proxy_nodeport_addresses: >-
{%- endif -%}
# If non-empty, will use this string as identification instead of the actual hostname
# kube_override_hostname: >-
# {%- if cloud_provider is defined and cloud_provider in ['aws'] -%}
# {%- else -%}
# {{ inventory_hostname }}
# {%- endif -%}
# kube_override_hostname: {{ inventory_hostname }}
## Encrypting Secret Data at Rest
kube_encrypt_secret_data: false

View File

@ -1,6 +0,0 @@
---
oci_security_list_management: All
oci_use_instance_principals: false
oci_cloud_controller_version: 0.7.0
oci_cloud_controller_pull_source: iad.ocir.io/oracle/cloud-provider-oci

View File

@ -1,67 +0,0 @@
---
- name: "OCI Cloud Controller | Credentials Check | oci_private_key"
fail:
msg: "oci_private_key is missing"
when:
- not oci_use_instance_principals
- oci_private_key is not defined or not oci_private_key
- name: "OCI Cloud Controller | Credentials Check | oci_region_id"
fail:
msg: "oci_region_id is missing"
when:
- not oci_use_instance_principals
- oci_region_id is not defined or not oci_region_id
- name: "OCI Cloud Controller | Credentials Check | oci_tenancy_id"
fail:
msg: "oci_tenancy_id is missing"
when:
- not oci_use_instance_principals
- oci_tenancy_id is not defined or not oci_tenancy_id
- name: "OCI Cloud Controller | Credentials Check | oci_user_id"
fail:
msg: "oci_user_id is missing"
when:
- not oci_use_instance_principals
- oci_user_id is not defined or not oci_user_id
- name: "OCI Cloud Controller | Credentials Check | oci_user_fingerprint"
fail:
msg: "oci_user_fingerprint is missing"
when:
- not oci_use_instance_principals
- oci_user_fingerprint is not defined or not oci_user_fingerprint
- name: "OCI Cloud Controller | Credentials Check | oci_compartment_id"
fail:
msg: "oci_compartment_id is missing. This is the compartment in which the cluster resides"
when:
- oci_compartment_id is not defined or not oci_compartment_id
- name: "OCI Cloud Controller | Credentials Check | oci_vnc_id"
fail:
msg: "oci_vnc_id is missing. This is the Virtual Cloud Network in which the cluster resides"
when:
- oci_vnc_id is not defined or not oci_vnc_id
- name: "OCI Cloud Controller | Credentials Check | oci_subnet1_id"
fail:
msg: "oci_subnet1_id is missingg. This is the first subnet to which loadbalancers will be added"
when:
- oci_subnet1_id is not defined or not oci_subnet1_id
- name: "OCI Cloud Controller | Credentials Check | oci_subnet2_id"
fail:
msg: "oci_subnet2_id is missing. Two subnets are required for load balancer high availability"
when:
- oci_cloud_controller_version is version_compare('0.7.0', '<')
- oci_subnet2_id is not defined or not oci_subnet2_id
- name: "OCI Cloud Controller | Credentials Check | oci_security_list_management"
fail:
msg: "oci_security_list_management is missing, or not defined correctly. Valid options are (All, Frontend, None)."
when:
- oci_security_list_management is not defined or oci_security_list_management not in ["All", "Frontend", "None"]

View File

@ -1,35 +0,0 @@
---
- name: OCI Cloud Controller | Check Oracle Cloud credentials
import_tasks: credentials-check.yml
- name: "OCI Cloud Controller | Generate Cloud Provider Configuration"
template:
src: controller-manager-config.yml.j2
dest: "{{ kube_config_dir }}/controller-manager-config.yml"
mode: "0644"
when: inventory_hostname == groups['kube_control_plane'][0]
- name: "OCI Cloud Controller | Slurp Configuration"
slurp:
src: "{{ kube_config_dir }}/controller-manager-config.yml"
register: controller_manager_config
- name: "OCI Cloud Controller | Encode Configuration"
set_fact:
controller_manager_config_base64: "{{ controller_manager_config.content }}"
when: inventory_hostname == groups['kube_control_plane'][0]
- name: "OCI Cloud Controller | Generate Manifests"
template:
src: oci-cloud-provider.yml.j2
dest: "{{ kube_config_dir }}/oci-cloud-provider.yml"
mode: "0644"
when: inventory_hostname == groups['kube_control_plane'][0]
- name: "OCI Cloud Controller | Apply Manifests"
kube:
kubectl: "{{ bin_dir }}/kubectl"
filename: "{{ kube_config_dir }}/oci-cloud-provider.yml"
state: latest
when: inventory_hostname == groups['kube_control_plane'][0]

View File

@ -1,89 +0,0 @@
{% macro private_key() %}{{ oci_private_key }}{% endmacro %}
{% if oci_use_instance_principals %}
# (https://docs.us-phoenix-1.oraclecloud.com/Content/Identity/Tasks/callingservicesfrominstances.htm).
# Ensure you have setup the following OCI policies and your kubernetes nodes are running within them
# allow dynamic-group [your dynamic group name] to read instance-family in compartment [your compartment name]
# allow dynamic-group [your dynamic group name] to use virtual-network-family in compartment [your compartment name]
# allow dynamic-group [your dynamic group name] to manage load-balancers in compartment [your compartment name]
useInstancePrincipals: true
{% else %}
useInstancePrincipals: false
{% endif %}
auth:
{% if oci_use_instance_principals %}
# This key is put here too for backwards compatibility
useInstancePrincipals: true
{% else %}
useInstancePrincipals: false
region: {{ oci_region_id }}
tenancy: {{ oci_tenancy_id }}
user: {{ oci_user_id }}
key: |
{{ oci_private_key }}
{% if oci_private_key_passphrase is defined %}
passphrase: {{ oci_private_key_passphrase }}
{% endif %}
fingerprint: {{ oci_user_fingerprint }}
{% endif %}
# compartment configures Compartment within which the cluster resides.
compartment: {{ oci_compartment_id }}
# vcn configures the Virtual Cloud Network (VCN) within which the cluster resides.
vcn: {{ oci_vnc_id }}
loadBalancer:
# subnet1 configures one of two subnets to which load balancers will be added.
# OCI load balancers require two subnets to ensure high availability.
subnet1: {{ oci_subnet1_id }}
{% if oci_subnet2_id is defined %}
# subnet2 configures the second of two subnets to which load balancers will be
# added. OCI load balancers require two subnets to ensure high availability.
subnet2: {{ oci_subnet2_id }}
{% endif %}
# SecurityListManagementMode configures how security lists are managed by the CCM.
# "All" (default): Manage all required security list rules for load balancer services.
# "Frontend": Manage only security list rules for ingress to the load
# balancer. Requires that the user has setup a rule that
# allows inbound traffic to the appropriate ports for kube
# proxy health port, node port ranges, and health check port ranges.
# E.g. 10.82.0.0/16 30000-32000.
# "None": Disables all security list management. Requires that the
# user has setup a rule that allows inbound traffic to the
# appropriate ports for kube proxy health port, node port
# ranges, and health check port ranges. E.g. 10.82.0.0/16 30000-32000.
# Additionally requires the user to mange rules to allow
# inbound traffic to load balancers.
securityListManagementMode: {{ oci_security_list_management }}
{% if oci_security_lists is defined and oci_security_lists | length > 0 %}
# Optional specification of which security lists to modify per subnet. This does not apply if security list management is off.
securityLists:
{% for subnet_ocid, list_ocid in oci_security_lists.items() %}
{{ subnet_ocid }}: {{ list_ocid }}
{% endfor %}
{% endif %}
{% if oci_rate_limit is defined and oci_rate_limit | length > 0 %}
# Optional rate limit controls for accessing OCI API
rateLimiter:
{% if oci_rate_limit.rate_limit_qps_read %}
rateLimitQPSRead: {{ oci_rate_limit.rate_limit_qps_read }}
{% endif %}
{% if oci_rate_limit.rate_limit_qps_write %}
rateLimitQPSWrite: {{ oci_rate_limit.rate_limit_qps_write }}
{% endif %}
{% if oci_rate_limit.rate_limit_bucket_read %}
rateLimitBucketRead: {{ oci_rate_limit.rate_limit_bucket_read }}
{% endif %}
{% if oci_rate_limit.rate_limit_bucket_write %}
rateLimitBucketWrite: {{ oci_rate_limit.rate_limit_bucket_write }}
{% endif %}
{% endif %}

View File

@ -1,69 +0,0 @@
apiVersion: v1
data:
cloud-provider.yaml: {{ controller_manager_config_base64 }}
kind: Secret
metadata:
name: oci-cloud-controller-manager
namespace: kube-system
type: Opaque
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: oci-cloud-controller-manager
namespace: kube-system
labels:
k8s-app: oci-cloud-controller-manager
spec:
selector:
matchLabels:
component: oci-cloud-controller-manager
tier: control-plane
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
component: oci-cloud-controller-manager
tier: control-plane
spec:
{% if oci_cloud_controller_pull_secret is defined %}
imagePullSecrets:
- name: {{ oci_cloud_controller_pull_secret }}
{% endif %}
serviceAccountName: cloud-controller-manager
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
nodeSelector:
node-role.kubernetes.io/control-plane: ""
tolerations:
- key: node.cloudprovider.kubernetes.io/uninitialized
value: "true"
effect: NoSchedule
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
volumes:
- name: cfg
secret:
secretName: oci-cloud-controller-manager
- name: kubernetes
hostPath:
path: /etc/kubernetes
containers:
- name: oci-cloud-controller-manager
image: {{ oci_cloud_controller_pull_source }}:{{ oci_cloud_controller_version }}
command: ["/usr/local/bin/oci-cloud-controller-manager"]
args:
- --cloud-config=/etc/oci/cloud-provider.yaml
- --cloud-provider=oci
- --leader-elect-resource-lock=configmaps
- -v=2
volumeMounts:
- name: cfg
mountPath: /etc/oci
readOnly: true
- name: kubernetes
mountPath: /etc/kubernetes
readOnly: true

View File

@ -1,124 +0,0 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: cloud-controller-manager
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: system:cloud-controller-manager
rules:
- apiGroups:
- ""
resources:
- nodes
verbs:
- '*'
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
- apiGroups:
- ""
resources:
- services
verbs:
- list
- watch
- patch
- apiGroups:
- ""
resources:
- services/status
verbs:
- update
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- update
# For leader election
- apiGroups:
- ""
resources:
- endpoints
verbs:
- create
- apiGroups:
- ""
resources:
- endpoints
resourceNames:
- "cloud-controller-manager"
verbs:
- get
- list
- watch
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- configmaps
resourceNames:
- "cloud-controller-manager"
verbs:
- get
- update
- apiGroups:
- ""
resources:
- serviceaccounts
verbs:
- create
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- list
# For the PVL
- apiGroups:
- ""
resources:
- persistentvolumes
verbs:
- list
- watch
- patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: oci-cloud-controller-manager
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:cloud-controller-manager
subjects:
- kind: ServiceAccount
name: cloud-controller-manager
namespace: kube-system

View File

@ -59,13 +59,6 @@
- inventory_hostname == groups['kube_control_plane'][0]
tags: node-webhook
- name: Configure Oracle Cloud provider
include_tasks: oci.yml
tags: oci
when:
- cloud_provider is defined
- cloud_provider == 'oci'
- name: PriorityClass | Copy k8s-cluster-critical-pc.yml file
copy:
src: k8s-cluster-critical-pc.yml

View File

@ -1,19 +0,0 @@
---
- name: Copy OCI RBAC Manifest
copy:
src: "oci-rbac.yml"
dest: "{{ kube_config_dir }}/oci-rbac.yml"
mode: "0640"
when:
- cloud_provider is defined
- cloud_provider == 'oci'
- inventory_hostname == groups['kube_control_plane'][0]
- name: Apply OCI RBAC
kube:
kubectl: "{{ bin_dir }}/kubectl"
filename: "{{ kube_config_dir }}/oci-rbac.yml"
when:
- cloud_provider is defined
- cloud_provider == 'oci'
- inventory_hostname == groups['kube_control_plane'][0]

View File

@ -2,9 +2,7 @@
dependencies:
- role: kubernetes-apps/external_cloud_controller/openstack
when:
- cloud_provider is defined
- cloud_provider == "external"
- external_cloud_provider is defined
- external_cloud_provider == "openstack"
- inventory_hostname == groups['kube_control_plane'][0]
tags:
@ -12,9 +10,7 @@ dependencies:
- external-openstack
- role: kubernetes-apps/external_cloud_controller/vsphere
when:
- cloud_provider is defined
- cloud_provider == "external"
- external_cloud_provider is defined
- external_cloud_provider == "vsphere"
- inventory_hostname == groups['kube_control_plane'][0]
tags:
@ -22,9 +18,7 @@ dependencies:
- external-vsphere
- role: kubernetes-apps/external_cloud_controller/hcloud
when:
- cloud_provider is defined
- cloud_provider == "external"
- external_cloud_provider is defined
- external_cloud_provider == "hcloud"
- inventory_hostname == groups['kube_control_plane'][0]
tags:
@ -32,9 +26,7 @@ dependencies:
- external-hcloud
- role: kubernetes-apps/external_cloud_controller/huaweicloud
when:
- cloud_provider is defined
- cloud_provider == "external"
- external_cloud_provider is defined
- external_cloud_provider == "huaweicloud"
- inventory_hostname == groups['kube_control_plane'][0]
tags:
@ -42,9 +34,7 @@ dependencies:
- external-huaweicloud
- role: kubernetes-apps/external_cloud_controller/oci
when:
- cloud_provider is defined
- cloud_provider == "external"
- external_cloud_provider is defined
- external_cloud_provider == "oci"
- inventory_hostname == groups['kube_control_plane'][0]
tags:

View File

@ -103,14 +103,6 @@ dependencies:
tags:
- container_engine_accelerator
- role: kubernetes-apps/cloud_controller/oci
when:
- cloud_provider is defined
- cloud_provider == "oci"
- inventory_hostname == groups['kube_control_plane'][0]
tags:
- oci
- role: kubernetes-apps/gateway_api
when:
- gateway_api_enabled

View File

@ -1,12 +1,5 @@
---
dependencies:
- role: kubernetes-apps/persistent_volumes/openstack
when:
- cloud_provider is defined
- cloud_provider in [ 'openstack' ]
tags:
- persistent_volumes_openstack
- role: kubernetes-apps/persistent_volumes/cinder-csi
when:
- cinder_csi_enabled

View File

@ -1,7 +0,0 @@
---
persistent_volumes_enabled: false
storage_classes:
- name: standard
is_default: true
parameters:
availability: nova

View File

@ -1,20 +0,0 @@
---
- name: Kubernetes Persistent Volumes | Lay down OpenStack Cinder Storage Class template
template:
src: "openstack-storage-class.yml.j2"
dest: "{{ kube_config_dir }}/openstack-storage-class.yml"
mode: "0644"
register: manifests
when:
- inventory_hostname == groups['kube_control_plane'][0]
- name: Kubernetes Persistent Volumes | Add OpenStack Cinder Storage Class
kube:
name: storage-class
kubectl: "{{ bin_dir }}/kubectl"
resource: StorageClass
filename: "{{ kube_config_dir }}/openstack-storage-class.yml"
state: "latest"
when:
- inventory_hostname == groups['kube_control_plane'][0]
- manifests.changed

View File

@ -1,27 +0,0 @@
{% for class in storage_classes %}
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: "{{ class.name }}"
annotations:
storageclass.kubernetes.io/is-default-class: "{{ class.is_default | default(false) | ternary("true","false") }}"
provisioner: kubernetes.io/cinder
{% if class.mount_options is defined %}
mountOptions:
{% for option in class.mount_options | default([]) %}
- "{{ option }}"
{% endfor %}
{% endif %}
parameters:
{% for key, value in (class.parameters | default({})).items() %}
"{{ key }}": "{{ value }}"
{% endfor %}
{% if class.reclaim_policy is defined %}
reclaimPolicy: "{{ class.reclaim_policy }}"
{% endif %}
{% if class.volume_binding_mode is defined %}
volumeBindingMode: "{{ class.volume_binding_mode }}"
{% endif %}
allowVolumeExpansion: {{ expand_persistent_volumes }}
{% endfor %}

View File

@ -184,13 +184,6 @@ kube_encryption_algorithm: "secretbox"
# Which kubernetes resources to encrypt
kube_encryption_resources: [secrets]
# If non-empty, will use this string as identification instead of the actual hostname
kube_override_hostname: >-
{%- if cloud_provider is defined and cloud_provider in ['aws'] -%}
{%- else -%}
{{ inventory_hostname }}
{%- endif -%}
secrets_encryption_query: "resources[*].providers[0].{{ kube_encryption_algorithm }}.keys[0].secret"
## Support tls min version, Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13.

View File

@ -24,7 +24,7 @@ nodeRegistration:
taints: []
{% endif %}
criSocket: {{ cri_socket }}
{% if cloud_provider is defined and cloud_provider in ["external"] %}
{% if cloud_provider == "external" %}
kubeletExtraArgs:
cloud-provider: external
{% endif %}
@ -210,10 +210,6 @@ apiServer:
{% if kube_apiserver_feature_gates or kube_feature_gates %}
feature-gates: "{{ kube_apiserver_feature_gates | default(kube_feature_gates, true) | join(',') }}"
{% endif %}
{% if cloud_provider is defined and cloud_provider in ["openstack", "azure", "vsphere", "aws", "gce"] %}
cloud-provider: {{ cloud_provider }}
cloud-config: {{ kube_config_dir }}/cloud_config
{% endif %}
{% if tls_min_version is defined %}
tls-min-version: {{ tls_min_version }}
{% endif %}
@ -230,13 +226,8 @@ apiServer:
{% if kube_apiserver_tracing %}
tracing-config-file: {{ kube_config_dir }}/tracing/apiserver-tracing.yaml
{% endif %}
{% if kubernetes_audit or kube_token_auth or kube_webhook_token_auth or ( cloud_provider is defined and cloud_provider in ["openstack", "azure", "vsphere", "aws", "gce"] ) or apiserver_extra_volumes or ssl_ca_dirs | length %}
{% if kubernetes_audit or kube_token_auth or kube_webhook_token_auth or apiserver_extra_volumes or ssl_ca_dirs | length %}
extraVolumes:
{% if cloud_provider is defined and cloud_provider in ["openstack", "azure", "vsphere", "aws", "gce"] %}
- name: cloud-config
hostPath: {{ kube_config_dir }}/cloud_config
mountPath: {{ kube_config_dir }}/cloud_config
{% endif %}
{% if kube_token_auth %}
- name: token-auth-config
hostPath: {{ kube_token_dir }}
@ -326,10 +317,6 @@ controllerManager:
{% for key in kube_kubeadm_controller_extra_args %}
{{ key }}: "{{ kube_kubeadm_controller_extra_args[key] }}"
{% endfor %}
{% if cloud_provider is defined and cloud_provider in ["openstack", "azure", "vsphere", "aws", "gce"] %}
cloud-provider: {{ cloud_provider }}
cloud-config: {{ kube_config_dir }}/cloud_config
{% endif %}
{% if kube_network_plugin is defined and kube_network_plugin not in ["cloud"] %}
configure-cloud-routes: "false"
{% endif %}
@ -343,18 +330,8 @@ controllerManager:
tls-cipher-suites: {% for tls in tls_cipher_suites %}{{ tls }}{{ "," if not loop.last else "" }}{% endfor %}
{% endif %}
{% if cloud_provider is defined and cloud_provider in ["openstack", "azure", "vsphere", "aws", "gce"] or controller_manager_extra_volumes %}
{% if controller_manager_extra_volumes %}
extraVolumes:
{% if cloud_provider is defined and cloud_provider in ["openstack"] and openstack_cacert is defined %}
- name: openstackcacert
hostPath: "{{ kube_config_dir }}/openstack-cacert.pem"
mountPath: "{{ kube_config_dir }}/openstack-cacert.pem"
{% endif %}
{% if cloud_provider is defined and cloud_provider in ["openstack", "azure", "vsphere", "aws", "gce"] %}
- name: cloud-config
hostPath: {{ kube_config_dir }}/cloud_config
mountPath: {{ kube_config_dir }}/cloud_config
{% endif %}
{% for volume in controller_manager_extra_volumes %}
- name: {{ volume.name }}
hostPath: {{ volume.hostPath }}

View File

@ -6,10 +6,3 @@ kubeadm_join_timeout: 120s
# Enable kubeadm file discovery if anonymous access has been removed
kubeadm_use_file_discovery: "{{ remove_anonymous_access }}"
# If non-empty, will use this string as identification instead of the actual hostname
kube_override_hostname: >-
{%- if cloud_provider is defined and cloud_provider in ['aws'] -%}
{%- else -%}
{{ inventory_hostname }}
{%- endif -%}

View File

@ -134,13 +134,6 @@ kubelet_logfiles_max_size: 10Mi
## Support custom flags to be passed to kubelet
kubelet_custom_flags: []
# If non-empty, will use this string as identification instead of the actual hostname
kube_override_hostname: >-
{%- if cloud_provider is defined and cloud_provider in ['aws'] -%}
{%- else -%}
{{ inventory_hostname }}
{%- endif -%}
# The read-only port for the Kubelet to serve on with no authentication/authorization.
kube_read_only_port: 0
@ -153,61 +146,6 @@ kubelet_healthz_bind_address: 127.0.0.1
# sysctl_file_path to add sysctl conf to
sysctl_file_path: "/etc/sysctl.d/99-sysctl.conf"
# For the openstack integration kubelet will need credentials to access
# openstack apis like nova and cinder. Per default this values will be
# read from the environment.
openstack_auth_url: "{{ lookup('env', 'OS_AUTH_URL') }}"
openstack_username: "{{ lookup('env', 'OS_USERNAME') }}"
openstack_password: "{{ lookup('env', 'OS_PASSWORD') }}"
openstack_region: "{{ lookup('env', 'OS_REGION_NAME') }}"
openstack_tenant_id: "{{ lookup('env', 'OS_TENANT_ID') | default(lookup('env', 'OS_PROJECT_ID') | default(lookup('env', 'OS_PROJECT_NAME'), true), true) }}"
openstack_tenant_name: "{{ lookup('env', 'OS_TENANT_NAME') }}"
openstack_domain_name: "{{ lookup('env', 'OS_USER_DOMAIN_NAME') }}"
openstack_domain_id: "{{ lookup('env', 'OS_USER_DOMAIN_ID') }}"
# For the vsphere integration, kubelet will need credentials to access
# vsphere apis
# Documentation regarding these values can be found
# https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/vsphere/vsphere.go#L105
vsphere_vcenter_ip: "{{ lookup('env', 'VSPHERE_VCENTER') }}"
vsphere_vcenter_port: "{{ lookup('env', 'VSPHERE_VCENTER_PORT') }}"
vsphere_user: "{{ lookup('env', 'VSPHERE_USER') }}"
vsphere_password: "{{ lookup('env', 'VSPHERE_PASSWORD') }}"
vsphere_datacenter: "{{ lookup('env', 'VSPHERE_DATACENTER') }}"
vsphere_datastore: "{{ lookup('env', 'VSPHERE_DATASTORE') }}"
vsphere_working_dir: "{{ lookup('env', 'VSPHERE_WORKING_DIR') }}"
vsphere_insecure: "{{ lookup('env', 'VSPHERE_INSECURE') }}"
vsphere_resource_pool: "{{ lookup('env', 'VSPHERE_RESOURCE_POOL') }}"
vsphere_scsi_controller_type: pvscsi
# vsphere_public_network is name of the network the VMs are joined to
vsphere_public_network: "{{ lookup('env', 'VSPHERE_PUBLIC_NETWORK') | default('') }}"
## When azure is used, you need to also set the following variables.
## see docs/azure.md for details on how to get these values
# azure_tenant_id:
# azure_subscription_id:
# azure_aad_client_id:
# azure_aad_client_secret:
# azure_resource_group:
# azure_location:
# azure_subnet_name:
# azure_security_group_name:
# azure_vnet_name:
# azure_route_table_name:
# supported values are 'standard' or 'vmss'
# azure_vmtype: standard
# Sku of Load Balancer and Public IP. Candidate values are: basic and standard.
azure_loadbalancer_sku: basic
# excludes control plane nodes from standard load balancer.
azure_exclude_master_from_standard_lb: true
# disables the outbound SNAT for public load balancer rules
azure_disable_outbound_snat: false
# use instance metadata service where possible
azure_use_instance_metadata: true
# use specific Azure API endpoints
azure_cloud: AzurePublicCloud
## Support tls min version, Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13.
# tls_min_version: ""

View File

@ -1,82 +0,0 @@
---
- name: Check azure_tenant_id value
fail:
msg: "azure_tenant_id is missing"
when: azure_tenant_id is not defined or not azure_tenant_id
- name: Check azure_subscription_id value
fail:
msg: "azure_subscription_id is missing"
when: azure_subscription_id is not defined or not azure_subscription_id
- name: Check azure_aad_client_id value
fail:
msg: "azure_aad_client_id is missing"
when: azure_aad_client_id is not defined or not azure_aad_client_id
- name: Check azure_aad_client_secret value
fail:
msg: "azure_aad_client_secret is missing"
when: azure_aad_client_secret is not defined or not azure_aad_client_secret
- name: Check azure_resource_group value
fail:
msg: "azure_resource_group is missing"
when: azure_resource_group is not defined or not azure_resource_group
- name: Check azure_location value
fail:
msg: "azure_location is missing"
when: azure_location is not defined or not azure_location
- name: Check azure_subnet_name value
fail:
msg: "azure_subnet_name is missing"
when: azure_subnet_name is not defined or not azure_subnet_name
- name: Check azure_security_group_name value
fail:
msg: "azure_security_group_name is missing"
when: azure_security_group_name is not defined or not azure_security_group_name
- name: Check azure_vnet_name value
fail:
msg: "azure_vnet_name is missing"
when: azure_vnet_name is not defined or not azure_vnet_name
- name: Check azure_vnet_resource_group value
fail:
msg: "azure_vnet_resource_group is missing"
when: azure_vnet_resource_group is not defined or not azure_vnet_resource_group
- name: Check azure_route_table_name value
fail:
msg: "azure_route_table_name is missing"
when: azure_route_table_name is not defined or not azure_route_table_name
- name: Check azure_loadbalancer_sku value
fail:
msg: "azure_loadbalancer_sku has an invalid value '{{ azure_loadbalancer_sku }}'. Supported values are 'basic', 'standard'"
when: azure_loadbalancer_sku not in ["basic", "standard"]
- name: "Check azure_exclude_master_from_standard_lb is a bool"
assert:
that: azure_exclude_master_from_standard_lb | type_debug == 'bool'
- name: "Check azure_disable_outbound_snat is a bool"
assert:
that: azure_disable_outbound_snat | type_debug == 'bool'
- name: "Check azure_use_instance_metadata is a bool"
assert:
that: azure_use_instance_metadata | type_debug == 'bool'
- name: Check azure_vmtype value
fail:
msg: "azure_vmtype is missing. Supported values are 'standard' or 'vmss'"
when: azure_vmtype is not defined or not azure_vmtype
- name: Check azure_cloud value
fail:
msg: "azure_cloud has an invalid value '{{ azure_cloud }}'. Supported values are 'AzureChinaCloud', 'AzureGermanCloud', 'AzurePublicCloud', 'AzureUSGovernmentCloud'."
when: azure_cloud not in ["AzureChinaCloud", "AzureGermanCloud", "AzurePublicCloud", "AzureUSGovernmentCloud"]

View File

@ -1,34 +0,0 @@
---
- name: Check openstack_auth_url value
fail:
msg: "openstack_auth_url is missing"
when: openstack_auth_url is not defined or not openstack_auth_url
- name: Check openstack_username value
fail:
msg: "openstack_username is missing"
when: openstack_username is not defined or not openstack_username
- name: Check openstack_password value
fail:
msg: "openstack_password is missing"
when: openstack_password is not defined or not openstack_password
- name: Check openstack_region value
fail:
msg: "openstack_region is missing"
when: openstack_region is not defined or not openstack_region
- name: Check openstack_tenant_id value
fail:
msg: "one of openstack_tenant_id or openstack_trust_id must be specified"
when:
- openstack_tenant_id is not defined or not openstack_tenant_id
- openstack_trust_id is not defined
- name: Check openstack_trust_id value
fail:
msg: "one of openstack_tenant_id or openstack_trust_id must be specified"
when:
- openstack_trust_id is not defined or not openstack_trust_id
- openstack_tenant_id is not defined

View File

@ -1,22 +0,0 @@
---
- name: Check vsphere environment variables
fail:
msg: "{{ item.name }} is missing"
when: item.value is not defined or not item.value
with_items:
- name: vsphere_vcenter_ip
value: "{{ vsphere_vcenter_ip }}"
- name: vsphere_vcenter_port
value: "{{ vsphere_vcenter_port }}"
- name: vsphere_user
value: "{{ vsphere_user }}"
- name: vsphere_password
value: "{{ vsphere_password }}"
- name: vsphere_datacenter
value: "{{ vsphere_datacenter }}"
- name: vsphere_datastore
value: "{{ vsphere_datastore }}"
- name: vsphere_working_dir
value: "{{ vsphere_working_dir }}"
- name: vsphere_insecure
value: "{{ vsphere_insecure }}"

View File

@ -137,53 +137,6 @@
tags:
- kube-proxy
- name: Check cloud provider credentials
include_tasks: "cloud-credentials/{{ cloud_provider }}-credential-check.yml"
when:
- cloud_provider is defined
- cloud_provider in [ 'openstack', 'azure', 'vsphere' ]
tags:
- cloud-provider
- facts
- name: Test if openstack_cacert is a base64 string
set_fact:
openstack_cacert_is_base64: "{% if openstack_cacert is search('^([A-Za-z0-9+/]{4})*([A-Za-z0-9+/]{3}= | [A-Za-z0-9+/]{2}==)?$') %}true{% else %}false{% endif %}"
when:
- cloud_provider is defined
- cloud_provider == 'openstack'
- openstack_cacert is defined
- openstack_cacert | length > 0
- name: Write cacert file
copy:
src: "{{ openstack_cacert if not openstack_cacert_is_base64 else omit }}"
content: "{{ openstack_cacert | b64decode if openstack_cacert_is_base64 else omit }}"
dest: "{{ kube_config_dir }}/openstack-cacert.pem"
group: "{{ kube_cert_group }}"
mode: "0640"
when:
- cloud_provider is defined
- cloud_provider == 'openstack'
- openstack_cacert is defined
- openstack_cacert | length > 0
tags:
- cloud-provider
- name: Write cloud-config
template:
src: "cloud-configs/{{ cloud_provider }}-cloud-config.j2"
dest: "{{ kube_config_dir }}/cloud_config"
group: "{{ kube_cert_group }}"
mode: "0640"
when:
- cloud_provider is defined
- cloud_provider in [ 'openstack', 'azure', 'vsphere', 'aws', 'gce' ]
notify: Node | restart kubelet
tags:
- cloud-provider
- name: Install kubelet
import_tasks: kubelet.yml
tags:

View File

@ -1,11 +0,0 @@
[Global]
zone={{ aws_zone|default("") }}
vpc={{ aws_vpc|default("") }}
subnetId={{ aws_subnet_id|default("") }}
routeTableId={{ aws_route_table_id|default("") }}
roleArn={{ aws_role_arn|default("") }}
kubernetesClusterTag={{ aws_kubernetes_cluster_tag|default("") }}
kubernetesClusterId={{ aws_kubernetes_cluster_id|default("") }}
disableSecurityGroupIngress={{ "true" if aws_disable_security_group_ingress|default(False) else "false" }}
disableStrictZoneCheck={{ "true" if aws_disable_strict_zone_check|default(False) else "false" }}
elbSecurityGroup={{ aws_elb_security_group|default("") }}

View File

@ -1,26 +0,0 @@
{
"cloud": "{{ azure_cloud }}",
"tenantId": "{{ azure_tenant_id }}",
"subscriptionId": "{{ azure_subscription_id }}",
"aadClientId": "{{ azure_aad_client_id }}",
"aadClientSecret": "{{ azure_aad_client_secret }}",
"resourceGroup": "{{ azure_resource_group }}",
"location": "{{ azure_location }}",
"subnetName": "{{ azure_subnet_name }}",
"securityGroupName": "{{ azure_security_group_name }}",
"securityGroupResourceGroup": "{{ azure_security_group_resource_group | default(azure_vnet_resource_group) }}",
"vnetName": "{{ azure_vnet_name }}",
"vnetResourceGroup": "{{ azure_vnet_resource_group }}",
"routeTableName": "{{ azure_route_table_name }}",
"routeTableResourceGroup": "{{ azure_route_table_resource_group | default(azure_vnet_resource_group) }}",
"vmType": "{{ azure_vmtype }}",
{% if azure_primary_availability_set_name is defined %}
"primaryAvailabilitySetName": "{{ azure_primary_availability_set_name }}",
{%endif%}
"useInstanceMetadata": {{azure_use_instance_metadata | lower }},
{% if azure_loadbalancer_sku == "standard" %}
"excludeMasterFromStandardLB": {{ azure_exclude_master_from_standard_lb | lower }},
"disableOutboundSNAT": {{ azure_disable_outbound_snat | lower }},
{% endif%}
"loadBalancerSku": "{{ azure_loadbalancer_sku }}"
}

View File

@ -1,2 +0,0 @@
[global]
node-tags = {{ gce_node_tags }}

View File

@ -1,54 +0,0 @@
[Global]
auth-url="{{ openstack_auth_url }}"
username="{{ openstack_username }}"
password="{{ openstack_password }}"
region="{{ openstack_region }}"
{% if openstack_trust_id is defined and openstack_trust_id != "" %}
trust-id="{{ openstack_trust_id }}"
{% else %}
tenant-id="{{ openstack_tenant_id }}"
{% endif %}
{% if openstack_tenant_name is defined and openstack_tenant_name != "" %}
tenant-name="{{ openstack_tenant_name }}"
{% endif %}
{% if openstack_domain_name is defined and openstack_domain_name != "" %}
domain-name="{{ openstack_domain_name }}"
{% elif openstack_domain_id is defined and openstack_domain_id != "" %}
domain-id ="{{ openstack_domain_id }}"
{% endif %}
{% if openstack_cacert is defined and openstack_cacert != "" %}
ca-file="{{ kube_config_dir }}/openstack-cacert.pem"
{% endif %}
[BlockStorage]
{% if openstack_blockstorage_version is defined %}
bs-version={{ openstack_blockstorage_version }}
{% endif %}
{% if openstack_blockstorage_ignore_volume_az is defined and openstack_blockstorage_ignore_volume_az|bool %}
ignore-volume-az={{ openstack_blockstorage_ignore_volume_az }}
{% endif %}
{% if node_volume_attach_limit is defined and node_volume_attach_limit != "" %}
node-volume-attach-limit="{{ node_volume_attach_limit }}"
{% endif %}
{% if openstack_lbaas_enabled and openstack_lbaas_subnet_id is defined %}
[LoadBalancer]
subnet-id={{ openstack_lbaas_subnet_id }}
{% if openstack_lbaas_floating_network_id is defined %}
floating-network-id={{ openstack_lbaas_floating_network_id }}
{% endif %}
{% if openstack_lbaas_use_octavia is defined %}
use-octavia={{ openstack_lbaas_use_octavia }}
{% endif %}
{% if openstack_lbaas_method is defined %}
lb-method={{ openstack_lbaas_method }}
{% endif %}
{% if openstack_lbaas_provider is defined %}
lb-provider={{ openstack_lbaas_provider }}
{% endif %}
create-monitor={{ openstack_lbaas_create_monitor }}
monitor-delay={{ openstack_lbaas_monitor_delay }}
monitor-timeout={{ openstack_lbaas_monitor_timeout }}
monitor-max-retries={{ openstack_lbaas_monitor_max_retries }}
{% endif %}

View File

@ -1,36 +0,0 @@
[Global]
user = "{{ vsphere_user }}"
password = "{{ vsphere_password }}"
port = {{ vsphere_vcenter_port }}
insecure-flag = {{ vsphere_insecure }}
datacenters = "{{ vsphere_datacenter }}"
[VirtualCenter "{{ vsphere_vcenter_ip }}"]
[Workspace]
server = "{{ vsphere_vcenter_ip }}"
datacenter = "{{ vsphere_datacenter }}"
folder = "{{ vsphere_working_dir }}"
default-datastore = "{{ vsphere_datastore }}"
{% if vsphere_resource_pool is defined and vsphere_resource_pool != "" %}
resourcepool-path = "{{ vsphere_resource_pool }}"
{% endif %}
[Disk]
scsicontrollertype = {{ vsphere_scsi_controller_type }}
{% if vsphere_public_network is defined and vsphere_public_network != "" %}
[Network]
public-network = {{ vsphere_public_network }}
{% endif %}
[Labels]
{% if vsphere_zone_category is defined and vsphere_zone_category != "" %}
zone = {{ vsphere_zone_category }}
{% endif %}
{% if vsphere_region_category is defined and vsphere_region_category != "" %}
region = {{ vsphere_region_category }}
{% endif %}

View File

@ -21,9 +21,7 @@ KUBELET_VOLUME_PLUGIN="--volume-plugin-dir={{ kubelet_flexvolumes_plugins_dir }}
{% if kube_network_plugin is defined and kube_network_plugin == "cloud" %}
KUBELET_NETWORK_PLUGIN="--hairpin-mode=promiscuous-bridge --network-plugin=kubenet"
{% endif %}
{% if cloud_provider is defined and cloud_provider in ["openstack", "azure", "vsphere", "aws", "gce"] %}
KUBELET_CLOUDPROVIDER="--cloud-provider={{ cloud_provider }} --cloud-config={{ kube_config_dir }}/cloud_config"
{% elif cloud_provider is defined and cloud_provider in ["external"] %}
{% if cloud_provider == "external" %}
KUBELET_CLOUDPROVIDER="--cloud-provider={{ cloud_provider }}"
{% else %}
KUBELET_CLOUDPROVIDER=""

View File

@ -87,9 +87,6 @@
{% for d in default_searchdomains | default([]) + searchdomains | default([]) -%}
{{ dns_domain }}.{{ d }}./{{ d }}.{{ d }}./com.{{ d }}./
{%- endfor %}
cloud_resolver: "{{ ['169.254.169.254'] if cloud_provider is defined and cloud_provider == 'gce' else
['169.254.169.253'] if cloud_provider is defined and cloud_provider == 'aws' else
[] }}"
- name: Check if kubelet is configured
stat:

View File

@ -151,13 +151,6 @@
- dashboard_enabled
- not ignore_assert_errors
- name: Stop if RBAC is not enabled when OCI cloud controller is enabled
assert:
that: rbac_enabled
when:
- cloud_provider is defined and cloud_provider == "oci"
- not ignore_assert_errors
- name: Stop if kernel version is too low
assert:
that: ansible_kernel.split('-')[0] is version('4.9.17', '>=')
@ -173,10 +166,19 @@
- name: Check cloud_provider value
assert:
that: cloud_provider in ['gce', 'aws', 'azure', 'openstack', 'vsphere', 'oci', 'external']
msg: "If set the 'cloud_provider' var must be set either to 'gce', 'aws', 'azure', 'openstack', 'vsphere', 'oci' or 'external'"
that: cloud_provider == 'external'
when:
- cloud_provider is defined
- cloud_provider
- not ignore_assert_errors
tags:
- cloud-provider
- facts
- name: Check external_cloud_provider value
assert:
that: external_cloud_provider in ['hcloud', 'huaweicloud', 'oci', 'openstack', 'vsphere']
when:
- cloud_provider == 'external'
- not ignore_assert_errors
tags:
- cloud-provider

View File

@ -253,11 +253,7 @@ kube_apiserver_bind_address: 0.0.0.0
kube_apiserver_port: 6443
# If non-empty, will use this string as identification instead of the actual hostname
kube_override_hostname: >-
{%- if cloud_provider is defined and cloud_provider in ['aws'] -%}
{%- else -%}
{{ inventory_hostname }}
{%- endif -%}
kube_override_hostname: "{{ inventory_hostname }}"
# define kubelet config dir for dynamic kubelet
# kubelet_config_dir:
@ -275,6 +271,15 @@ kubelet_shutdown_grace_period: 60s
# to give normal pods time to be gracefully evacuated
kubelet_shutdown_grace_period_critical_pods: 20s
# Cloud Provider
# This variable can only be set to "external" or empty string, otherwise the check will fail.
cloud_provider: ""
# External Cloud Controller Manager (Formerly known as cloud provider)
# cloud_provider must be "external", otherwise this setting is invalid.
# Supported external cloud controllers are: 'openstack', 'vsphere', 'oci', 'huaweicloud' and 'hcloud'
# If you fill in a value other than the above, the check will fail.
external_cloud_provider: ""
# Whether to deploy the container engine
deploy_container_engine: "{{ 'k8s_cluster' in group_names or etcd_deployment_type == 'docker' }}"

View File

@ -24,17 +24,6 @@
delegate_to: "{{ groups['kube_control_plane'][0] }}"
- name: Stop if incompatible network plugin and cloudprovider
assert:
that:
- calico_ipip_mode == 'Never'
- calico_vxlan_mode in ['Always', 'CrossSubnet']
msg: "When using cloud_provider azure and network_plugin calico calico_ipip_mode must be 'Never' and calico_vxlan_mode 'Always' or 'CrossSubnet'"
when:
- cloud_provider is defined and cloud_provider == 'azure'
run_once: true
delegate_to: "{{ groups['kube_control_plane'][0] }}"
- name: Stop if supported Calico versions
assert:
that:

View File

@ -29,7 +29,7 @@
register: calico_kubelet_name
delegate_to: "{{ groups['kube_control_plane'][0] }}"
when:
- "cloud_provider is defined"
- cloud_provider
- name: Calico | Gather os specific variables
include_vars: "{{ item }}"

View File

@ -9,7 +9,7 @@
| select('contains', '/' ~ calico_pool_blocksize|d(26))
| select('contains', 'tunl0') | length == 0
when:
- (calico_ipip_mode is defined and calico_ipip_mode != 'Never' or cloud_provider is defined)
- (calico_ipip_mode is defined and calico_ipip_mode != 'Never')
- kube_network_plugin | default('calico') == 'calico'