Commit Graph

88 Commits (master)

Author SHA1 Message Date
Max Gautier 1818993a8a
CI: for upgrade testing, checkout old version before provisioning
Otherwise, modifying the test inventory + it's expected structure will
always fail.
2024-09-21 15:10:59 +02:00
Max Gautier 76c42b4d3f
CI: cleanup '-scale' tests infra (#11535)
There is actually no test using this since ad6fecefa8,
so there is no reason to keep that infra in our tests scripts.
2024-09-18 13:04:50 +01:00
Bogdan Sass 4b324cb0f0
Rename master to control plane - non-breaking changes only (#11394)
K8s is moving away from the "master" terminology, so kubespray should follow the same naming conventions. See 65d886bb30/sig-architecture/naming/recommendations/001-master-control-plane.md
2024-09-06 07:56:19 +01:00
Max Gautier b0be5f2dad
Print the name of faulty jinja templates in pre-commit (#11484) 2024-08-30 06:43:30 +01:00
Takuya Murakami 60b323b17f
[CI] Add a CI job to test cluster upgrading, and fix bug of testcases_run.sh (#11458)
* Fix: fix testcases_run.sh for upgrade tests

Need to git checkout ${CI_COMMIT_SHA} before running upgrade playbook (revert #11173 partially)

* feat: add CI job to test upgrade

Add a packet_ubuntu22-calico-all-in-one-upgrade job
2024-08-29 15:47:32 +01:00
Antoine Legrand a0587e0b8e
CI: rework pipeline: short/extended based on labels (#11324)
* CI: reduce VM resources requests to improve scheduling

* CI: Reduce default jobs; add labels(ci-full/extended) to run more test

* CI: use jobs dependencies instead of stages

* precommit one-job

* CI: Use Kubevirt VM to run Molecule and Vagrant jobs
2024-07-01 03:25:36 -07:00
Max Gautier 19851bb07c
collection-build-install convert to pre-commit 2024-05-28 13:26:46 +02:00
Max Gautier 7f7b65d388
Convert check_typo to pre-commit + use maintained version
client9/misspell is unmaintained, and has been forked by the golangci
team, see https://github.com/client9/misspell/issues/197#issuecomment-1596318684.

They haven't yet added a pre-commit config, so use my fork with the
pre-commit hook config until the pull request is merged.
2024-05-28 13:26:45 +02:00
Max Gautier 77bfb53455
Fix ci-matrix pre-commit hook
- Remove dependency of pydblite which fails to setup on recent pythons
- Discard shell script and put everything into pre-commit
2024-05-28 13:26:44 +02:00
Payback159 5d01dfa179 add auto generate documentation sidebar script, introduce script as pre-commit-hook, adapt existing scripts to work with documentation structure 2024-05-17 15:09:54 +02:00
Max Gautier 0b2533143f
Don't force python interpreter on ubuntu (#11183)
This allows local override in tasks for using virtualenvs.
2024-05-13 19:51:11 -07:00
Max Gautier d321e42d9e
CI: Factorize ansible-playbook flags (#11173)
We have inconsistent sets of options passed to the playbooks during our
CI runs.

Don't run ansible-playbook directly, instead factorize the execution in
a bash function using all the common flags.
Also remove various ENABLE_* variables and instead directly test for the
relevant conditions at execution time, as this makes it more obvious and
does not force one to go back and forth in the script.
2024-05-13 03:20:48 -07:00
Max Gautier ef95eb078a
Cleanup unused CI tooling (#11014) 2024-03-15 03:57:27 -07:00
Kay Yan 90b0151caf
support node feature discovery (#10861)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
2024-03-05 08:36:08 -08:00
Max Gautier e90cae9344
Refactor check_galaxy + fix version (#10729)
* Remove checks for docs using exact tags

Instead use a more generic documentation for installing kubespray as a
collection from git.

* Check that we upgraded galaxy.yml to next version

This is only intented to check for human error. The version in galaxy
should be the next (which does not mean the same if we're on master or a
release branch).

* Set collection version to KUBESPRAY_NEXT_VERSION
2024-01-11 15:49:31 +01:00
Max Gautier 51069223f5
Decouple kubespray-defaults from download (#10626)
* Decouple role kubespray-defaults from download

Avoids doing re-importing the download role on every invocation of
kubespray-defaults (and skipping everything).

This has a measurable effect on playbook performance.

* Update docs refering to moved download defaults
2023-12-11 16:56:17 +01:00
Max Gautier d2944d2813
Check jinja templates for syntax error (#10667)
Allow to fail early (pre-commit time) for jinja error, rather than
waiting until executing the playbook and the invalid template.

I could not find a simple jinja pre-commit hook in the wild.
2023-12-06 03:05:24 +01:00
Florian Ruynat 9696936b59
Fixup recover control plane playbook + add debian12/cilium test (#10411)
* Add debian12 cilium testing

* Fixup recover control plane playbook
2023-09-05 10:42:52 -07:00
Luke Simmons 1955943d4a
Removes Ansible reinstall from pipeline (#10032) 2023-08-14 05:11:21 -07:00
Arthur Outhenin-Chalandre 25cb90bc2d
Upgrade ansible (#10190)
* project: update all dependencies including ansible

Upgrade to ansible 7.x and ansible-core 2.14.x. There seems to be issue
with ansible 8/ansible-core 2.15 so we remain on those versions for now.
It's quite a big bump already anyway.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* tests: install aws galaxy collection

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* ansible-lint: disable various rules after ansible upgrade

Temporarily disable a bunch of linting action following ansible upgrade.
Those should be taken care of separately.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve deprecated-module ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve no-free-form ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[meta] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[playbook] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[tasks] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve risky-file-permissions ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve risky-shell-pipe ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: remove deprecated warn args

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: use fqcn for non builtin tasks

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve syntax-check[missing-file] for contrib playbook

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: use arithmetic inside jinja to fix ansible 6 upgrade

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

---------

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-06-26 03:15:45 -07:00
Victor Morales bf31a3a872
Split defaults main file (#10121) 2023-06-22 02:19:40 -07:00
Arthur Outhenin-Chalandre edc73bc3c8
project: upgrade test dependencies and drop ansible-core 2.11 (#10034)
Molecule 5.0 require ansible-core 2.12.10.
So this commit we update ansible-core from 2.12.5 to 2.12.10.
We also drop supporting two ansible-core version. Also we now use the "oldest"
still supported ansible-core version as both 2.11 is EOL and not
supported by molecule.



tests/molecule: remove linting in molecule to support molecule 5



tests/molecule: remove role name check for molecule 5 support

Kubespray doesn't use ansible galaxy style naming so we have to disable
that check.



contrib/inventory_builder: fix tox.ini for tox4



tests/molecule: fix get_playbook in testinfra tests



tests: upgrade most tests requirements

Exclude ansible-lint for now, I will do that in a separate PR.



tests/molecule: force kvm driver option

If we don't do this it fallbacks to qemu emulated on our CI for some
reasons.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-06-02 20:40:40 -07:00
Kay Yan 82633c6f61
Remove the Support of Debian 9 because Debian 9 is EOF (#10097)
* remove-debian9-support

* Add six module into openstack-cleanup/requirements.txt (#10099)

To fix tf-elastx_cleanup job which was failed with the following error:

   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/generic/password.py", line 16, in <module>
     from keystoneauth1.identity import v3
   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/v3/__init__.py", line 27, in <module>
     from keystoneauth1.identity.v3.oauth2_mtls_client_credential import *  # noqa
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/v3/oauth2_mtls_client_credential.py", line 17, in <module>
     import six
 ModuleNotFoundError: No module named 'six'

---------

Co-authored-by: Kenichi Omichi <ken1ohmichi@gmail.com>
2023-05-18 15:42:33 -07:00
Florian Ruynat 951face343
Migrate CI_BUILD_ID to CI_JOB_ID and CI_BUILD_REF to CI_COMMIT_SHA (#10063) 2023-05-11 04:21:17 -07:00
Luke Simmons acbf44a1b4
Adds support for Ansible collections (#9582) 2023-03-27 02:25:55 -07:00
Kenichi Omichi 7fec254f62
Drop part for supporting ansible 2.9 and 2.10 (#9842)
requirements-$ANSIBLE_VERSION.yml doesn't exist in Kubespray repo.
That was for supporting ansible 2.10-, and now Kubespray supports
2.11+. So this drops the part to avoid confusion.
2023-03-02 01:54:58 -08:00
Bas 2c93c997cf
pre-commit autocorrected files (#9750) 2023-02-06 01:35:16 -08:00
Florian Ruynat 1f36df666d
Update fedora35 vagrant box url (#9699)
* Update fedora35 vagrant box url

* Update Terraform to 1.3.7

* Update Vagrant to 2.3.4
2023-01-26 21:28:25 -08:00
Cyclinder bf8c64af08
feature request: add cyclinder to reviewers (#9646)
Signed-off-by: cyclinder qifeng.guo@daocloud.io

Signed-off-by: cyclinder qifeng.guo@daocloud.io
2023-01-10 03:05:27 -08:00
Kenichi Omichi ed38d8d3a1
Add ingress-nginx check for updating README (#9533)
To detect the version mismatch.
2022-12-01 01:16:27 -08:00
Kenichi Omichi e5ec0f18c0
Add packet_ubuntu20-calico-aio-hardening (#9359)
To verify the hardening method works always.
The configuration comes from docs/hardening.md

Fix yaml format of hardening.yml

Add condition to skip 040 test for hardening
2022-10-19 05:35:04 -07:00
Kenichi Omichi 24632ae81b
Add check_typo job (#9361)
To block merging pull requests which contain typo automatically.
2022-10-07 02:21:53 -07:00
Kenichi Omichi 7e862939db
Add kube-vip check to check_readme_versions.sh (#9155)
To check the kube-vip version between readme.md and the default value
on the role, this updates check_readme_versions.sh
2022-08-06 08:26:20 -07:00
Kenichi Omichi 7414409aa0
Add target components on check_readme_versions.sh (#9045)
This adds target components on check_readme_versions.sh after
merging https://github.com/kubernetes-sigs/kubespray/pull/9044
In addition, this fixes typo on check_readme_versions.sh

This adds `foo_version` variables for some components because
check_readme_versions.sh verifies the corresponding version for
`<component name>_version` from main.yml. This change also makes
consistency in the main.yml. In long-term, we will be able to
remove the existing `foo_image_tag` variables, but that is not now
for backwards compatibility for users.
2022-07-05 08:02:47 -07:00
Kenichi Omichi 6cd243f14e
Add component version check for README.md (#9042)
During code-review, reviwers needed to take care of README.md also
should be updated when the pull request updated component versions.
This adds the corresponding check to reduce reviwer's burden.
2022-06-29 00:14:05 -07:00
Calin Cristian Andrei 9143810a4d [CI] add remove node job 2022-05-04 06:35:51 -07:00
Kenichi Omichi f184725c5f
Use ansible 2.12 for testcases_prepare (#8763)
tests/requirements.txt links to tests/requirements-2.12.txt, so
Kubespray uses ansible 2.12 by default for testing. However we
forgot to update testcases_prepare.sh to use ansible 2.12.
This updates testcases_prepare to use ansible 2.12.
2022-05-02 11:34:31 -07:00
Cristian Calin 835fd86a08
[CI] split molecule testes to run in parallel (#8756)
* add parametrization to molecule_run.sh

* [CI] split molecule tests to allow parallelization of work
2022-04-29 00:09:12 -07:00
Cristian Calin a090038d02
[CI] add ara to collect CI job logs (#8545) 2022-02-23 07:36:19 -08:00
Cristian Calin 2f44b40d68
OEL7: Fix CentOS7 Extras for OEL7 (#8219)
* OEL7: Fix CentOS7 Extras for OEL7

* Molecule: add logs collection for jobs
2021-11-29 13:39:21 -08:00
Florian Ruynat a5f88e14d0
Cleanup tests (#8234)
* Add Fedora 35 image, support and CI

* Cleanup tests and allow_failure for vagrant
2021-11-26 09:00:51 -08:00
Cristian Calin e78bda65fe
Defaults: replace docker with containerd as our default container_manager (#8175)
* Defaults: replace docker with containerd as our default container_manager

* CI: Use docker for download_localhost test

* Defaults: with container_manager=containerd we need etcd_deployment_type=host

* CI: Run weave jobs with docker

* CI: Vagrant don't download_force_cache

* CI: Fix upgrade tests

* should run compatible with old settings, this means docker
* we need to run with a distro that has at least modern containerd,
  this means move from debian9 to debian10 to allow `containerd_version`
  to match between 2.17 and master
2021-11-25 06:54:33 -08:00
Cristian Calin 0d0468e127
Exercise multiple ansible versions in CI (#8172)
* Ansible: separate requirements files for supported ansible versions

* Ansible: allow using ansible 2.11

* CI: Exercise Ansible 2.9 and Ansible 2.11 in a basic AIO CI job

* CI: Allow running a reset test outside of idempotency tests and running it in stage1

* CI: move ubuntu18-calico-aio job to stage2 and relay only on ubuntu20 with the variously supported ansible versions for stage1

* CI: add capability to install collections or roles from ansible-galaxy to mitigate missing behavior in older ansible versions
2021-11-10 16:11:50 -08:00
Cristian Calin 8d553f7e91
Mitogen: deprecate the use of mitogen and remove coverage from CI (#8147) 2021-11-05 00:57:52 -07:00
Cristian Calin 2ed211ba15
Fix-CI: python was upgraded in CI to 3.10 and pathlib is now included in python base making this dependency break the CI (#8153) 2021-11-03 12:52:32 -07:00
Florian Ruynat c7529270ff
Fix CI script for Terraform >0.15 (#7928) 2021-09-01 04:30:59 -07:00
jayonlau 6b5da84014
Clean up extra spaces last one (#7904)
Although these errors are not important, they affect the code specification.
2021-08-25 12:06:09 -07:00
Cristian Calin 7516fe142f
Move to Ansible 3.4.0 (#7672)
* Ansible: move to Ansible 3.4.0 which uses ansible-base 2.10.10

* Docs: add a note about ansible upgrade post 2.9.x

* CI: ensure ansible is removed before ansible 3.x is installed to avoid pip failures

* Ansible: use newer ansible-lint

* Fix ansible-lint 5.0.11 found issues

* syntax issues
* risky-file-permissions
* var-naming
* role-name
* molecule tests

* Mitogen: use 0.3.0rc1 which adds support for ansible 2.10+

* Pin ansible-base to 2.10.11 to get package fix on RHEL8
2021-07-12 00:00:47 -07:00
Florian Ruynat d1388d69d0
Fix tests following python change (#7775)
* Fix ansible detection for python3 and ubuntu

* Fix oracle missing centos-extras repo for containerd/docker dependencies
2021-07-08 18:52:53 -07:00
Kenichi Omichi 486b223e01
Replace kube-master with kube_control_plane (#7256)
This replaces kube-master with kube_control_plane because of [1]:

  The Kubernetes project is moving away from wording that is
  considered offensive. A new working group WG Naming was created
  to track this work, and the word "master" was declared as offensive.
  A proposal was formalized for replacing the word "master" with
  "control plane". This means it should be removed from source code,
  documentation, and user-facing configuration from Kubernetes and
  its sub-projects.

NOTE: The reason why this changes it to kube_control_plane not
      kube-control-plane is for valid group names on ansible.

[1]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint/README.md#motivation
2021-03-23 17:26:05 -07:00