Commit Graph

99 Commits (master)

Author SHA1 Message Date
Kay Yan e5d2452828
fix elastx cleanup security groups (#11411) 2024-09-15 13:59:13 +01:00
ChengHao Yang 8cb081a3d0
Fix: download hash crictl link (#11534)
Signed-off-by: ChengHao Yang <17496418+tico88612@users.noreply.github.com>
2024-09-14 17:35:13 +01:00
Max Gautier 230cb37626
download_hash: document usage 2024-09-08 14:27:03 +02:00
Max Gautier dec4e711d1
download_hash: cache request for 'multi-hash' files
This avoid re-downloading the same file for different arch and
re-parsing it
2024-09-08 14:27:03 +02:00
Max Gautier 1b1045c0e2
download_hash: use persistent session
This allows to reuse http connection and be more efficient.
From rough measuring it saves around 25-30% of execution time.
2024-09-08 14:27:02 +02:00
Max Gautier 86855be634
download_hash: document missing support 2024-09-08 14:27:02 +02:00
Max Gautier b2e64aed4b
download_hash: support 'multi-hash' components 2024-09-08 14:27:01 +02:00
Max Gautier a2644c7a4f
download_hash: add support for 'simple hash' components 2024-09-08 14:14:00 +02:00
Max Gautier e256f74f2a
download_hash: propagate new patch versions to all archs 2024-09-08 14:11:47 +02:00
Max Gautier 2710e984c8
download_hash: argument handling with argparse
Allow the script to be called with a list of components, to only
download new versions checksums for those.
By default, we get new versions checksums for all supported (by the
script) components.
2024-09-08 11:13:25 +02:00
Max Gautier da0e445d69
download_hash.py: support for 'multi-hash' file + runc
runc upstream does not provide one hash file per assets in their
releases, but one file with all the hashes.
To handle this (and/or any arbitrary format from upstreams), add a
dictionary mapping the name of the download to a lambda function which
transform the file provided by upstream into a dictionary of hashes,
keyed by architecture.
2024-09-08 11:13:24 +02:00
Max Gautier a7616231a4
download_hash.py: generalized and data-driven
The script is currently limited to one hardcoded URL for kubernetes
related binaries, and a fixed set of architectures.

The solution is three-fold:
1. Use an url template dictionary for each download -> this allow to easily
   add support for new downloads.
2. Source the architectures to search from the existing data
3. Enumerate the existing versions in the data and start searching from
   the last one until no newer version is found (newer in the version
   order sense, irrespective of actual age)
2024-09-08 11:13:23 +02:00
Vlad Korolev 9a7b021eb8
Do not use ‘yes/no’ for boolean values (#11472)
Consistent boolean values in ansible playbooks
2024-08-28 06:30:56 +01:00
Bas 8f5f75211f
Improving yamllint configuration (#11389)
Signed-off-by: Bas Meijer <bas.meijer@enexis.nl>
2024-07-25 18:42:20 -07:00
Kay Yan 647092b483
fix openstack cleanup (#11299)
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
2024-06-21 10:30:55 -07:00
Max Gautier d50f61eae5
pre-commit: apply autofixes hooks and fix the rest manually
- markdownlint (manual fix)
- end-of-file-fixer
- requirements-txt-fixer
- trailing-whitespace
2024-05-28 13:26:44 +02:00
Max Gautier 96bb0a3e12
sidebar_gen: force C locale
If we don't, find output can change depending on system locale, which
will produce inconsistent results.
2024-05-19 12:04:07 +02:00
Max Gautier fae41172ed
gen_docs_sidebar: redirect output to sidebar.md as part of the script
The script should be self-sufficient (with no need to manually specify a
redirection).
This makes it easier to run it as part of pre-commit and CI.
2024-05-19 11:54:39 +02:00
Payback159 5d01dfa179 add auto generate documentation sidebar script, introduce script as pre-commit-hook, adapt existing scripts to work with documentation structure 2024-05-17 15:09:54 +02:00
Abhishek Jain f3065cc5c4
bump skopeo version and checksum (#11044)
Signed-off-by: Abhishek Jain <jain.abhishek1991@gmail.com>
2024-04-16 01:19:27 -07:00
Arthur Outhenin-Chalandre 26af6c7fda
scripts: ignore download_hash download failures (#10998)
* scripts: ignore download_hash download failures

Binary names on github releases often change and this script might break
because of that, this commit allow to ignore these failures as a mean to
be able to run the script anyway.

Signed-off-by: Arthur Outhenin-Chalandre <arthur@cri.epita.fr>

* scripts: use sha256sums for crio as well

Signed-off-by: Arthur Outhenin-Chalandre <arthur@cri.epita.fr>

* scripts: add ppc64le support for crio

Signed-off-by: Arthur Outhenin-Chalandre <arthur@cri.epita.fr>

---------

Signed-off-by: Arthur Outhenin-Chalandre <arthur@cri.epita.fr>
2024-03-11 09:36:56 -07:00
Arthur Outhenin-Chalandre 43c1e3b15e
scripts: refactor scripts/download_hash.sh (#10713)
The new version brings the following improvements:
- remove having to resort to python python to limit tags (it it slower than
the sh equivalent as python has a somewhat significant startup time).

- Introduce a concept of min version so that it can only get Kubernetes
version supported by Kubespray.

- Fix an issue with kata changing their file scheme (the arch
  specifically)

- Now download sha256/sha256sum files if provided rather than
  downloading the full file and computing the hash

- A few minor style tweaks

Signed-off-by: Arthur Outhenin-Chalandre <arthur@cri.epita.fr.fr>
2024-03-11 04:53:26 -07:00
Max Gautier ee8b909a67
Download hash script: auto discover versions (#10849)
* Download patches version automatically from a minor

* Automate versions discovery for hash download

* Small refactoring
2024-01-29 18:06:10 -08:00
Simon Wessel 0e26f6f3e2
chore: improve performance of python script for hash download (#10335)
The old version of the script downloaded all binaries and generated file checksums locally.
This was a slow process since all binaries of all architectures needed to be downloaded.
The new version simply downloads the .sha256 files containing the binary checksum in text
form which saves a lot of traffic and time.
2024-01-23 16:41:20 +01:00
Louis Tu 8f2390a120
Fix the path of download.yml (#10711)
Signed-off-by: tu1h <lihai.tu@daocloud.io>
2023-12-12 13:47:27 +01:00
Max Gautier 51069223f5
Decouple kubespray-defaults from download (#10626)
* Decouple role kubespray-defaults from download

Avoids doing re-importing the download role on every invocation of
kubespray-defaults (and skipping everything).

This has a measurable effect on playbook performance.

* Update docs refering to moved download defaults
2023-12-11 16:56:17 +01:00
ERIK 9c610ee11d
not requiring 'v' in youki version (#10346)
Signed-off-by: bo.jiang <bo.jiang@daocloud.io>
2023-08-08 19:33:51 -07:00
Arthur Outhenin-Chalandre 36e5d742dc
Resolve ansible-lint name errors (#10253)
* project: fix ansible-lint name

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: ignore jinja template error in names

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: capitalize ansible name

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: update notify after name capitalization

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

---------

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-07-26 07:36:22 -07:00
Victor Morales 78c1775661
Upgrade versions (#9798)
The following applications have been upgraded:

* Cilium
* Helm
* crun
* Katacontainers
* youki
* gvisor
* skopeo
* yq

Signed-off-by: Victor Morales <chipahuac@hotmail.com>
2023-07-05 03:32:58 -07:00
Arthur Outhenin-Chalandre 5d00b851ce
project: fix var-spacing ansible rule (#10266)
* project: fix var-spacing ansible rule

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: fix spacing on the beginning/end of jinja template

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: fix spacing of default filter

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: fix spacing between filter arguments

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: fix double space at beginning/end of jinja

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: fix remaining jinja[spacing] ansible-lint warning

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

---------

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-07-04 20:36:54 -07:00
Arthur Outhenin-Chalandre 25cb90bc2d
Upgrade ansible (#10190)
* project: update all dependencies including ansible

Upgrade to ansible 7.x and ansible-core 2.14.x. There seems to be issue
with ansible 8/ansible-core 2.15 so we remain on those versions for now.
It's quite a big bump already anyway.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* tests: install aws galaxy collection

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* ansible-lint: disable various rules after ansible upgrade

Temporarily disable a bunch of linting action following ansible upgrade.
Those should be taken care of separately.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve deprecated-module ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve no-free-form ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[meta] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[playbook] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[tasks] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve risky-file-permissions ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve risky-shell-pipe ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: remove deprecated warn args

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: use fqcn for non builtin tasks

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve syntax-check[missing-file] for contrib playbook

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: use arithmetic inside jinja to fix ansible 6 upgrade

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

---------

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-06-26 03:15:45 -07:00
Victor Morales bf31a3a872
Split defaults main file (#10121) 2023-06-22 02:19:40 -07:00
Victor Morales d79ada931d
Update download hash bash script (#10120) 2023-06-19 02:52:22 -07:00
Kay Yan 808524bed6
fix-ci-tf-elastx_cleanup (#10133) 2023-05-25 08:38:52 -07:00
Ricky Sadowski a4fa9aed75
fix: use dl.k8s.io, not kubernetes-release bucket (#10118)
Signed-off-by: Ricky Sadowski <richard.j.sadowski@gmail.com>
2023-05-22 17:50:21 -07:00
Kenichi Omichi c14d9c5c97
Add six module into openstack-cleanup/requirements.txt (#10099)
To fix tf-elastx_cleanup job which was failed with the following error:

   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/generic/password.py", line 16, in <module>
     from keystoneauth1.identity import v3
   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/v3/__init__.py", line 27, in <module>
     from keystoneauth1.identity.v3.oauth2_mtls_client_credential import *  # noqa
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/usr/local/lib/python3.11/site-packages/keystoneauth1/identity/v3/oauth2_mtls_client_credential.py", line 17, in <module>
     import six
 ModuleNotFoundError: No module named 'six'
2023-05-17 20:22:33 -07:00
Kulwant Singh 9948863d3a
use dl.k8s.io not gs://kubernetes-release (#10066) 2023-05-16 21:02:33 -07:00
Florian Ruynat 951face343
Migrate CI_BUILD_ID to CI_JOB_ID and CI_BUILD_REF to CI_COMMIT_SHA (#10063) 2023-05-11 04:21:17 -07:00
Victor Morales fff400513b
Improve method to get binary checksums (#9782) 2023-03-09 13:56:30 -08:00
mgiessing 6a4fd33a03
Added ppc64le support (#8505)
* Added ppc64le support

* Fixed linting errors
2022-02-04 00:14:00 -08:00
Kenichi Omichi c0d1bb1a5c
Remove subnet from router on tf-elastx_cleanup (#8425)
The tf-elastx_cleanup test job was failed with error message:

Port xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx cannot be deleted
directly via the port API: has device owner network:router_interface.

That means necessary to remove a subnet from the router before
deleting the port.
This adds a method to removes a subnet from the router automatically.
2022-01-15 00:50:15 -08:00
Victor Morales c7d12cddec
Ensure python main function return values (#7860)
The main functions are wrapped by a sys.exit function which expects and
argument. The curent implementation isn't returning values in all cases.
This change ensures main functions return a value in all cases.
2021-08-19 06:51:24 -07:00
Victor Morales c2d4700571
Remove unused python imports (#7859) 2021-08-13 13:35:32 -07:00
Cristian Calin 7516fe142f
Move to Ansible 3.4.0 (#7672)
* Ansible: move to Ansible 3.4.0 which uses ansible-base 2.10.10

* Docs: add a note about ansible upgrade post 2.9.x

* CI: ensure ansible is removed before ansible 3.x is installed to avoid pip failures

* Ansible: use newer ansible-lint

* Fix ansible-lint 5.0.11 found issues

* syntax issues
* risky-file-permissions
* var-naming
* role-name
* molecule tests

* Mitogen: use 0.3.0rc1 which adds support for ansible 2.10+

* Pin ansible-base to 2.10.11 to get package fix on RHEL8
2021-07-12 00:00:47 -07:00
Maxime Guyot f94182f77d
Add cleanup of openstack network ports in CI (#7201) 2021-01-21 12:51:02 -08:00
Maxime Guyot e1265b2e7b
Fix order of OS CI cleanup (#6714) 2020-09-18 16:20:28 -07:00
Maxime Guyot 91742055e0
Fix E306 in scripts/ (#6496) 2020-08-05 01:56:28 -07:00
Maxime Guyot e70f27dd79
Add noqa and disable .ansible-lint global exclusions (#6410) 2020-07-27 06:24:17 -07:00
Florent Monbillard bf8c8976dd
Upgrade etcd to 3.4.3 (#5998) 2020-07-20 07:26:51 -07:00
Maxime Guyot 45e12df8a3
Cleanup OpenStack network things (#6283) 2020-06-26 00:03:39 -07:00