Commit Graph

31 Commits (70bb19dd2334c746bdae288072dee27da7d74e85)

Author SHA1 Message Date
Mathieu Parent 09f3caedaa
[download] Don't fail on 304 Not Modified (#10452)
i.e when file was not modified since last download
2023-09-21 00:20:20 -07:00
Arthur Outhenin-Chalandre d21bfb84ad
project: resolve ansible-lint key-order rule (#10314)
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-08-10 00:57:27 -07:00
Arthur Outhenin-Chalandre 36e5d742dc
Resolve ansible-lint name errors (#10253)
* project: fix ansible-lint name

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: ignore jinja template error in names

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: capitalize ansible name

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: update notify after name capitalization

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

---------

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-07-26 07:36:22 -07:00
Arthur Outhenin-Chalandre 5d00b851ce
project: fix var-spacing ansible rule (#10266)
* project: fix var-spacing ansible rule

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: fix spacing on the beginning/end of jinja template

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: fix spacing of default filter

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: fix spacing between filter arguments

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: fix double space at beginning/end of jinja

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: fix remaining jinja[spacing] ansible-lint warning

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

---------

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-07-04 20:36:54 -07:00
Arthur Outhenin-Chalandre 25cb90bc2d
Upgrade ansible (#10190)
* project: update all dependencies including ansible

Upgrade to ansible 7.x and ansible-core 2.14.x. There seems to be issue
with ansible 8/ansible-core 2.15 so we remain on those versions for now.
It's quite a big bump already anyway.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* tests: install aws galaxy collection

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* ansible-lint: disable various rules after ansible upgrade

Temporarily disable a bunch of linting action following ansible upgrade.
Those should be taken care of separately.

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve deprecated-module ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve no-free-form ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[meta] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[playbook] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve schema[tasks] ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve risky-file-permissions ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve risky-shell-pipe ansible-lint error

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: remove deprecated warn args

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: use fqcn for non builtin tasks

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: resolve syntax-check[missing-file] for contrib playbook

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

* project: use arithmetic inside jinja to fix ansible 6 upgrade

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>

---------

Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
2023-06-26 03:15:45 -07:00
yjqg6666 0b102287d1
[#10148] The download.timeout can be changed by variable download.timeout (#10149)
Reference:
  https://docs.ansible.com/ansible/latest/collections/ansible/builtin/get_url_module.html#parameter-timeout
2023-05-31 18:15:45 -07:00
Maxime Picaud f6e4a231cb
fix(download): validate mirrors on localhost (#9669) 2023-03-23 08:04:32 -07:00
tu1h 8cf5fefe84
Add download retries option (#9911)
Signed-off-by: tu1h <lihai.tu@daocloud.io>
2023-03-21 09:38:12 -07:00
ERIK 9ad2d24ad8
Add unsafe_show_logs switch (#9164)
Signed-off-by: bo.jiang <bo.jiang@daocloud.io>

Signed-off-by: bo.jiang <bo.jiang@daocloud.io>
2022-08-16 18:52:48 -07:00
Calin Cristian Andrei c2700266b0 [download] fix dependencies for downloads 2022-06-15 00:57:20 -07:00
Kenichi Omichi cd7381d8de
Drop Ansible support for v2.9 and v2.10 (#8925)
Ansible v2.9 and v2.10 are EOL as [1].
This drops those version supports by following the upstream Ansible.

This sets use_ssh_args true always because that is required to use
ssh_args on ansible.cfg on Ansible v2.11 or later[2].

ansible_ssh_host is replaced with ansible_host because ansible_ssh_host
has been deprecated already and cenots7 jobs were failed due to the
deprecated ansible_ssh_host.

[1]: https://docs.ansible.com/ansible/devel/reference_appendices/release_and_maintenance.html#ansible-core-changelogs
[2]: https://docs.ansible.com/ansible/latest/collections/ansible/posix/synchronize_module.html#parameter-use_ssh_args
2022-06-09 07:07:42 -07:00
Cristian Calin c6e5314fab
implement download mirrors support (#8474)
* [download] add mechanism to support mirrors

* [calico] support alternate download url
2022-02-14 13:19:32 -08:00
Cristian Calin 7516fe142f
Move to Ansible 3.4.0 (#7672)
* Ansible: move to Ansible 3.4.0 which uses ansible-base 2.10.10

* Docs: add a note about ansible upgrade post 2.9.x

* CI: ensure ansible is removed before ansible 3.x is installed to avoid pip failures

* Ansible: use newer ansible-lint

* Fix ansible-lint 5.0.11 found issues

* syntax issues
* risky-file-permissions
* var-naming
* role-name
* molecule tests

* Mitogen: use 0.3.0rc1 which adds support for ansible 2.10+

* Pin ansible-base to 2.10.11 to get package fix on RHEL8
2021-07-12 00:00:47 -07:00
Etienne Champetier a790935d02
Only setup *_PROXY env variables where needed (#7095)
no_proxy is a pain to get right, and having proxy variables present causes issues
(k8s components get proxy configuration after upgrade, see #7100)

It's better to only configure what require proxy:
- the runtime (containerd/docker/crio)
- the package manager + apt_key
- the download tasks

Tested with the following clusters
- 4 CentOS 8 nodes
- 1 Ubuntu 20.04 node

Signed-off-by: Etienne Champetier <champetier.etienne@gmail.com>
2021-01-11 07:21:08 -08:00
bozzo 276c450759
Use `connection: local` when `delegate_to: localhost` (#6322)
This will avoid SSH connection on the local host
2020-06-25 08:14:38 -07:00
Kubernetes Prow Robot 66408a87ee
Refactor download role (#5697)
* download file

* download containers

* fix push image to nodes

* pull if none image on host

* fix

* improve docker image tag checks.
do not pull already cached images

* rebase fix merge conflict

* add support download_run_once when upgrade and scale cluster
add some test with download_run_once

* set default values to temp flag for every download cycle

* add save,load abilty for containerd and crio when download_run_once=true

* return redefine image save/load command to  set_docker_image_facts.yml

* move set command to set_container_facts

* ctr in containerd_bin_dir

* fix order of ctr image export arguments

* temporary disable download_run_once for containerd and crio
due https://github.com/containerd/containerd/issues/4075

* remove unused files

* fix strict yaml linter warning and errors

* refactor logical conditions to pull and cache container images

* remove comment due lint check

* document role

* remove image_load_on_localhost, because cached images are always loaded to docker on remote sites

* remove XXX from debug output
2020-03-05 07:31:39 -08:00
Sylvain Chateau 0ca7aa126b
added "Flatcar", "Flatcar Container Linux by Kinvolk" for all coreOS role (#5607) 2020-02-18 00:15:29 -08:00
Maxime Guyot b15d41a96a Add support to Ansible 2.9 (#5361) 2019-12-05 07:24:32 -08:00
Matthew Mosesohn a43e0d3f95 Switch to Kubernetes v1.16.0 (#5189)
* Switch to Kubernetes v1.16.0

Change-Id: I5d6a9528b2d443750fc5e031aff15ad3ffead158

* Fix download localhost cached file path

Change-Id: I65e79b70e3d1b37265ebc60f41b460cf4b0a0d47

* fix kubeadm etcd for v1.16

Change-Id: I6888a00fd48b530a38b0b31c4095492476af42d2

* disable tf packet jobs

Change-Id: I075c4666547fdea4c50ec04864f38e2cfaa79154

* Disable contiv packet jobs. Fix kube-router

Change-Id: I3170e8789e60711d4cee8faf65f2094480b79b8d

* bump sonobuoy version

Change-Id: Ib946905629c7c53ed88f08fb2f41c454457a0097
2019-10-02 02:21:07 -07:00
陈谭军 99dbc6d780 clean-up doc,spelling mistakes (#5206) 2019-09-26 04:25:08 -07:00
Sergey 494a6512b8 fix bug: run Copy image to ansible host cache on download_delegate host (#5094)
* run 'task download_container | Copy image to ansible host cache' with synchronize on download_delegate host

* try to run task copy file to ansible host on all inventory, not only on first random host
2019-08-21 23:38:30 -07:00
Johnny Halfmoon 23c9071c30 Added file and container image caching (#4828)
* File and container image downloads are now cached localy, so that repeated vagrant up/down runs do not trigger downloading of those files. This is especially useful on laptops with kubernetes runnig locally on vm's. The total size of the cache, after an ansible run, is currently around 800MB, so bandwidth (=time) savings can be quite significant.

* When download_run_once is false, the default is still not to cache, but setting download_force_cache will still enable caching.

* The local cache location can be set with download_cache_dir and defaults to /tmp/kubernetes_cache

* A local docker instance is no longer required to cache docker images; Images are cached to file. A local docker instance is still required, though, if you wish to download images on localhost.

* Fixed a FIXME, wher the argument was that delegate_to doesn't play nice with omit. That is a correct observation and the fix is to use default(inventory_host) instead of default(omit). See ansible/ansible#26009

* Removed "Register docker images info" task from download_container and set_docker_image_facts because it was faulty and unused.

* Removed redundant when:download.{container,enabled,run_once} conditions from {sync,download}_container.yml

* All features of commit d6fd0d2aca by Timoses <timosesu@gmail.com>, merged May 1st 2019, are included in this patch. Not all code was included verbatim, but each feature of that commit was checked to be working in this patch. One notable change: The actual downloading of the kubeadm images was moved to {download,sync)_container, to enable caching.

Note 1: I considered splitting this patch, but most changes that are not directly related to caching, are a pleasant by-product of implementing the caching code, so splitting would be impractical.

Note 2: I have my doubts about the usefulness of the upload, download and upgrade tags in the download role. Must they remain or can they be removed? If anybody knows, then please speak up.
2019-06-10 11:21:07 -07:00
MarkusTeufelberger e67f848abc ansible-lint: add spaces around variables [E206] (#4699) 2019-05-02 14:24:21 -07:00
Timoses d6fd0d2aca Enable delegating all downloads (binaries, images, kubeadm images) (#4420)
* Download to delegate and sync files when download_run_once

* Fail on error after saving container image

* Do not set changed status when downloaded container was up to date

* Only sync containers when they are actually required

Previously, non-required images (pull_required=false as
image existed on target host) were synced to the target
hosts. This failed as the image was not downloaded to
the download_delegate and hence was not available for
syncing.

* Sync containers when only missing on some hosts

* Consider images with multiple repo tags

* Enable kubeadm images pull/syncing with download_delegate

* Use kubeadm images list to pull/sync

'kubeadm config images pull' is replaced by collecting the images
list with 'kubeadm config images list' and using the commonly
used method of pull/syncing the images.

* Ensure containers are downloaded and synced for all hosts

* Fix download/syncing when download_delegate is a kubernetes host
2019-05-01 01:10:56 -07:00
Manuel Cintron 7633e6d582 Added pass through parameters to enable basic auth for downloads 2019-01-08 19:36:13 -06:00
Antoine Legrand 2a3aa591e0
Download role (#3553)
* codestyle tests

* Download destination can be different than local_release_dir
2018-10-20 13:56:55 +02:00
Anatoly Rugalev 8f85ea89fa Added `download_validate_certs` option which allows to disables SSL validation for file downloads 2018-09-21 11:51:17 +02:00
Andreas Krüger 2c89a02db3 Only download container/file if host is in defined group (#2565)
* Only download container/file if host is in defined group

* Set correct when clause

* Fix last entries

* Update download groups
2018-03-30 22:40:01 -04:00
Spencer Smith b27453d8d8 improved proxy support 2017-10-30 11:42:14 -04:00
Matthew Mosesohn a52bc44f5a Fix broken CI jobs (#1854)
* Fix broken CI jobs

Adjust image and image_family scenarios for debian.
Checkout CI file for upgrades

* add debugging to file download

* Fix download for alternate playbooks

* Update ansible ssh args to force ssh user

* Update sync_container.yml
2017-10-25 11:45:54 +01:00
Matthew Mosesohn fc9a65be2b Refactor downloads to use download role directly (#1824)
* Refactor downloads to use download role directly

Also disable fact delegation so download delegate works acros OSes.

* clean up bools and ansible_os_family conditionals
2017-10-19 09:17:11 +01:00