kubespray/scripts
k8s-infra-cherrypick-robot d80686acb0
[release-2.26] Refactor and expand download_hash.py (#11538)
* download_hash.py: generalized and data-driven

The script is currently limited to one hardcoded URL for kubernetes
related binaries, and a fixed set of architectures.

The solution is three-fold:
1. Use an url template dictionary for each download -> this allow to easily
   add support for new downloads.
2. Source the architectures to search from the existing data
3. Enumerate the existing versions in the data and start searching from
   the last one until no newer version is found (newer in the version
   order sense, irrespective of actual age)

* download_hash.py: support for 'multi-hash' file + runc

runc upstream does not provide one hash file per assets in their
releases, but one file with all the hashes.
To handle this (and/or any arbitrary format from upstreams), add a
dictionary mapping the name of the download to a lambda function which
transform the file provided by upstream into a dictionary of hashes,
keyed by architecture.

* download_hash: argument handling with argparse

Allow the script to be called with a list of components, to only
download new versions checksums for those.
By default, we get new versions checksums for all supported (by the
script) components.

* download_hash: propagate new patch versions to all archs

* download_hash: add support for 'simple hash' components

* download_hash: support 'multi-hash' components

* download_hash: document missing support

* download_hash: use persistent session

This allows to reuse http connection and be more efficient.
From rough measuring it saves around 25-30% of execution time.

* download_hash: cache request for 'multi-hash' files

This avoid re-downloading the same file for different arch and
re-parsing it

* download_hash: document usage

---------

Co-authored-by: Max Gautier <mg@max.gautier.name>
2024-09-16 08:39:14 +01:00
..
gitlab-branch-cleanup Remove unused python imports (#7859) 2021-08-13 13:35:32 -07:00
openstack-cleanup fix openstack cleanup (#11299) 2024-06-21 10:30:55 -07:00
collect-info.yaml Do not use ‘yes/no’ for boolean values (#11472) 2024-08-28 06:30:56 +01:00
download_hash.py [release-2.26] Refactor and expand download_hash.py (#11538) 2024-09-16 08:39:14 +01:00
download_hash.sh bump skopeo version and checksum (#11044) 2024-04-16 01:19:27 -07:00
gen_docs_sidebar.sh sidebar_gen: force C locale 2024-05-19 12:04:07 +02:00
gen_tags.sh Fix gen_tags.sh script 2018-10-25 02:16:48 +08:00
gitlab-runner.sh Document gitlab-runner.sh 2017-02-13 15:04:35 +01:00
premoderator.sh Migrate CI_BUILD_ID to CI_JOB_ID and CI_BUILD_REF to CI_COMMIT_SHA (#10063) 2023-05-11 04:21:17 -07:00