Commit Graph

255 Commits (65b8f46a432a9c5b70537dc7cb8565a1eb689cc7)

Author SHA1 Message Date
Dimitri Savineau a71a80c167 tox: remove ceph_dev variables and dev_setup calls
Since pacific is a stable release then we don't need to use shaman for getting
the pacific build.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-06-30 09:29:58 +02:00
Guillaume Abrioux ab857d8b54 tests: use master build for iscsigws
pacific builds for iscsi pkgs aren't available, as a workaround we can
use builds from master.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-03-24 21:36:24 +01:00
Guillaume Abrioux 6832c8d7a5 tests: increase nb of rerun in pytest
In order to avoid false positive in the CI that I've been unable to
reproduce.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit f7fd1c2298)
2021-03-03 14:12:46 +01:00
Guillaume Abrioux 7dd4a8a059 tests: use shaman to test against ceph pacific
Given there's no pacific packages available at
https://download.ceph.com, let's use shaman in order to test against
Ceph Pacific

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-12 09:15:24 +01:00
Guillaume Abrioux 8eda590130 tests: remove legacy
remove a legacy in tox environment definition

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-06 00:34:14 +01:00
Guillaume Abrioux c3eadbc31a tests: follow up on 7c9063b
7c9063b1d2 broke some scenarios.
This commit fixes them.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-06 00:34:14 +01:00
Guillaume Abrioux 7c9063b1d2 tests: use lvm batch on osd2 (all_daemons)
in order to test lvm batch in purge scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-02 17:24:17 +01:00
Dimitri Savineau 523966d45f tox: test cephadm-adopt.yml playbook idempotency
Rerun the cephadm-adopt.yml playbook a second time for idempotency
purpose.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-29 21:07:39 +01:00
Guillaume Abrioux 71a5e666e3 rgw: multisite refact
Add the possibility to deploy rgw multisite configuration with a mix of
secondary and primary zones on a same rgw node.
Before that, on a same node, all instances were either primary
zones *OR* secondary.

Now you can define a rgw instance like following:

```
rgw_instances:
  - instance_name: 'rgw0'
    rgw_zonemaster: false
    rgw_zonesecondary: true
    rgw_zonegroupmaster: false
    rgw_realm: 'france'
    rgw_zonegroup: 'zonegroup-france'
    rgw_zone: paris-00
    radosgw_address: "{{ _radosgw_address }}"
    radosgw_frontend_port: 8080
    rgw_zone_user: jacques.chirac
    rgw_zone_user_display_name: "Jacques Chirac"
    system_access_key: P9Eb6S8XNyo4dtZZUUMy
    system_secret_key: qqHCUtfdNnpHq3PZRHW5un9l0bEBM812Uhow0XfB
    endpoint: http://192.168.101.12:8080
```

Basically it's now possible to define `rgw_zonemaster`,
`rgw_zonesecondary` and `rgw_zonegroupmaster` at the intsance
level instead of the whole node level.

Also, this commit adds an option `deploy_secondary_zones` (default True)
which can be set to `False` in order to explicitly ask the playbook to
not deploy secondary zones in case where the corresponding endpoint are
not deployed yet.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1915478

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-27 15:46:43 +01:00
Dimitri Savineau fa2bb3af86 switch2container: disable ceph-osd enabled-runtime
When deploying the ceph OSD via the packages then the ceph-osd@.service
unit is configured as enabled-runtime.
This means that each ceph-osd service will inherit from that state.
The enabled-runtime systemd state doesn't survive after a reboot.
For non containerized deployment the OSD are still starting after a
reboot because there's the ceph-volume@.service and/or ceph-osd.target
units that are doing the job.

$ systemctl list-unit-files|egrep '^ceph-(volume|osd)'|column -t
ceph-osd@.service     enabled-runtime
ceph-volume@.service  enabled
ceph-osd.target       enabled

When switching to containerized deployment we are stopping/disabling
ceph-osd@XX.servive, ceph-volume and ceph.target and then removing the
systemd unit files.
But the new systemd units for containerized ceph-osd service will still
inherit from ceph-osd@.service unit file.

As a consequence, if an OSD host is rebooting after the playbook execution
then the ceph-osd service won't come back because they aren't enabled at
boot.

This patch also adds a reboot and testinfra run after running the switch
to container playbook.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1881288

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-12 20:05:39 +01:00
Guillaume Abrioux 1cc9666c09 common: drop `fetch_directory` feature
This commit drops the `fetch_directory` feature.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-21 13:22:16 +02:00
Guillaume Abrioux 20718582da infrastructure-playbooks: drop add-osd playbook
This playbook isn't needed anymore, we can achieve this operation by
running main playbook with `--limit` option.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-06 08:54:16 +02:00
Guillaume Abrioux f83f798206 tests: reboot and test idempotency on collocation
test reboot and idempotency on collocation scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-06 07:38:44 +02:00
Guillaume Abrioux 876b4ad248 tests: remove ooo_collocation job
This job is redundant with 'collocation' job.
The only difference is osd/rgw collocation so let's add this usecase in
'collocation'.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 19d683d7acfb5344b38ac1ba4c123dcdd4d80f35)
2020-10-04 11:19:15 +02:00
Guillaume Abrioux e835c77a0e tests: remove sleep commands from tox ini files
Since we use the rerun plugin in tox, we shouldn't need to add these
`sleep` commands.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-30 17:09:42 -04:00
Guillaume Abrioux 657e6c8c3b tests: clean legacy
clean some legacies since quay.ceph.io migration

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-09 14:42:41 +02:00
Guillaume Abrioux 2cbb7de3b2 tests: migrate to quay.ceph.io registry
in order to avoid docker.io rate limiting

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-07 12:54:29 +02:00
Guillaume Abrioux 83d1b33a9b tox: only wait 30sec for right jobs
There's no need to call `sleep 30` for other job than `all_daemons` and
`all_in_one`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-06 17:22:52 +02:00
Guillaume Abrioux 44caa062b7 tox: remove ubuntu references
since we've dropped ubuntu testing on PRs and nightlies, we don't need
these references anymore in tox files.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-22 18:57:31 -04:00
Dimitri Savineau 14eed63921 tox: add cephadm_adopt scenario
This adds an optional cephadm_adopt scenario which is based on
all_daemons.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-10 16:00:24 +02:00
Guillaume Abrioux 40307f810c tests: add docker hub authentication in jobs
This commit makes all jobs authenticating to docker hub in order to
avoid the rate limit.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-08 09:52:53 -04:00
Guillaume Abrioux 4fb9722c48 tests: force using the more recent build
We should use  `latest-master-devel` for switch_to_containers job.
Otherwise it might happen we actually downgrade the ceph version when
the image used is older than the rpm initially used for installing ceph.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-05-14 11:34:51 -04:00
Guillaume Abrioux 6d9acb5e6d test: set sitepackages=false in tox
Otherwise it might try to use the system installed version of ansible
when there's one available.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-05-14 13:09:42 +02:00
Dimitri Savineau 92f538f1af tox: replace testinfra by pytest for add-mgrs
The add-mgrs scenario is still using the testinfra command instead of
pytest so the tests exectution are failling.

ERROR: InvocationError for command could not find executable testinfra

This also adds the missing --ssh-config option to testinfra.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-04-03 09:59:06 +02:00
Guillaume Abrioux 9219991441 remove *docker*.yml symlinks
This commits removes these two symlinks.
They were there for backward compatibility and were marked deprecated as
of stable-4.0

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-03-31 10:03:22 -04:00
Guillaume Abrioux 3788826371 tests: remove some legacy in tox.ini
This commit removes some leftover in tox.ini

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-03-25 12:44:31 +01:00
Dimitri Savineau 2f4413f5ce tox: update shrink scenario configuration
The shrink scenarios don't need the docker variables (except for OSD).
Removing pytest for shrink-mgr.
Adding environment variables for xxx_to_kill ansible variable.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-03-03 10:32:15 +01:00
Guillaume Abrioux 3e7dbb4b16 tests: add 'all_in_one' scenario
Add new scenario 'all_in_one' in order to catch more collocated related
issues.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-01-27 15:30:45 -05:00
Dimitri Savineau 3cc7d5651c tox: use vagrant_up.sh instead of vagrant up
We should use the same vagrant wrapper everywhere instead of the vagrant
command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-01-15 17:52:35 +01:00
Guillaume Abrioux fef1cd4c4b tests: use main playbook for add_osds job
This commit replaces the playbook used for add_osds job given
accordingly to the add-osd.yml playbook removal

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-01-13 09:59:08 -05:00
Guillaume Abrioux 217d95abb2 common: add centos8 support
Ceph octopus only supports CentOS 8.

This commit adds CentOS 8 support:
  - update vagrant image in tox configurations.
  - add CentOS 8 repository for el8 dependencies.
  - CentOS 8 container engine is podman (same than RHEL 8).
  - don't use the epel mirror on sepia because it's epel7 only.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
Co-authored-by: Dimitri Savineau <dsavinea@redhat.com>
2020-01-08 11:13:46 +01:00
Guillaume Abrioux 7bc7e3669d purge: rename playbook (container)
Since we now support podman, let's rename the playbook so it's more
generic.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-12-03 11:10:21 -05:00
Guillaume Abrioux db77fbda15 tests: add coverage on purge playbook
This commit adds a playbook to be played before we run purge playbook,
it first creates an rbd image then map an rbd device on client0 so the
purge playbook will try to unmap it.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-11-08 09:06:11 -05:00
Dimitri Savineau 3c2840da03 tests: update container tag for ooo_collocation
It doesn't make sense to test the old 3.0.x container images with
nautilus+ ceph releases.
Also disable the dashboard deployment and switch to bluestore backend.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-10-22 13:45:19 +02:00
Guillaume Abrioux bcaf8cedee tests: update tox due to pipeline removal
This commit reflects the recent changes in ceph/ceph-build#1406

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-10-08 17:35:25 +02:00
Dimitri Savineau 010158ff84 tests: fix rgw multisite vagrant variables
The secondary vagrant variables didn't have the grafana vm variable
set which create an vagrant error.

There was an error loading a Vagrantfile. The file being loaded
and the error message are shown below. This is usually caused by
an invalid or undefined variable.

This patch also changes the ssh-extra-args parameter to ssh-common-args
to get the same values for ssh/sftp/scp. Otherwise we can see warnings
from ansible and some tasks are failing.

[WARNING]: sftp transfer mechanism failed on [mon0]. Use ANSIBLE_DEBUG=1
to see detailed information

It also updates the ssh-common-args value for the rgw-multisite scenario
to reflect the ANSIBLE_SSH_ARGS environment variable value.

Finally changing the IP addresses due to the Vagrant refact done in the
commit 778c51a

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-10-04 15:12:50 -04:00
Harald Jensås e695efcaf7 Replace ipaddr() with ips_in_ranges()
This change implements a filter_plugin that is used in the
ceph-facts, ceph-validate roles and infrastucture-playbooks.
The new filter plugin will return a list of all IP address
that reside in any one of the given IP ranges. The new filter
replaces the use of the ipaddr filter.

ceph.conf already support a comma separated list of CIDRs
for the public_network and cluster_network options.

Changes: [1] and [2] introduced a regression in ceph-ansible
where public_network can no longer be a comma separated list
of cidrs.

With this change a comma separated list of subnet CIDRs can
also be used for monitor_address_block and radosgw_address_block.

[1] commit: d67230b2a2
[2] commit: 20e4852888

Related-To: https://bugs.launchpad.net/tripleo/+bug/1840030
Related-To: https://bugzilla.redhat.com/show_bug.cgi?id=1740283

Closes: #4333
Please backport to stable-4.0

Signed-off-by: Harald Jensås <hjensas@redhat.com>
2019-09-27 10:11:53 +02:00
Guillaume Abrioux 5bb6a4da42 tests: set copy_admin_key at group_vars level
setting it at extra vars level prevent from setting it per node.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-09-26 11:35:24 +02:00
Dimitri Savineau 7bd03c95f6 tox: Fix incorrect ANSIBLE_CONFIG value
The ANSIBLE_CONFIG value wasn't set correctly for two scenarios. This
environment variable doesn't use '-F'.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-09-09 11:23:30 -04:00
Guillaume Abrioux e0ba011f50 tests: change container image tag for switch_to_containers
test switch_to_containers job against the latest ceph@master
ceph-container image tag available.
In order to be sure the ceph release deployed in the first step (non
containerized deployment) isn't newer than the tag used for the
containerized migration (which would mean we try to downgrade the
version).

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-28 10:48:21 -04:00
Guillaume Abrioux 5e33d224d3 tests: tests switch_to_containers against octopus
since we have container images for ceph@master, we shouldn't use
nautilus anymore.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-14 16:42:02 +02:00
Igor 2fdf7316a4 tests: fix wrong paths for lv-create in tox.ini
solution: change paths inside tox.ini file
Fixes: #4311
Signed-off-by: Bogomolov Igor <igor95n@gmail.com>
2019-08-08 13:55:41 +02:00
Rishabh Dave 236b081a3a tests/functional: add a test for shrink-rgw.yml
Add a new functional test that deploys a Ceph cluster with three nodes
for MON, OSD and RGW and then runs shrink-rgw.yml to test it.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-30 08:45:57 +02:00
Guillaume Abrioux 87b173d022 tests: remove useless setting
this setting is not needed here since we explicitely set it for
container and non container context.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-07-17 09:13:29 +02:00
Rishabh Dave f80521f773 tests/functional: add a test for shrink-rbdmirror.yml
Add a new functional test that deploys Ceph cluster with three nodes for
MON, OSD and RBD Mirror and, then, runs shrink-rbdmirror.yml to test it.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-15 11:22:17 +02:00
Rishabh Dave 5c95c34d4b tests/functional: add a test for shrink-mgr.yml
Add a new functional test that deploys a Ceph cluster with three nodes
for MON, OSD and MGR and then runs shrink-mgr.yml to test it.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-09 14:37:02 +02:00
Rishabh Dave 324b3b4a6c tests/functional: add a test for shrink-mds.yml
Add a new functional test that deploys a Ceph cluster with three nodes
for MON, OSD and MDS and then runs shrink-mds.yml to test it.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-07-08 11:05:28 +02:00
Guillaume Abrioux ca84a5359f tests: wait 30sec before running testinfra
adding back a sleep 30s after nodes have rebooted before running
testinfra.
This was removed accidentally by d5be83e

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-07-03 16:14:38 +02:00
Dimitri Savineau 34f9d51178 tests: Update ansible ssh_args variable
Because we're using vagrant, a ssh config file will be created for
each nodes with options like user, host, port, identity, etc...
But via tox we're override ANSIBLE_SSH_ARGS to use this file. This
remove the default value set in ansible.cfg.

Also adding PreferredAuthentications=publickey because CentOS/RHEL
servers are configured with GSSAPIAuthenticationis enabled for ssh
server forcing the client to make a PTR DNS query.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-06-17 09:24:24 +02:00
Guillaume Abrioux a78fb209b1 tests: test podman against atomic os instead rhel8
the rhel8 image used is an outdated beta version, it is not worth it to
maintain this image upstream, since it's possible to test podman with a
newer version of centos/atomic-host image.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-06-04 11:32:41 -04:00