Commit Graph

544 Commits (dd24e4cd90a68a9ba6e906ec448758b8d58970df)

Author SHA1 Message Date
Guillaume Abrioux b74ff6e22c rbd-mirror: major refactor
- Use config-key store to add cluster peer.
- Support multiple pools mirroring.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-07-29 17:33:25 +02:00
Guillaume Abrioux 3962cf6cb9 Revert "tests: temporarily disable nfs-ganesha"
This reverts commit 3f923d69ac.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-06-22 01:17:20 +02:00
Guillaume Abrioux aa68b06c99 ansible: bump to ansible 2.12
Add required changes to support ansible 2.12

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-06-15 08:09:10 +02:00
David Galloway bcedff95bd master->main
Signed-off-by: David Galloway <dgallowa@redhat.com>
2022-05-30 15:15:15 +02:00
Guillaume Abrioux ef0455a0b1 tests: update vagrant_box default value
This updates the default value for the vagrant_box variable
in all vagrant_variables.yml files

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-04-27 15:01:56 +02:00
David Galloway 1c740c424a tests/setup: Use local mirror of centos 8 stream repo
The mirrors provided by CentOS' mirrorlists are super slow

Signed-off-by: David Galloway <dgallowa@redhat.com>
2022-04-21 11:43:39 +02:00
Guillaume Abrioux 3e87df5e8f tests: update the system before deploying
Having a system up-to-date is usually a good idea.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-03-20 08:13:38 +01:00
Guillaume Abrioux bc36f60e8d tests: use centos stream-8 instead of centos 8
CentOS 8 is EOL as of December 2021.
Let's use CentOS stream 8 instead.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-01-31 15:58:58 +01:00
Guillaume Abrioux 3f923d69ac tests: temporarily disable nfs-ganesha
This commit [1] seems to have broken a selinux policy preventing nfs-ganesha from
starting properly.

Since we can't address the issue in ceph-ansible, let's disable temporarily nfs-ganesha testing.

[1] dae2da63d5

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-01-05 15:41:44 +01:00
Guillaume Abrioux fb8a66149b tests: add new scenario subset_update
new scenario in order to test the subset upgrade approach using tags.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-10-21 20:51:14 +02:00
Seena Fallah 5822936252 cephadm: install cephadm from repository
Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2021-10-08 16:56:47 +02:00
Seena Fallah 339212a7c6 cephadm-adopt: configure repository for cephadm installation
Configure repository for cephadm installation and use package install in both containerized and non containerized deployment

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2021-10-08 16:56:47 +02:00
Guillaume Abrioux c2e46fe5a5 tests: set rgw_instances in collect-logs.yml
in order to gather rgw logs, we need rgw_instances to be set.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-09-30 16:13:34 +02:00
Guillaume Abrioux b2ccc7234a tests: update collect-logs.yml playbook
- change `ceph -s` output to json-pretty.
- gather rgw logs
- add `health detail` command

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-09-30 08:31:22 +02:00
Guillaume Abrioux 702564518b tests: move collect-logs.yml to ceph-ansible repo
related ceph-build PR: ceph/ceph-build#1914

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-09-29 15:18:45 +02:00
Guillaume Abrioux b6c470c7e2 tests: add osd node in collocation
we update the pool size from 1 to 2 in idempotency test
but only 1 node is available.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-09-29 10:29:42 +02:00
Dimitri Savineau 9125bba48d tests: auth_allow_insecure_global_id_reclaim false
Otherwise the clients won't be able to reconnect after the reboot in the
all_daemons and collocation jobs.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-09-17 07:34:40 +02:00
Guillaume Abrioux 66f3eb377c tests: fix container-cephadm job
add missing variable `containerized_deployment` in group_vars

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-09-16 16:57:16 +02:00
Guillaume Abrioux c42ad1f487 Revert "tests: rename grafana to monitoring"
This reverts commit a36586a777.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-09-09 10:10:13 -04:00
Dimitri Savineau a36586a777 tests: rename grafana to monitoring
Since the grafana-server group has been renamed to monitoring then
changing the associated tests.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-09-09 13:27:27 +02:00
Dimitri Savineau f2bd8ae70f tests/rgw: use json format output for user info
If the radosgw user already exists then we need to have the output in json
format because we are expecting to load the output with json.loads()
Otherwise we have pytest failure like:

```console
self = <json.decoder.JSONDecoder object at 0x7fa2f00a5fd0>, s = '', idx = 0

    def raw_decode(self, s, idx=0):
        """Decode a JSON document from ``s`` (a ``str`` beginning with
        a JSON document) and return a 2-tuple of the Python
        representation and the index in ``s`` where the document ended.

        This can be used to decode a JSON document from a string that may
        have extraneous data at the end.

        """
        try:
            obj, end = self.scan_once(s, idx)
        except StopIteration as err:
>           raise JSONDecodeError("Expecting value", s, err.value) from None
E           json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-08-27 13:50:20 -04:00
Dimitri Savineau f01ae82eec tests/rgw: add timeout 5s to radosgw-admin command
If the radosgw daemons aren't up and running correctly (like not registered
in the servicemap or the OSD are down) then the radosgw-admin will hang
forever.
Jenkins will kill the jobs after 3h but we don't want to wait until this global
timeout.
Adding the timeout 5 command to the radosgw-admin commands (which is already
present on other ceph calls) allows the job to fail earlier.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-08-27 13:50:20 -04:00
Guillaume Abrioux 304d1cbb97 tests: remove legacy file
This inventory isn't used anywhere.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-06-29 14:53:19 +02:00
Guillaume Abrioux b49cdea750 tests: allocate more memory for all_in_one job
Since we fire up much less VMs than other job, we can affoard allocating
more memory here for this job.
Each VM hosts more daemon so 1024Mb can be too few.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-06-14 10:10:15 +02:00
Guillaume Abrioux d4dfa204d2 Revert "tests: disable test_mgr_dashboard_is_listening"
This reverts commit 2e19d1705e.

A new build of ceph@master including the fix is available so
this is not needed anymore.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-06-08 09:03:20 +02:00
Guillaume Abrioux 2e19d1705e tests: disable test_mgr_dashboard_is_listening
Due to a recent commit that has introduced a regression in ceph, this
test is failing.
Temporarily disabling it to unblock the CI.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-06-04 14:01:28 +02:00
Aksh Gupta a0bc5afe26 core: refactor code quality issues
The following commit fixes a few code quality issues detected by DeepSource:
- Use literals instead of function calls to create data structure.
- Refactor unnecessary list comprehension.
- Simplify if statement.
- Refactor useless else block in the loop.

Signed-off-by: Aksh Gupta <akshgpt7@gmail.com>
2021-03-25 14:57:02 +01:00
Guillaume Abrioux c90b0985e5 tests: switch to quay.ceph.io for dashboard images
for some reason, `quay.io/app-sre/grafana` no longer exist.
as a workaround, all dashboard related images have been mirrored on
quay.ceph.io.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-03-23 19:58:27 +01:00
Guillaume Abrioux b8080bac41 tests: fix `test_rgw_is_up` test
The data structure seems to have been modified in ceph@master (quincy).

This commit update the test accordingly.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-03-23 19:58:27 +01:00
Guillaume Abrioux 7e1db0b599 tests: fix `test_nfs_is_up` test
the data structure seems to have been modified in ceph@master (quincy).

This commit update the test accordingly.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-03-23 19:58:27 +01:00
Guillaume Abrioux ee1f0ce444 Revert "tests: disable nfs testing on master"
This reverts commit 8372b6792f.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-03-17 13:42:20 +01:00
Guillaume Abrioux 49668378fb tests: remove 1 client VM in external_clients job
We only use 2 client in this scenario, there's no need to fire up a
third VM.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-03-16 19:38:04 +01:00
Guillaume Abrioux 8372b6792f tests: disable nfs testing on master
nfs-ganesha builds in shaman are broken.
This commit disables nfs-ganesha testing in order to unlock the CI.

This is a temporary commit intented to be reverted.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-03-16 19:38:04 +01:00
Alex Schultz a7f2fa73e6 Use ansible_facts
It has come to our attention that using ansible_* vars that are
populated with INJECT_FACTS_AS_VARS=True is not very performant.  In
order to be able to support setting that to off, we need to update the
references to use ansible_facts[<thing>] instead of ansible_<thing>.

Related: ansible#73654
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1935406
Signed-off-by: Alex Schultz <aschultz@redhat.com>
2021-03-08 20:54:02 +01:00
Guillaume Abrioux 682116023d tests: increase `mon_max_pg_per_osd`
we aren't deploying enough OSD daemon, so it fails like following:

```
  stderr: 'Error ERANGE: pool id 10 pg_num 256 size 2 would mean 1536 total pgs, which exceeds max 1500 (mon_max_pg_per_osd 250 * num_in_osds 6)'
```

Let's increase the value of `mon_max_pg_per_osd` in order to get around
this issue in the CI.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-11 16:35:55 +01:00
Guillaume Abrioux 54bae480d2 tests: set `mon_max_pg_per_osd` in rgw_multisite
Otherwise, the job fails when it tries to create a bucket with `s3cmd mb`
command because we have too many PGs per OSD.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-10 07:01:21 +01:00
Guillaume Abrioux 7c9063b1d2 tests: use lvm batch on osd2 (all_daemons)
in order to test lvm batch in purge scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-02 17:24:17 +01:00
Guillaume Abrioux 71a5e666e3 rgw: multisite refact
Add the possibility to deploy rgw multisite configuration with a mix of
secondary and primary zones on a same rgw node.
Before that, on a same node, all instances were either primary
zones *OR* secondary.

Now you can define a rgw instance like following:

```
rgw_instances:
  - instance_name: 'rgw0'
    rgw_zonemaster: false
    rgw_zonesecondary: true
    rgw_zonegroupmaster: false
    rgw_realm: 'france'
    rgw_zonegroup: 'zonegroup-france'
    rgw_zone: paris-00
    radosgw_address: "{{ _radosgw_address }}"
    radosgw_frontend_port: 8080
    rgw_zone_user: jacques.chirac
    rgw_zone_user_display_name: "Jacques Chirac"
    system_access_key: P9Eb6S8XNyo4dtZZUUMy
    system_secret_key: qqHCUtfdNnpHq3PZRHW5un9l0bEBM812Uhow0XfB
    endpoint: http://192.168.101.12:8080
```

Basically it's now possible to define `rgw_zonemaster`,
`rgw_zonesecondary` and `rgw_zonegroupmaster` at the intsance
level instead of the whole node level.

Also, this commit adds an option `deploy_secondary_zones` (default True)
which can be set to `False` in order to explicitly ask the playbook to
not deploy secondary zones in case where the corresponding endpoint are
not deployed yet.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1915478

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-27 15:46:43 +01:00
Dimitri Savineau bbcad9609c grafana: update container tag to 6.7.4
This update the grafana container tag to 6.7.4.
The RHCS version is now based on the RHCS 5 container image which is
also based on 6.7.4.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-27 15:08:31 +01:00
Guillaume Abrioux 41314f49bf Revert "tests: temporarily use david's flavor"
This reverts commit ed9f0641ee.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-11 15:50:55 +01:00
Guillaume Abrioux ed9f0641ee tests: temporarily use david's flavor
master nfs ganesha builds are broken, let's use this flavor instead for
now.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-07 17:09:46 -05:00
Guillaume Abrioux e2ea403d5e tests: rgw_multisite playbook test refactor
Currently we create an object from the primary sites but we try to read
that object still from the master which doesn't make sense, we should
try to read it from a secondary site.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-12-14 17:53:21 +01:00
Dimitri Savineau 3b9cdc8502 tests: remove pyyaml workaround on OSD nodes
Since [1] has been resolved then we don't need to apply this workaround
anymore.

[1] https://tracker.ceph.com/issues/46759

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-10 09:15:03 +01:00
Dimitri Savineau c3ed124d31 library: add cephadm_bootstrap module
This adds cephadm_bootstrap ansible module for replacing the command module
usage with the cephadm bootstrap command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 10:30:05 +01:00
Guillaume Abrioux e5713ea5d5 tests: change cephfs pool size
`all_daemons` scenario can't handle pools with `size: 3` because we have
1 osd node in root=HDD and two nodes in root=default.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-06 09:23:52 -04:00
Guillaume Abrioux 8596f1d52c flake8: fix pep8 syntax on tests/functional/tests/
tests/conftest.py and tests present in tests/functional/tests/ has been
missed from previous commit

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-06 08:00:06 +02:00
Guillaume Abrioux f83f798206 tests: reboot and test idempotency on collocation
test reboot and idempotency on collocation scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-06 07:38:44 +02:00
Guillaume Abrioux 876b4ad248 tests: remove ooo_collocation job
This job is redundant with 'collocation' job.
The only difference is osd/rgw collocation so let's add this usecase in
'collocation'.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 19d683d7acfb5344b38ac1ba4c123dcdd4d80f35)
2020-10-04 11:19:15 +02:00
Dimitri Savineau 246e31c0d3 Revert "tests: disable nfs-ganesha testing"
This reverts commit 7348e9a253.

Since the nfs-ganesha rpm build for CentOS 8 has been fixed, and
the nfs-ganesha segfault caused by an issue in librgw has also been
fixed.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-10-02 07:43:07 +02:00
Guillaume Abrioux c101cb3931 defaults: change defaults value
this commit changes defaults value in default pool definitions.

there's no need to define `pg_num`, `pgp_num`, `size` and `min_size`,
`ceph_pool` module will use the current default if needed.

This also drops the 3 following `set_fact` in `ceph-facts`:

- osd_pool_default_pg_num,
- osd_pool_default_pgp_num,
- osd_pool_default_size_num

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-02 07:42:40 +02:00