Commit Graph

623 Commits (86a8889ee3adffc81d3435895a7a117824e779ad)

Author SHA1 Message Date
Dimitri Savineau 2e417ab901 library: add ceph_crush_rule module
This adds ceph_crush_rule ansible module for replacing the command
module usage with the ceph osd crush rule commands.
This module can manage both erasure and replicated crush rules.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 17:52:41 +01:00
Dimitri Savineau c3ed124d31 library: add cephadm_bootstrap module
This adds cephadm_bootstrap ansible module for replacing the command module
usage with the cephadm bootstrap command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 10:30:05 +01:00
Dimitri Savineau 5da593604a library: add ceph_osd_flag module
This adds ceph_osd_flag ansible module for replacing the command module
usage with the ceph osd set/unset commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 10:29:11 +01:00
Dimitri Savineau 0b5b1de963 library: add ceph_osd module
This adds ceph_osd ansible module for replacing the command module
usage with the ceph osd destroy/down/in/out/purge/rm commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 16:53:45 +01:00
Dimitri Savineau eaf0ebfc85 library: add ceph_mgr_module module
This adds ceph_mgr_module ansible module for replacing the command module
usage with the ceph mgr module enable/disable commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 16:52:02 +01:00
Dimitri Savineau c610a1b411 tests: use ca_test_common with ceph_volume_simple
To avoid duplicate code.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 10:05:45 +01:00
Dimitri Savineau d76fbb38d5 improve plugins/filter testing
- The plugins/filter directory wasn't present in the flake8 workflow
configuration.
- Fix the flake8 syntax.
- Add the directory to PYTHONPATH environment variable for pytest
to avoid importing the plugin filter via sys.
- Add unittest on missing netaddr module import.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 10:05:01 +01:00
Dimitri Savineau 239a14279d tests: add module_utils directory to flake8/pytest
This adds the module_utils and associated test directory into the flake8
and pytest workflow configuration.
It also moves the ca_common module_utils test file from tests/library to
it's own directory tests/module_utils.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-27 06:35:56 +01:00
Guillaume Abrioux 67467fd87c module_utils: refactor
- update `generate_ceph_cmd()` so `user_key` is automatically built from
`cluster` and `user` params.

- update and add testing.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-26 11:21:09 +01:00
Guillaume Abrioux 926a1b2f7b library: use exit_module from ca_common
pr #6054 got merged but could have been updated to use `ca_common` for
`exit_module()`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-26 09:13:31 +01:00
Dimitri Savineau 3baac5ad5b library: add ceph_volume_simple_{activate,scan}
This adds ceph_volume_simple_{activate,scan} ansible modules for replacing
the command module usage with the ceph-volume simple activate/scan commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-25 10:09:42 +01:00
Guillaume Abrioux 970c6a4ee6 mon: refact initial keyring generation
adding monitor is no longer possible because we generate a new mon
keyring each time the playbook is run.

Fixes: #5864

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-25 09:34:44 +01:00
Guillaume Abrioux 7c72632418 library: import ca_test_common in test_ceph_key
since we added `ca_test_common.py` let's use it in `test_ceph_key.py`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-24 10:38:28 +01:00
Guillaume Abrioux 497d27dcf1 library: add ec_profile module
This commit adds a new module `ceph_ec_profile` to manage erasure code
profiles.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-24 10:38:28 +01:00
Guillaume Abrioux d7fd46842d library: add module_utils namespace
This adds a new `module_utils` namespace in order to avoid defining same
functions in each module.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-24 10:38:28 +01:00
Dimitri Savineau 3e79f0322a tests: use github workflow for pytest
Move the pytest testing from TravisCI to Github workflow.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-17 17:52:03 +01:00
Dimitri Savineau 7d3d51d6da library/ceph_key: add output format parameter
The ceph_key module currently only supports the json output for the
info state.
When using this state on an entity then we something want the output
as:
  - plain for copying it to another node.
  - json in order to get only a subset information of the entity (like
the key or caps).

This patch adds the output_format parameter which uses json as a
default value for backward compatibility. It removes the internal and
hardcoded variable also called output_format.
In addition of json and plain outputs, there's also xml and yaml
values available.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-02 17:17:29 +01:00
Guillaume Abrioux e5713ea5d5 tests: change cephfs pool size
`all_daemons` scenario can't handle pools with `size: 3` because we have
1 osd node in root=HDD and two nodes in root=default.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-06 09:23:52 -04:00
Dimitri Savineau 1281e8bcc8 library: add radosgw_zone module
This adds radosgw_zone ansible module for replacing the command module
usage with the radosgw-admin zone command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-10-06 10:07:58 +02:00
Dimitri Savineau 65dbe0782e library: add radosgw_zonegroup module
This adds radosgw_zonegroup ansible module for replacing the command
module usage with the radosgw-admin zonegroup command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-10-06 10:07:58 +02:00
Dimitri Savineau d171f4068d library: add radosgw_realm module
This adds radosgw_realm ansible module for replacing the command module
usage with the radosgw-admin realm command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-10-06 10:07:58 +02:00
Dimitri Savineau 235c7e27cc library: add radosgw_user module
This adds radosgw_user ansible module for replacing the command module
usage with the radosgw-admin user command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-10-06 10:07:58 +02:00
Dimitri Savineau bd611a785b library: add ceph_fs module
This adds the ceph_fs ansible module for replacing the command module
usage with the ceph fs command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-10-06 08:02:58 +02:00
Guillaume Abrioux 8596f1d52c flake8: fix pep8 syntax on tests/functional/tests/
tests/conftest.py and tests present in tests/functional/tests/ has been
missed from previous commit

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-06 08:00:06 +02:00
Guillaume Abrioux 12e6260266 ceph_key: support using different keyring
Currently the `ceph_key` module doesn't support using a different
keyring than `client.admin`.
This commit adds the possibility to use a different keyring.

Usage:
```
      ceph_key:
        name: "client.rgw.myrgw-node.rgw123"
        cluster: "ceph"
        user: "client.bootstrap-rgw"
        user_key: /var/lib/ceph/bootstrap-rgw/ceph.keyring
        dest: "/var/lib/ceph/radosgw/ceph-rgw.myrgw-node.rgw123/keyring"
        caps:
          osd: 'allow rwx'
          mon: 'allow rw'
          import_key: False
        owner: "ceph"
        group: "ceph"
        mode: "0400"
```

Where:
`user` corresponds to `-n (--name)`
`user_key` corresponds to `-k (--keyring)`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-06 07:38:44 +02:00
Guillaume Abrioux f83f798206 tests: reboot and test idempotency on collocation
test reboot and idempotency on collocation scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-06 07:38:44 +02:00
Guillaume Abrioux 876b4ad248 tests: remove ooo_collocation job
This job is redundant with 'collocation' job.
The only difference is osd/rgw collocation so let's add this usecase in
'collocation'.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 19d683d7acfb5344b38ac1ba4c123dcdd4d80f35)
2020-10-04 11:19:15 +02:00
Guillaume Abrioux e49a5241f0 flake8: fix all tests/library/*.py files
This commit modifies all *.py files in ./tests/library/ so flake8
passes.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-02 08:51:39 +02:00
Dimitri Savineau 246e31c0d3 Revert "tests: disable nfs-ganesha testing"
This reverts commit 7348e9a253.

Since the nfs-ganesha rpm build for CentOS 8 has been fixed, and
the nfs-ganesha segfault caused by an issue in librgw has also been
fixed.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-10-02 07:43:07 +02:00
Guillaume Abrioux c101cb3931 defaults: change defaults value
this commit changes defaults value in default pool definitions.

there's no need to define `pg_num`, `pgp_num`, `size` and `min_size`,
`ceph_pool` module will use the current default if needed.

This also drops the 3 following `set_fact` in `ceph-facts`:

- osd_pool_default_pg_num,
- osd_pool_default_pgp_num,
- osd_pool_default_size_num

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-02 07:42:40 +02:00
Guillaume Abrioux 8f5db079ae ceph_pool: update tests
update test_ceph_pool.py due to recent refact

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-02 07:42:40 +02:00
Guillaume Abrioux 787878f0c3 ceph_pool: pep8
Adopt pep8 syntax in ceph_pool module

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-02 07:42:40 +02:00
Guillaume Abrioux 29fc115f4a ceph_pool: refact module
remove complexity about current defaults in running cluster

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-10-02 07:42:40 +02:00
Guillaume Abrioux eefe11d90c defaults: change default grafana-server name
This change default value of grafana-server group name.
Adding some tasks in ceph-defaults in order to keep backward
compatibility.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-29 07:42:26 +02:00
Dimitri Savineau e11453c6f5 Remove unused centos docker tasks
The `enable extras on centos` task just doesn't work when using the
variable ceph_docker_enable_centos_extra_repo to true.

fatal: [xxx]; FAILED! => {"changed": false, "msg": "Parameter
'baseurl', 'metalink' or 'mirrorlist' is required."}

The CentOS extras repository is enabled by default so it's pretty
safe to remove this task and the associated variable.

This also removes the ceph_docker_on_openstack variable as it's a
leftover and it is unused.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-09-29 07:35:10 +02:00
Dimitri Savineau 7dfa205610 tests: disable container nfs testing
Looks like nfs-ganesha 3.3 and 4.-dev doesn't work with recent changes
in librgw 16.0.0.
The nfs-ganesha daemon is segfaulting and restart in a loop.

See https://tracker.ceph.com/issues/47520

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-09-17 16:51:33 -04:00
Dimitri Savineau 3ba11c1434 tests/library: rename ceph_dashboard_user class
Rename the test class with the right information.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-09-14 15:17:20 -04:00
Guillaume Abrioux 5650a6d7d0 tests: do not run node_exporter test on clients
We need to skip these tests on client nodes since we don't deploy
node_exporter on them anymore

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-14 14:12:21 -04:00
Dimitri Savineau ee6f0547ba library: add ceph_dashboard_user module
This adds the ceph_dashboard_user ansible module for replacing the
command module usage with the ceph dashboard ac-user-xxx command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-09-11 10:16:08 +02:00
Dimitri Savineau 78cb9f44bd tests: add quay registry for collocation baremetal
Even if the non containerized collocation scenario deploys ceph with
RPMs then we also deploy the dashboard/monitoring but with containers.
This requires to set the registry variable to ceph's quay.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-09-10 14:23:21 -04:00
Dimitri Savineau 98c9afceb9 tests: use grafana from quay.io
This changes the grafana container image regitry from docker.io to
quay.io to avoid rate limit.
This also adds the missing container image values for docker2podman
and podman scenarios.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-09-09 10:35:02 -04:00
Guillaume Abrioux 657e6c8c3b tests: clean legacy
clean some legacies since quay.ceph.io migration

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-09 14:42:41 +02:00
Guillaume Abrioux 7348e9a253 tests: disable nfs-ganesha testing
This commit diables nfs-ganesha testing on master for non-containerized
deployment because the dev repos are broken at the moment.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-07 12:54:29 +02:00
Guillaume Abrioux 2cbb7de3b2 tests: migrate to quay.ceph.io registry
in order to avoid docker.io rate limiting

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-09-07 12:54:29 +02:00
Dimitri Savineau 4f308dcf4a tests: reenable ceph-iscsi testing
This re-adds the ceph-iscsi testing for both non containerized and
containerized deployment since the rados connection error on ceph
dev has been fixed [1].

[1] https://tracker.ceph.com/issues/47002

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-08-27 11:13:36 -04:00
Guillaume Abrioux 27ca884d99 tests: add more coverage for test_ceph_key
This commit adds more coverage regarding the testing of ceph_key module

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-21 09:44:47 +02:00
Dimitri Savineau 6c11695fbe tests: reenable nfs-ganesha testing
This re-adds the nfs-ganesha testing in non containerized deployment.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-08-20 16:58:54 +02:00
Guillaume Abrioux 8476beb5b1 tests: move erasure pool testing in lvm_osds
This commit moves the erasure pool creation testing from `all_daemons`
to `lvm_osds` so we can decrease the number of osd nodes we spawn so the
OVH Jenkins slaves aren't less overwhelmed when a `all_daemons` based
scenario is being tested.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-20 11:50:28 +02:00
Guillaume Abrioux 093e1dcb21 tests: remove hosts-ubuntu inventories
Since we've dropped ubuntu testing, we don't need these inventories
anymore. Let's remove this leftover.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-18 11:20:48 +02:00
Guillaume Abrioux bd9e126357 tests: disable iscsigw testing (container)
Temporarily disable iscsigw testing for containerized deployments
because it's broken upstream on ceph@master.
non-containerized deployments use stable build for iscsigw to get around
this issue.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-18 11:20:48 +02:00