Commit Graph

5831 Commits (df0901e8799c289f10e312a964b162fe10768b2a)
 

Author SHA1 Message Date
Dimitri Savineau 801e7a29cf tests/vagrant: update box version to CentOS 8.3
This updates the CentOS libvirt box version to 8.3

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-09 14:46:45 -05:00
Dimitri Savineau a2cbab16a4 rhcs: drop fetch_directory override
Since the fetch_directory variable has been dropped then we don't need
the override in rhcs file.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-03 10:48:21 +01:00
Jukka Nousiainen eb7473491b ceph-mon: No become during gen mon initial keyring
Since the backing generate_secret() just hands out urandom output,
running as privileged doesn't seem to be required. It's not
desireable to provide sudo in some Ansible runner environments.

Signed-off-by: Jukka Nousiainen <jukka.nousiainen@csc.fi>
2020-12-03 10:04:21 +01:00
Dimitri Savineau 08f118077f library: add cephadm_adopt module
This adds cephadm_adopt ansible module for replacing the command module
usage with the cephadm adopt command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-02 09:15:44 +01:00
Guillaume Abrioux 86a8889ee3 common: do not use pipefail when not needed
Let's discard the ansible lint error 306 and add a "# noqa 306" on tasks
where we don't need `set -o pipefail`

Fixes: #6090

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-12-01 15:07:09 -05:00
Dimitri Savineau cf7345f143 consume ceph_volume module when possible
We should always use the ceph_volume ansible module when possible.
This patch replace the ceph-volume inventory and lvm {list,zap} commands
called via the command/shell modules by the corresponding call with the
ceph_volume module.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 17:54:10 +01:00
Dimitri Savineau 2e417ab901 library: add ceph_crush_rule module
This adds ceph_crush_rule ansible module for replacing the command
module usage with the ceph osd crush rule commands.
This module can manage both erasure and replicated crush rules.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 17:52:41 +01:00
Guillaume Abrioux 5c4ae5356d osd: add tag on 'wait for all osd to be up' task
This allows skipping this task if really desired.
Use it carefully. Use it at your own risk.

Fixes: #6073

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-12-01 11:00:25 +01:00
Dimitri Savineau 1831b4955f ceph-client: use group_by instead of add_host
Instead of iterate over all client nodes with a loop sequentially, we
can use the group_by ansible buildin.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 10:58:48 +01:00
Dimitri Savineau c3ed124d31 library: add cephadm_bootstrap module
This adds cephadm_bootstrap ansible module for replacing the command module
usage with the cephadm bootstrap command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 10:30:05 +01:00
Dimitri Savineau 5da593604a library: add ceph_osd_flag module
This adds ceph_osd_flag ansible module for replacing the command module
usage with the ceph osd set/unset commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 10:29:11 +01:00
Guillaume Abrioux d40dd764e0 iscsigw: remove `--cap-add=all` from `podman run` cmd
As of podman `2.0.5`, `--cap-add` and `--privileged` are exclusive
options.

```
Nov 30 13:56:30 magna089 podman[171677]: Error: invalid config provided: CapAdd and privileged are mutually exclusive options
```

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1902149

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-30 12:24:11 -05:00
Guillaume Abrioux c68b124ba8 container: remove `--ignore` from `podman rm` command
As of podman 2.0.5, `--ignore` param conflicts with `--storage`.
```
Nov 30 13:53:10 magna089 podman[164443]: Error: --storage conflicts with --volumes, --all, --latest, --ignore and --cidfile
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-30 12:24:11 -05:00
Dimitri Savineau 0b5b1de963 library: add ceph_osd module
This adds ceph_osd ansible module for replacing the command module
usage with the ceph osd destroy/down/in/out/purge/rm commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 16:53:45 +01:00
Dimitri Savineau eaf0ebfc85 library: add ceph_mgr_module module
This adds ceph_mgr_module ansible module for replacing the command module
usage with the ceph mgr module enable/disable commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 16:52:02 +01:00
Dimitri Savineau c610a1b411 tests: use ca_test_common with ceph_volume_simple
To avoid duplicate code.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 10:05:45 +01:00
Dimitri Savineau d76fbb38d5 improve plugins/filter testing
- The plugins/filter directory wasn't present in the flake8 workflow
configuration.
- Fix the flake8 syntax.
- Add the directory to PYTHONPATH environment variable for pytest
to avoid importing the plugin filter via sys.
- Add unittest on missing netaddr module import.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 10:05:01 +01:00
Guillaume Abrioux 0b05620597 switch2containers: do not stop ceph.target in osd play
`ceph.target` should be disabled only. Otherwise, in collocation
scenario you stop other collocated services in the OSD play which isn't
what we want to do. Each daemon has its corresponding play for managing
the transition to container.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1901865

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-27 09:34:01 -05:00
Dimitri Savineau eb452d35bc alertmanager/prometheus: fix owner/group
Set the owner/group on alertmanager and prometheus directories and
files to nobody and nogroup (uid and gid 65534) to avoid permission
issues.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1901543

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-27 14:02:42 +01:00
Dimitri Savineau 239a14279d tests: add module_utils directory to flake8/pytest
This adds the module_utils and associated test directory into the flake8
and pytest workflow configuration.
It also moves the ca_common module_utils test file from tests/library to
it's own directory tests/module_utils.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-27 06:35:56 +01:00
Guillaume Abrioux 67467fd87c module_utils: refactor
- update `generate_ceph_cmd()` so `user_key` is automatically built from
`cluster` and `user` params.

- update and add testing.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-26 11:21:09 +01:00
Guillaume Abrioux 926a1b2f7b library: use exit_module from ca_common
pr #6054 got merged but could have been updated to use `ca_common` for
`exit_module()`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-26 09:13:31 +01:00
Dimitri Savineau 0078230f1d rhcs: default to containerized deployment
Starting RHCS 5, only containerized deployment is available.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-25 10:46:16 +01:00
Dimitri Savineau 0cc34fa54e library: remove useless condition in ceph_volume
Since the action values are already defined as a list of choices in
ansible then we will never enter into this condition.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-25 10:44:48 +01:00
Dimitri Savineau 3baac5ad5b library: add ceph_volume_simple_{activate,scan}
This adds ceph_volume_simple_{activate,scan} ansible modules for replacing
the command module usage with the ceph-volume simple activate/scan commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-25 10:09:42 +01:00
Guillaume Abrioux 970c6a4ee6 mon: refact initial keyring generation
adding monitor is no longer possible because we generate a new mon
keyring each time the playbook is run.

Fixes: #5864

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-25 09:34:44 +01:00
Guillaume Abrioux 5ff2ca270f mon: replace `command` task by `copy`
We can achieve this task using `copy` module.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-25 09:34:44 +01:00
Dimitri Savineau 40a87c4b92 ceph-iscsi: set the pool name in the config file
When using a custom pool for iSCSI gateway then we need to set the pool
name in the configuration otherwise the default rbd pool name will be
used.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-24 20:41:54 +01:00
Guillaume Abrioux 6ba4c8c70b tests: update pytest workflow
This sets the `PYTHONPATH` environment variable so we can run pytest
with the recent module refactor (module_utils introduction)

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-24 10:38:28 +01:00
Guillaume Abrioux 7c72632418 library: import ca_test_common in test_ceph_key
since we added `ca_test_common.py` let's use it in `test_ceph_key.py`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-24 10:38:28 +01:00
Guillaume Abrioux 81233dd963 rgw: call `ceph_ec_profile` when needed
Let's replace `command` tasks with `ceph_ec_profile` calls

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-24 10:38:28 +01:00
Guillaume Abrioux 497d27dcf1 library: add ec_profile module
This commit adds a new module `ceph_ec_profile` to manage erasure code
profiles.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-24 10:38:28 +01:00
Guillaume Abrioux d7fd46842d library: add module_utils namespace
This adds a new `module_utils` namespace in order to avoid defining same
functions in each module.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-24 10:38:28 +01:00
Guillaume Abrioux 94c37b9de8 tests: use github workflow for nbsp char check
Let's use a github workflow instead of travis for this.

With this commit we can get rid of Travis.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux 195d88fcda lint: ignore 302,303,505 errors
ignore 302,303 and 505 errors

[302] Using command rather than an argument to e.g. file
[303] Using command rather than module
[505] referenced files must exist

they aren't relevant on these tasks.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux c948b668eb lint: do not use 'local_action'
Fix ansible-lint 504 error:

[504] Do not use 'local_action', use 'delegate_to: localhost'

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux dfc7e6e4bd lint: trailing whitespace
Fix ansible-lint 201 error:

[201] Trailing whitespace

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux 97dd9218dd lint: all tasks should be named
Fix ansible-lint 502 error:

[502] All tasks should be named

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux 11b4bf5083 lint: use shell only when shell functionality is required
Fix ansible-lint 305 error:

[305] Use shell only when shell functionality is required

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux 2011e4dbc8 lint: don't compare to literal true/false
Fix ansible lint 601 error:

[601] Don't compare to literal True/False

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux 9fba6eecfa lint: variables should have spaces before and after
Fix ansible lint 206 error:

[206] Variables should have spaces before and after: {{ var_name }}

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux 5450de58b3 lint: commands should not change things
Fix ansible lint 301 error:

[301] Commands should not change things if nothing needs doing

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux 1879c26eb9 lint: set pipefail on shell tasks
Fix ansible lint 306 error:

[306] Shells that use pipes should set the pipefail option

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux d4400f911a tests: use github workflow for ansible-lint
let's use github workflow instead of travis.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-23 08:33:47 +01:00
Guillaume Abrioux 873fc8ec0f osd: ensure /var/lib/ceph/osd/{cluster}-{id} is present
This commit ensures that the `/var/lib/ceph/osd/{{ cluster }}-{{ osd_id }}` is
present before starting OSDs.

This is needed specificly when redeploying an OSD in case of OS upgrade
failure.
Since ceph data are still present on its devices then the node can be
redeployed, however those directories aren't present since they are
initially created by ceph-volume. We could recreate them manually but
for better user experience we can ask ceph-ansible to recreate them.

NOTE:
this only works for OSDs that were deployed with ceph-volume.
ceph-disk deployed OSDs would have to get those directories recreated
manually.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1898486

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-19 09:20:28 +01:00
Dimitri Savineau e150df789e ceph-facts: fix read osd pool default crush fact
We don't need to use run_once on that task when having running monitors
otherwise the read task could be skip and the set task will fail.

The conditional check 'crush_rule_variable.rc == 0' failed. The error
was: error while evaluating conditional (crush_rule_variable.rc == 0):
'dict object' has no attribute 'rc'

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1898856

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-18 12:55:43 -05:00
Dimitri Savineau 3e79f0322a tests: use github workflow for pytest
Move the pytest testing from TravisCI to Github workflow.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-17 17:52:03 +01:00
Guillaume Abrioux f5ba6d9b01 containers: modify bindmount option
This commit changes the bind mount option for the mount point
`/var/lib/ceph` in the systemd template for mon and mgr containers. This
is needed in case of collocating mon/mgr with osds using dmcrypt
scenario.
Once mon/mgr got converted to containers, the dmcrypt layer sub mount is
still seen in `/var/lib/ceph`. For some reason it makes the
corresponding devices busy so any other container can't open/close it.
As a result, it prevents osds from starting properly.

Since it only happens on the nodes converted before the OSD play, the idea is
to bind mount `/var/lib/ceph` on mon and mgr with the `rshared` option
so once the sub mount is unmounted, it is propagated inside the
container so it doesn't see that mount point.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1896392

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-17 09:19:23 -05:00
Dimitri Savineau 35ed9977aa switch2container: chown symlink in mon/mgr plays
fa2bb3a only fix the symlink owner/group issue in the OSD play. If the
OSDs are collocated with other services like MONs and MGRs then the
chown command will fail.

$ find /var/lib/ceph/osd/ceph-0 -not -user 167 -execdir chown 167:167 {} +
chown: cannot dereference './block': Permission denied

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1896448

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-16 13:40:57 -05:00
Guillaume Abrioux 5ba7824c55 container: force rm --storage on ExecStartPre
This is a workaround to avoid error like following:
```
Error: error creating container storage: the container name "ceph-mgr-magna022" is already in use by "4a5f674e113f837a0cc561dea5d2cd55d16ca159a647b7794ab06c4c276ef701"
```

that doesn't seem to be 100% reproducible but it shows up after a
reboot. The only workaround we came up with at the moment is to run
`podman rm --storage <container>` before starting it.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1887716

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-16 10:38:40 -05:00