Commit Graph

5814 Commits (4f6da9d92ff264e05dbcd7c9d36398acd1692935)
 

Author SHA1 Message Date
Dimitri Savineau 13427eddac cephadm-adopt: add grafana group conversion
The grafana group conversion task wasn't present in the cephadm-adopt.yml
playbook.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1917530

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-18 20:52:58 +01:00
Guillaume Abrioux 4af0845702 mon: fix cephx disabled deployment
Due to missing condition on `cephx` variable, cephx disabled deployments
are broken.
This commit fixes this.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1910151

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-18 11:30:02 -05:00
Dimitri Savineau 6616908577 module_utils: don't add newline to the data
When executing a command via the run_command method and passing some
data with stdin then the default behavior is to add append a newline.
This breaks the value of password used by our modules.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-18 11:29:30 -05:00
Dimitri Savineau 5a14510354 tests/library: remove duplicate parameter
Remove duplicate fake_params parameter as it's already defined later
as a dict (instead of an empty list).

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-14 10:11:17 +01:00
Guillaume Abrioux e66f12d138 fs2bs: skip migration when a mix of fs and bs is detected
Since the default of `osd_objectstore` has changed as of 3.2, some
deployments might have a mix of filestore and bluestore OSDs on a same
node. In some specific cases, there's a possibility that a filestore OSD
shares a journal/db device with a bluestore OSD. We shouldn't try to
redeploy in this context because ceph-volume will complain. (either
because in lvm batch you can't pass partition or about gpt header).
The safest option is to skip the migration on the node when such a mix
is detected or force all osds including those already using bluestore
(option `force_filestore_to_bluestore=True` has to be passed as an extra var).
If all OSDs are using filestore, then they will be migrated to
bluestore.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1875777

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-12 14:40:25 -05:00
Guillaume Abrioux ae196bf946 validate: check virtual_ips variable
This commit checks the length of `virtual_ips` doesn't exceed the length
of `groups[rgwloadbalancer_group_name]`.
It also ensure this variable is defined when
`groups[rgwloadbalancer_group_name]` contains at least one node.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-12 11:03:12 +01:00
Benoît Knecht 3116f46422 ceph-rgw-loadbalancer: Fix keepalived master selection
While 2ca33641 fixed a bug in the way the `keepalived.conf.j2` template matched
hostnames to set the VRRP `MASTER`/`BACKUP` states, it also introduced a
regression in the case where `virtual_ips` is a list of more than one IP
address.

The previous behavior would result in each host in the `rgwloadbalancers` group
to be `MASTER` for one of the `virtual_ips`, but the new behavior caused the
first host to be `MASTER` for all the IP address in `virtual_ips`.

This commit restores the original behavior.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
2021-01-12 11:03:12 +01:00
Guillaume Abrioux 175ffa1b88 switch2container: fix mon quorum check
The current check makes no sense because it checks any of other monitor
than the one being played (either a previous one already converted or a
next that isn't yet converted) is present on the quorum.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1909011

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-11 14:42:45 -05:00
Guillaume Abrioux 41314f49bf Revert "tests: temporarily use david's flavor"
This reverts commit ed9f0641ee.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-11 15:50:55 +01:00
Dimitri Savineau 3f64ced36b ceph-osd: replace sysctl command task by slurp
Instead of using the command module for retrieving a sysctl value then
we can use the slurp module and read the value directly from /proc.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-11 13:24:23 +01:00
Guillaume Abrioux ed9f0641ee tests: temporarily use david's flavor
master nfs ganesha builds are broken, let's use this flavor instead for
now.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-07 17:09:46 -05:00
Guillaume Abrioux ef975ef5ea dashboard: configure passwords via stdin
Due to recent changes in ceph, the few dashboard passwors
must be passed via `-i`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-07 17:09:46 -05:00
Guillaume Abrioux 2725db3e9f library: refact ceph_dashboard_user
refact this module due to recent changes in ceph pacific.
The password must be passed with `-i` option.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-07 17:09:46 -05:00
Dimitri Savineau 9cc607e9af spec: add module_utils directory
Since d7fd468 the ansible modules are using the common code shared in
the module_utils directory but that one wasn't added to the spec file.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1910214

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-06 20:26:48 +01:00
Mike Currin 4cbc9a48c9 Path for ceph config missing in crash template
The path where ceph.conf is located (/etc/ceph) missing in the Docker container bind mounts, this throws errors

Signed-off-by: Mike Currin <currin@gmail.com>
2021-01-06 16:50:18 +01:00
Guillaume Abrioux 513c8cfe55 rgw: support switching from single-site to multisite
When collocating rgw with either a mon, mgr or osd, switching from
single site to a multisite rgw setup failed because of the handlers
triggered between the ansible play of the collocated daemon and the play
of the rgw. Since the multisite changes are not yet applied the handlers
fail.
The idea here is to ensure we run the multisite configuration from the
ceph-handler role before the restart happens, this way it won't complain
because of non existing multisite configuration.

(Note: this is also valid when simply changing a multisite configuration)

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1888630

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-06 09:58:45 -05:00
Dimitri Savineau 613ab11b9b library: remove containerized parameter from cv
The ceph-volume module relies on environment variables to determine if
the command should be executed within a container or not.
The containerized parameter isn't used anymore and we can remove it.

Fixes: #6153

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-06 10:12:15 +01:00
Dimitri Savineau 31811b9e6a library: add no_log to {access,secret}_key params
This sets the no_log parameter on both the access and the secret
RGW key variables.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-06 10:11:39 +01:00
Dimitri Savineau 5b6f907a72 cephadm: remove loop on host add tasks
Instead of iterate over the host list for adding the node/label to the
host orchestrator configuration then we can do it parallelly.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-16 15:14:28 +01:00
Fabien Brachere 4026ba9da1 library: add missing `target_size_ratio` parameter support in ceph_pool module
When creating a new pool, target_size_ratio was ignored by ansible module ceph_pool.py.
target_size_ratio is now used when pg_autoscale_mode is on.
Tests added to library tests.
This adds too the use in the role ceph-rgw.

Signed-off-by: Fabien Brachere <fabien.brachere@celeste.fr>
2020-12-16 15:10:27 +01:00
Dimitri Savineau 827b23353f ceph-config: fix ceph-volume lvm batch report
Since the major ceph-volume lvm batch refactoring, the report value
is different.
Before the refact, the report was a dict with the OSDs list to be created
under the "osds" key.
After the refact, the report is a list of dict.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-15 21:19:04 +01:00
Guillaume Abrioux 5e879dd964 Revert "mergify: add configuration for 4.2z1 branch"
This reverts commit fb7dced598.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-12-15 17:28:56 +01:00
Guillaume Abrioux fb7dced598 mergify: add configuration for 4.2z1 branch
So we get backports against 4.2z1 branch (downstream related) automatically
created by mergify

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-12-15 10:01:46 +01:00
Guillaume Abrioux 011c97786b tests: force box removal
This avoids interactive mode for `vagrant box remove`.
This can happen for some reason when there's leftover from previous
deployment (VMs not destroyed as expected)

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-12-14 17:53:21 +01:00
Guillaume Abrioux e2ea403d5e tests: rgw_multisite playbook test refactor
Currently we create an object from the primary sites but we try to read
that object still from the master which doesn't make sense, we should
try to read it from a secondary site.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-12-14 17:53:21 +01:00
Karl-Heinz Preuß 6ce34ef59f fix broken ceph-fetch-keys role
set fetch_directory variable in default/main.yml instead of using the
defaults jinja filter in tasks/main.yml.

Fixes: #6072

Signed-off-by: Karl-Heinz Preuß <karl-heinz.preuss@cms.hu-berlin.de>
2020-12-14 17:36:17 +01:00
Seena Fallah 5e9444fa5c ceph-osd: use global crush_device_class in lvm_volumes
Use global crush_device_class variable if it's not set per OSD

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2020-12-12 06:56:53 +01:00
Dimitri Savineau aa6e1f20ea Revert "config: Always use osd_memory_target if set"
This reverts commit 4d1fdd2b05.

This breaks the backward compatibility with previous osd_memory_target
calculation and we could have a value lower than the minimum value allowed
(896M) which causes some ceph commands to fail (like ceph assimilate-conf).

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-12 06:56:32 +01:00
Dimitri Savineau 5a41026347 monitoring: use config_template module for config
The alertmanager, grafana and prometheus configuration file are
generated with the template module which doesn't allow for using
config overrides.
Instead we could use the config_template plugin action and add a
new variable for overrides (one for each component).

With this patch, one should be able to add configuration to
prometheus with the following:

---
alertmanager_conf_overrides:
  global:
    smtp_smarthost: 'localhost:25'
...

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1902999

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-12 06:55:27 +01:00
Dimitri Savineau d82249a8c0 ceph-rgw: add cluster parameter on ceph_ec_profile
81233dd introduced a regression with the ceph_ec_profile module call in
the ceph-rgw role due the missing cluster module parameter.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-12 06:54:46 +01:00
Dimitri Savineau 2aeab882f3 ceph-facts: fix grafana group conversion
The conversion fact task was only executed when the grafana_server_group_name
variable was explicitly set in the user configuration. If an user was using
the default value then the conversion wasn't executed.

This also adds back the default grafana_server_group_name value in case user
was using the default value and to avoid undefined variable error.

Instead of hardcoding the "monitoring" group name then we can reuse the
monitoring_group_name variable.

There's no need to override the monitoring_group_name variable, it's either
using the default value or the one defined by the user.

Finally removing the delegate_to statement on the add_host task since it's
always executed on the ansible controller.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1903732

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-10 16:51:16 +01:00
Dimitri Savineau 3b9cdc8502 tests: remove pyyaml workaround on OSD nodes
Since [1] has been resolved then we don't need to apply this workaround
anymore.

[1] https://tracker.ceph.com/issues/46759

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-10 09:15:03 +01:00
Dimitri Savineau 0108c9f941 purge-container-cluster: always prune force
Since podman 2.x, there's now a confirmation when running podman
container prune command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-09 14:46:45 -05:00
Dimitri Savineau 801e7a29cf tests/vagrant: update box version to CentOS 8.3
This updates the CentOS libvirt box version to 8.3

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-09 14:46:45 -05:00
Dimitri Savineau a2cbab16a4 rhcs: drop fetch_directory override
Since the fetch_directory variable has been dropped then we don't need
the override in rhcs file.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-03 10:48:21 +01:00
Jukka Nousiainen eb7473491b ceph-mon: No become during gen mon initial keyring
Since the backing generate_secret() just hands out urandom output,
running as privileged doesn't seem to be required. It's not
desireable to provide sudo in some Ansible runner environments.

Signed-off-by: Jukka Nousiainen <jukka.nousiainen@csc.fi>
2020-12-03 10:04:21 +01:00
Dimitri Savineau 08f118077f library: add cephadm_adopt module
This adds cephadm_adopt ansible module for replacing the command module
usage with the cephadm adopt command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-02 09:15:44 +01:00
Guillaume Abrioux 86a8889ee3 common: do not use pipefail when not needed
Let's discard the ansible lint error 306 and add a "# noqa 306" on tasks
where we don't need `set -o pipefail`

Fixes: #6090

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-12-01 15:07:09 -05:00
Dimitri Savineau cf7345f143 consume ceph_volume module when possible
We should always use the ceph_volume ansible module when possible.
This patch replace the ceph-volume inventory and lvm {list,zap} commands
called via the command/shell modules by the corresponding call with the
ceph_volume module.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 17:54:10 +01:00
Dimitri Savineau 2e417ab901 library: add ceph_crush_rule module
This adds ceph_crush_rule ansible module for replacing the command
module usage with the ceph osd crush rule commands.
This module can manage both erasure and replicated crush rules.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 17:52:41 +01:00
Guillaume Abrioux 5c4ae5356d osd: add tag on 'wait for all osd to be up' task
This allows skipping this task if really desired.
Use it carefully. Use it at your own risk.

Fixes: #6073

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-12-01 11:00:25 +01:00
Dimitri Savineau 1831b4955f ceph-client: use group_by instead of add_host
Instead of iterate over all client nodes with a loop sequentially, we
can use the group_by ansible buildin.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 10:58:48 +01:00
Dimitri Savineau c3ed124d31 library: add cephadm_bootstrap module
This adds cephadm_bootstrap ansible module for replacing the command module
usage with the cephadm bootstrap command.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 10:30:05 +01:00
Dimitri Savineau 5da593604a library: add ceph_osd_flag module
This adds ceph_osd_flag ansible module for replacing the command module
usage with the ceph osd set/unset commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 10:29:11 +01:00
Guillaume Abrioux d40dd764e0 iscsigw: remove `--cap-add=all` from `podman run` cmd
As of podman `2.0.5`, `--cap-add` and `--privileged` are exclusive
options.

```
Nov 30 13:56:30 magna089 podman[171677]: Error: invalid config provided: CapAdd and privileged are mutually exclusive options
```

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1902149

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-30 12:24:11 -05:00
Guillaume Abrioux c68b124ba8 container: remove `--ignore` from `podman rm` command
As of podman 2.0.5, `--ignore` param conflicts with `--storage`.
```
Nov 30 13:53:10 magna089 podman[164443]: Error: --storage conflicts with --volumes, --all, --latest, --ignore and --cidfile
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-11-30 12:24:11 -05:00
Dimitri Savineau 0b5b1de963 library: add ceph_osd module
This adds ceph_osd ansible module for replacing the command module
usage with the ceph osd destroy/down/in/out/purge/rm commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 16:53:45 +01:00
Dimitri Savineau eaf0ebfc85 library: add ceph_mgr_module module
This adds ceph_mgr_module ansible module for replacing the command module
usage with the ceph mgr module enable/disable commands.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 16:52:02 +01:00
Dimitri Savineau c610a1b411 tests: use ca_test_common with ceph_volume_simple
To avoid duplicate code.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 10:05:45 +01:00
Dimitri Savineau d76fbb38d5 improve plugins/filter testing
- The plugins/filter directory wasn't present in the flake8 workflow
configuration.
- Fix the flake8 syntax.
- Add the directory to PYTHONPATH environment variable for pytest
to avoid importing the plugin filter via sys.
- Add unittest on missing netaddr module import.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-11-30 10:05:01 +01:00