Commit Graph

5786 Commits (07813aba14e2eabdecd1fe04c02a869bee6515e7)
 

Author SHA1 Message Date
Guillaume Abrioux 036b03a7bb purge: support osd_auto_discovery
This adds a task that zaps by osd id so we can support the scenario
where osds were deployed with `osd_auto_discovery` is true.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1876860

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 4144074a50)
2021-07-22 12:43:57 -04:00
Guillaume Abrioux b36e8ec935 purge: merge playbooks
This refactor merges the two playbooks so we only have to maintain 1
playbook.
(Symlink the old purge-container-cluster.yml playbook for backward
 compatibility).

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 17cd83bf3a)
2021-07-22 12:43:57 -04:00
Guillaume Abrioux fc5440b71c purge: drop variables from 'hosts' sections
Those variables are useless given this is not possible to override them.
Let's replace them with the hardcoded name instead.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 6b50401d0c)
2021-07-22 12:43:57 -04:00
Dimitri Savineau e54c8e93ee cephadm-adopt: set application on ganesha pool
Set the nfs application to the ganesha pool.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1956840

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit aeb9f562e5)
2021-07-21 16:22:25 +02:00
Dimitri Savineau 3ec8e90b34 cephadm-adopt: enable osd memory autotune for HCI
This enables the osd_memory_target_autotune option on HCI environment.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1973149

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit a305296384)
2021-07-21 16:22:10 +02:00
Dimitri Savineau f6cd8b9816 common: remove unnecessary run_once statements
1303611 introduced tasks for disabling the pg_autoscaler on pools and
the balancer but thoses tasks are already executed on the first monitor
node so we don't need to add the run_once statement.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 738fa9428a)
2021-07-21 10:01:15 -04:00
Dimitri Savineau cf734e19b7 common: fix py2 pool_list from_json when skipped
When using python 2 and the task with a loop is skipped then it generates
an error.

Unexpected templating type error occurred on
({{ (pool_list.stdout | from_json)['pools'] }}): expected string or buffer

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit cf6e33346e)
2021-07-21 14:00:30 +02:00
Guillaume Abrioux 3cc8c667d0 common: disable/enable pg_autoscaler
The PG autoscaler can disrupt the PG checks so the idea here is to
disable it and re-enable it back after the restart is done.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 13036115e2)
2021-07-20 11:04:25 -04:00
Dimitri Savineau 15103d3b34 ceph-mgr: move mgr module list to common
Populating the ceph_mgr_modules list in the mgr_modules doesn't make sense
since that file is only executed if the list isn't empty or we're using the
dashboard.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit cd06e7c046)
2021-07-19 14:32:55 -04:00
Dimitri Savineau f2a95728f7 ceph-nfs: allow overriding NFS_CORE_PARAM
We already have config override variables for existing block (like
ganesha_ceph_export_overrides, ganesha_log_overrides, etc...) or a
global one (ganesha_conf_overrides) but redefining the NFS_CORE_PARAM
block in that variable will erase all previous values (currently only
Bind_Addr).

ganesha_core_param_overrides: |
        Enable_UDP = false;
        NFS_Port = 2050;

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1941775

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 9817d29543)
2021-07-19 14:12:55 -04:00
Guillaume Abrioux 93f37e4b5a lib/ceph-volume: support zapping by osd_id
This commit adds the support for zapping an osd by osd_id in the
ceph_volume module.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 70f1d6e2cd)
2021-07-13 17:02:36 -04:00
Guillaume Abrioux fb825e0659 purge: reindent playbook
This commit reindents the playbook.
Also improve readability by adding an extra line between plays.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 60aa70a128)
2021-07-13 14:47:44 -04:00
Dimitri Savineau e08cb421d4 rolling_update: check quorum state before upgrade
If one a the monitor is out of the quorum then nothing prevents the upgrade
playbook to run.
We only check if we have at least three monitor nodes but we should also
check if those monitor nodes are correctly present in the quorum.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1952571

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 97148dd58c)
2021-07-12 12:58:02 -04:00
Guillaume Abrioux bf5d0b7374 update: fail the playbook if straw2 conversion failed
It's better to fail the playbook so the user is aware the straw2
migration has failed.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit c396122ad9)
2021-07-09 16:32:47 -04:00
Neelaksh Singh e0b6bb96ec Sensitive key data now hidden in output log
Fixes: #6529

Signed-off-by: Neelaksh Singh <neelaksh48@gmail.com>
(cherry picked from commit d18a9860cd)
2021-07-09 16:03:02 +02:00
Guillaume Abrioux 0a348bd396 update: followup on pr #6689
add mising 'osd' command.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 4eb4268dee)
2021-07-09 11:34:12 +02:00
Guillaume Abrioux ea8f0c7bcb update: convert straw bucket
After an upgrade, the presence of straw buckets will produce the
following warning (HEALTH_WARN):

```
crush map has legacy tunables (require firefly, min is hammer)
```

because straw bucket is a firefly feature it needs to be converted to
straw2.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1967964

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit eee576477c)
2021-07-09 09:15:24 +02:00
Guillaume Abrioux 27755f9bff dashboard: remove "certificate is valid for" error
When deploying dashboard with ssl certificates generated by
ceph-ansible, we enforce the CN to 'ceph-dashboard' which can makes
application such alertmanager complain like following:

`err="Post https://mgr0:8443/api/prometheus_receiver: x509: certificate is valid for ceph-dashboard, not mgr0" context_err="context deadline exceeded"`

The idea here is to add alternative names matching all mgr/mon instances
in the certificate so this error won't appear in logs.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1978869

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 72a0336c71)
2021-07-07 17:19:00 +02:00
Wong Hoi Sing Edison 35264b7381 library: flake8 ceph-ansible modules
This commit ensure all ceph-ansible modules pass flake8 properly.

Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
(cherry picked from commit beda1fe773)
2021-07-07 13:16:04 +02:00
Dimitri Savineau ec648981e6 infra: add playbook to purge dashboard/monitoring
The dashboard/monitoring stack can be deployed via the dashboard_enabled
variable. But there's nothing similar if we can to remove that part only
and keep the ceph cluster up and running.
The current purge playbooks remove everything.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1786691

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 8e4ef7d6da)
2021-07-06 11:40:31 -04:00
Guillaume Abrioux a7d2a53b37 dashboard: support dedicated network for the dashboard
This introduces a new variable `dashboard_network` in order to support
deploying the dashboard on a different subnet.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1927574

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit f4f73b6197)
2021-07-06 14:54:00 +02:00
Dimitri Savineau 083b55a760 ceph-crash: add install checkpoint
The ceph crash insatll checkpoint callback was missing in the main
playbooks.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 993d06c4d9)
2021-07-05 18:11:32 +02:00
Guillaume Abrioux f80837c23e cephadm_adopt: add any_errors_fatal on play
Add any_errors_fatal: true in cephadm-adopt playbook.
We should stop the playbook execution when a task throws an error.
Otherwise it can lead to unexpected behavior.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1976179

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3b804a61dd)
2021-07-03 11:58:57 +02:00
Dimitri Savineau b3cf8212fa ceph-facts: move device facts to its own file
Instead of reusing the condition 'inventory_hostname in groups[osds]'
on each device facts tasks then we can move all the tasks into a
dedicated file and set the condition on the import_tasks statement.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit d704b05e52)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 255b3763ef ceph-validate: check logical volumes
We currently don't check if the logical volume used in lvm_volumes list
for either bluestore data/db/wal or filestore data/journal exist.
We're only doing this on raw devices for batch scenario.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 55bca07cb6)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 80d5bff7e5 ceph-validate: check db/journal/wal devices too
When using dedicated devices for db/journal/wal objecstore with
ceph-volume lvm batch then we should also validate that those devices
exist and don't use a gpt partition table in addition of the devices
and lvm_volume.data variables.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 808e7106de)
2021-07-02 22:21:20 +02:00
Dimitri Savineau f8ecb08ec2 ceph-validate: use root device from ansible_mounts
Instead of using findmnt command to find the device associated to the
root mount point then we can use the ansible_mounts fact.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 7e50380f7f)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 23ddab7f53 ceph-validate: do not resolve devices
This is already done in the ceph-facts role.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 0df99dda8d)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 727aa93292 ceph-validate: check block presence first
Instead of doing two parted calls we can check first if the device exist
and then test the partition table.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 14d458b3b4)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 65b8f46a43 ceph-validate: check devices from lvm_volumes
2888c08 introduced a regression as the check_devices tasks file was
only included based on the devices variable.
But that file also validate some devices from the lvm_volumes variable.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1906022

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit ac0342b72e)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 04b1665e5e prometheus: fix prometheus target url
The prometheus service isn't binding on localhost.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1933560

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 1d56818658)
2021-07-02 14:37:34 -04:00
Guillaume Abrioux 0d4b029057 purge: add monitoring group in final cleanup play
This adds the monitoring group in the "final cleanup play" so any cid
files generated are well removed when purging the cluster.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1974536

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 037d8cd05e)
2021-07-02 14:36:48 -04:00
Dimitri Savineau 3bd3dddcc2 container: set tcmalloc value by default
All ceph daemons need to have the TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES
environment variable set to 128MB by default in container setup.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1970913

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 9758e3c513)
2021-07-01 15:45:54 +02:00
Guillaume Abrioux 676aad9ea2 update: do not gather facts on each play
There's no benefit to gather facts again on each play in
rolling_update.yml

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 2c77d0094c)
2021-06-30 20:39:25 +02:00
Guillaume Abrioux adfb9d3b2a ceph_key: handle error in a better way
When calling the `ceph_key` module with `state: info`, if the ceph
command called fails, the actual error is hidden by the module which
makes it pretty difficult to troubleshoot.

The current code always states that if rc is not equal to 0 the keyring
doesn't exist.

`state: info` should always return the actual rc, stdout and stderr.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1964889

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit d58500ade0)
2021-06-30 20:34:17 +02:00
Dimitri Savineau 48f47e7023 rhcs: remove ISO install method
Starting RHCS 5, there's no ISO available anymore.
This removes all ISO variables and the ceph_repository_type variable.

Closes: #6626

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit a05730b38a)
2021-06-30 20:33:44 +02:00
Boris Ranto 5b18429c07 dashboard: Add new prometheus alert
It was requested for us to update our alerting definitions to include a
slow OSD Ops health check.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1951664

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 2491d4e004)
2021-06-30 15:30:31 +02:00
Dimitri Savineau a71a80c167 tox: remove ceph_dev variables and dev_setup calls
Since pacific is a stable release then we don't need to use shaman for getting
the pacific build.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-06-30 09:29:58 +02:00
Dimitri Savineau f64a4258ea switch2container: run ceph-validate role
This adds the ceph-validate role before starting the switch to a containerized
deployment.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1968177

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit fc160b3be1)
2021-06-30 09:29:58 +02:00
Guillaume Abrioux 5b8d0b11d2 workflows: test against 1 python version only
Let's drop py3.6 and py3.7

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit d191ba38d3)
2021-06-30 08:17:40 +02:00
Guillaume Abrioux 5fd24a3793 workflows: add signed-off check
This adds a github workflow for checking the signed off line in commit
messages.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 8c09497567)
2021-06-30 08:17:40 +02:00
Guillaume Abrioux 3e894ca899 workflow: add group_vars/defaults checks
let's use github workflow for checking defaults values.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit d71db816c6)
2021-06-30 08:17:40 +02:00
Guillaume Abrioux 51612aa7d3 workflow: add syntax check
This adds the ansible --syntax-check test in the ansible-lint workflow

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5ed423ad88)
2021-06-30 08:17:40 +02:00
Guillaume Abrioux 5787048599 tests: remove legacy file
This inventory isn't used anywhere.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 304d1cbb97)
2021-06-29 17:52:22 +02:00
Guillaume Abrioux 16dc991351 shrink-mgr: modify existing mgr check
Do not rely on the inventory aliases in order to check if the selected
manager to be removed is present.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1967897

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 26a7256c4c)
2021-06-29 17:52:22 +02:00
Guillaume Abrioux 0856d3e47f cephadm-adopt/rgw: add host target in svc_id
If multi-realms were deployed with several instances belonging to the same
realm and zone using the same port on different nodes, the service id
expected by cephadm will be the same and therefore only one service will
be deployed. We need to create a service called
`<node>.<realm>.<zone>.<port>` to be sure the service name will be unique
and well deployed on the expected node in order to preserve backward
compatibility with the rgws instances that were deployed with
ceph-ansible.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1967455

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 31311b03ed)
2021-06-29 15:18:49 +02:00
Guillaume Abrioux aa332ac64d cephadm-adopt: support rgw multisite adoption
We need to support rgw multisite deployments.
This commit makes the adoption playbook support this kind of deployment.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1967455

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit fc784fc44c)
2021-06-24 09:48:27 +02:00
Guillaume Abrioux 1d0651e465 nfs: do no copy client.bootstrap-rgw when using mds
There's no need to copy this keyring when using nfs with mds

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 8dbee99882)
2021-06-17 08:15:36 +02:00
Guillaume Abrioux 0a26f118f1 multisite: fix bug during switch2containers
When running the switch-to-containers playbook with multisite enabled,
the fact "rgw_instances" is only set for the node being processed
(serial: 1), the consequence of that is that the set_fact of
'rgw_instances_all' can't iterate over all rgw node in order to look up
each 'rgw_instances_host'.

Adding a condition checking whether hostvars[item]["rgw_instances_host"]
is defined fixes this issue.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1967926

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 8279d14d32)
2021-06-17 08:15:09 +02:00
VasishtaShastry e49c38f8b7 Container: Fixing service name lvm2-lvmetad
Playbook failing saying:

msg: 'Could not find the requested service lvmetad: host'

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1955040
Signed-off-by: VasishtaShastry <vipin.indiasmg@gmail.com>
2021-06-17 01:50:27 +02:00