Commit Graph

2898 Commits (99344e61f6bef378eb88863107da510203c1dcaf)

Author SHA1 Message Date
Dimitri Savineau 490a1c6ba6 ceph-mon: do not log monitor keyring
We don't want to display the keyring in the ansible log.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit e44075abd6)
2021-08-12 13:30:50 +02:00
Guillaume Abrioux 634baa9b63 common: do not log keyring secret
let's not display any keyring secret by default in ansible log.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1980744

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 7511195738)
2021-08-11 14:59:25 -04:00
Dimitri Savineau 4b090b661e ceph-dashboard: fix TLS cert openssl generation
With OpenSSL version prior 1.1.1 (like CentOS 7 with 1.0.2k), the -addext
doesn't exist.
As a solution, this uses the default openssl.cnf configuration file as a
template and add the subjectAltName in the v3_ca section. This temp openssl
configuration file is removed after the TLS certificate creation.
This patch also move the run_once statement at the block level.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1978869

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 5e0ace7e54)
2021-08-09 15:14:30 -04:00
Guillaume Abrioux f02295be85 dashboard: subj_alt_names fact refactor
the current way the variable is built results in:

```
2021-08-03 04:18:23,020 - ceph.ceph - INFO - ok: [ceph-sangadi-4x-indpt6-node1-installer] => changed=false
  ansible_facts:
    subj_alt_names: |-
      subjectAltName=ceph-sangadi-4x-indpt6-node1-installer/subjectAltName=10.0.210.223/subjectAltName=ceph-sangadi-4x-indpt6-node1-installersubjectAltName=ceph-sangadi-4x-indpt6-node2/subjectAltName=10.0.210.252/subjectAltName=ceph-sangadi-4x-indpt6-node2/
```

which is incorrect.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1978869

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 6f1a0634f7)
2021-08-09 15:14:30 -04:00
Teoman ONAY f8facde33a podman pids.max default value is 2048, docker's one is 4096 which are
sufficient for the default value (512) of rgw thread pool size.
But if its value is increased near to the pids-limit value,
it does not leave place for the other processes to spawn and run within
the container and the container crashes.

pids-limit set to unlimited regardless of the container engine.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1987041

Signed-off-by: Teoman ONAY <tonay@redhat.com>
(cherry picked from commit 9b5d97adb9)
2021-08-05 11:04:18 -04:00
Dimitri Savineau 31cc8bd2aa osds: use osd pool ls instead of osd dump command
The ceph osd pool ls detail command is a subset of the ceph osd dump
command.

$ ceph osd dump --format json|wc -c
10117
$ ceph osd pool ls detail --format json|wc -c
4740

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 06471a4b82)
2021-08-03 13:57:14 -04:00
Benoît Knecht 39fa5e2f2c ceph-handler: Fix osd handler in check mode
Run the Ceph commands that only gather information (without making any changes
to the cluster) when running Ansible in check mode.

This allows the tasks that depend on the variables set by those tasks to
succeed in check mode.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 498acd7527)
2021-08-02 15:53:49 +02:00
Dimitri Savineau 877b99b17e ceph-defaults: update grafana dashboards source
We currently download the grafana dashboars from the ceph@master branch
for all ceph releases.
We should use the right ceph branch according to the ceph release.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-07-27 11:44:43 -04:00
Dimitri Savineau 6f5f1a1955 ceph-defaults: add missing grafana dashboards
The radosgw-sync-overview and rbd-details grafana dashboars were missing
from the list.

Closes: #6758

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit f0ccf3ebf0)
2021-07-27 10:53:40 -04:00
Dimitri Savineau d0a122e296 alertmanager: allow disable dashboard tls verify
When using self-signed/untrusted CA certificates, alertmanager displays
an error in logs. With this commit this should make those messages
disappear.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1936299

Co-authored-by: Guillaume Abrioux <gabrioux@redhat.com>

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 9f77b929d1)
2021-07-25 22:01:43 -04:00
Dimitri Savineau ebc961f7ff multisite: use node fqdn for endpoints when https
When the rgw_multisite_proto variable is set to https then we shoudn't use
the IP address in the zone endpoints list but the node FQDN to match the
TLS certificate CN.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1965504

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit ad05a08160)
2021-07-22 22:47:50 +02:00
Dimitri Savineau cf734e19b7 common: fix py2 pool_list from_json when skipped
When using python 2 and the task with a loop is skipped then it generates
an error.

Unexpected templating type error occurred on
({{ (pool_list.stdout | from_json)['pools'] }}): expected string or buffer

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit cf6e33346e)
2021-07-21 14:00:30 +02:00
Guillaume Abrioux 3cc8c667d0 common: disable/enable pg_autoscaler
The PG autoscaler can disrupt the PG checks so the idea here is to
disable it and re-enable it back after the restart is done.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 13036115e2)
2021-07-20 11:04:25 -04:00
Dimitri Savineau 15103d3b34 ceph-mgr: move mgr module list to common
Populating the ceph_mgr_modules list in the mgr_modules doesn't make sense
since that file is only executed if the list isn't empty or we're using the
dashboard.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit cd06e7c046)
2021-07-19 14:32:55 -04:00
Dimitri Savineau f2a95728f7 ceph-nfs: allow overriding NFS_CORE_PARAM
We already have config override variables for existing block (like
ganesha_ceph_export_overrides, ganesha_log_overrides, etc...) or a
global one (ganesha_conf_overrides) but redefining the NFS_CORE_PARAM
block in that variable will erase all previous values (currently only
Bind_Addr).

ganesha_core_param_overrides: |
        Enable_UDP = false;
        NFS_Port = 2050;

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1941775

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 9817d29543)
2021-07-19 14:12:55 -04:00
Neelaksh Singh e0b6bb96ec Sensitive key data now hidden in output log
Fixes: #6529

Signed-off-by: Neelaksh Singh <neelaksh48@gmail.com>
(cherry picked from commit d18a9860cd)
2021-07-09 16:03:02 +02:00
Guillaume Abrioux 27755f9bff dashboard: remove "certificate is valid for" error
When deploying dashboard with ssl certificates generated by
ceph-ansible, we enforce the CN to 'ceph-dashboard' which can makes
application such alertmanager complain like following:

`err="Post https://mgr0:8443/api/prometheus_receiver: x509: certificate is valid for ceph-dashboard, not mgr0" context_err="context deadline exceeded"`

The idea here is to add alternative names matching all mgr/mon instances
in the certificate so this error won't appear in logs.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1978869

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 72a0336c71)
2021-07-07 17:19:00 +02:00
Guillaume Abrioux a7d2a53b37 dashboard: support dedicated network for the dashboard
This introduces a new variable `dashboard_network` in order to support
deploying the dashboard on a different subnet.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1927574

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit f4f73b6197)
2021-07-06 14:54:00 +02:00
Dimitri Savineau b3cf8212fa ceph-facts: move device facts to its own file
Instead of reusing the condition 'inventory_hostname in groups[osds]'
on each device facts tasks then we can move all the tasks into a
dedicated file and set the condition on the import_tasks statement.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit d704b05e52)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 255b3763ef ceph-validate: check logical volumes
We currently don't check if the logical volume used in lvm_volumes list
for either bluestore data/db/wal or filestore data/journal exist.
We're only doing this on raw devices for batch scenario.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 55bca07cb6)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 80d5bff7e5 ceph-validate: check db/journal/wal devices too
When using dedicated devices for db/journal/wal objecstore with
ceph-volume lvm batch then we should also validate that those devices
exist and don't use a gpt partition table in addition of the devices
and lvm_volume.data variables.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 808e7106de)
2021-07-02 22:21:20 +02:00
Dimitri Savineau f8ecb08ec2 ceph-validate: use root device from ansible_mounts
Instead of using findmnt command to find the device associated to the
root mount point then we can use the ansible_mounts fact.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 7e50380f7f)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 23ddab7f53 ceph-validate: do not resolve devices
This is already done in the ceph-facts role.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 0df99dda8d)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 727aa93292 ceph-validate: check block presence first
Instead of doing two parted calls we can check first if the device exist
and then test the partition table.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 14d458b3b4)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 65b8f46a43 ceph-validate: check devices from lvm_volumes
2888c08 introduced a regression as the check_devices tasks file was
only included based on the devices variable.
But that file also validate some devices from the lvm_volumes variable.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1906022

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit ac0342b72e)
2021-07-02 22:21:20 +02:00
Dimitri Savineau 04b1665e5e prometheus: fix prometheus target url
The prometheus service isn't binding on localhost.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1933560

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 1d56818658)
2021-07-02 14:37:34 -04:00
Dimitri Savineau 3bd3dddcc2 container: set tcmalloc value by default
All ceph daemons need to have the TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES
environment variable set to 128MB by default in container setup.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1970913

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 9758e3c513)
2021-07-01 15:45:54 +02:00
Guillaume Abrioux adfb9d3b2a ceph_key: handle error in a better way
When calling the `ceph_key` module with `state: info`, if the ceph
command called fails, the actual error is hidden by the module which
makes it pretty difficult to troubleshoot.

The current code always states that if rc is not equal to 0 the keyring
doesn't exist.

`state: info` should always return the actual rc, stdout and stderr.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1964889

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit d58500ade0)
2021-06-30 20:34:17 +02:00
Dimitri Savineau 48f47e7023 rhcs: remove ISO install method
Starting RHCS 5, there's no ISO available anymore.
This removes all ISO variables and the ceph_repository_type variable.

Closes: #6626

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit a05730b38a)
2021-06-30 20:33:44 +02:00
Boris Ranto 5b18429c07 dashboard: Add new prometheus alert
It was requested for us to update our alerting definitions to include a
slow OSD Ops health check.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1951664

Signed-off-by: Boris Ranto <branto@redhat.com>
(cherry picked from commit 2491d4e004)
2021-06-30 15:30:31 +02:00
Guillaume Abrioux 1d0651e465 nfs: do no copy client.bootstrap-rgw when using mds
There's no need to copy this keyring when using nfs with mds

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 8dbee99882)
2021-06-17 08:15:36 +02:00
Guillaume Abrioux 0a26f118f1 multisite: fix bug during switch2containers
When running the switch-to-containers playbook with multisite enabled,
the fact "rgw_instances" is only set for the node being processed
(serial: 1), the consequence of that is that the set_fact of
'rgw_instances_all' can't iterate over all rgw node in order to look up
each 'rgw_instances_host'.

Adding a condition checking whether hostvars[item]["rgw_instances_host"]
is defined fixes this issue.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1967926

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 8279d14d32)
2021-06-17 08:15:09 +02:00
VasishtaShastry e49c38f8b7 Container: Fixing service name lvm2-lvmetad
Playbook failing saying:

msg: 'Could not find the requested service lvmetad: host'

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1955040
Signed-off-by: VasishtaShastry <vipin.indiasmg@gmail.com>
2021-06-17 01:50:27 +02:00
Guillaume Abrioux 9b57f33e78 rolling_update: fix mon+rgw/multisite collocation
When monitors and rgw are collocated with multisite enabled, the
rolling_update playbook fails because during the workflow, we run some
radosgw-admin commands very early on the first mon even though this is
the monitor being upgraded, it means the container doesn't exist since
it was stopped.

This block is relevant only for scaling out rgw daemons or initial
deployment. In rolling_update workflow, it is not needed so let's skip
it.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1970232

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit f7166cccbf)
2021-06-14 13:58:50 +02:00
Guillaume Abrioux 8440ccabe1 dashboard: set cookie_secure in grafana
When using grafana behind https `cookie_secure` should be set to `true`.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1966880

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 4daed1f137)
2021-06-07 15:12:19 +02:00
Guillaume Abrioux e0bcd59c04 prometheus: enforce osd nodes in templates
When osd nodes are collocated in the clients group (HCI context for
instance), the current logic will exclude osd nodes since they are
present in the client group.

The best fix would be to exclude clients node only when they are not
member of another group but for now, as a workaround, we can enforce
the addition of osd nodes to fix this specific case.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1947695

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 664dae0564)
2021-05-25 18:59:03 +02:00
Guillaume Abrioux 01256ffe1b container: conditionnally disable lvmetad
Enabling lvmetad in containerized deployments on el7 based OS might
cause issues.
This commit make it possible to disable this service if needed.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1955040

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-05-25 16:50:30 +02:00
Guillaume Abrioux f453e4737d nfs: get org.ganesha.nfsd.conf from container
Since we need to revert 33bfb10, this is an alternative to initial approach.
We can avoid maintaining this file since it is present in container
image. The idea is to simply get it from the image container and write
it to the host.

Fixes: #6501

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit e6d8b058ba)
2021-05-07 16:34:33 +02:00
Dimitri Savineau dfcb6ed45f ceph-rgw: fix pg_autoscale_mode for pool
The pg_autoscale_mode for rgw pools introduced in 9f03a52 was wrong
and was missing a `value` keyword because `rgw_create_pools` is a
dict.

Fixes: #6516

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit a670982a38)
2021-05-07 13:36:09 +02:00
Benoît Knecht 0ce27a73b6 ceph-mon: Fix check mode for deploy monitor tasks
Skip the `get initial keyring when it already exists` task when both commands
whose `stdout` output it requires have been skipped (e.g. when running in check
mode).

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 2437f14581)
2021-04-30 15:04:12 +02:00
Seena Fallah ea3ec134d5 ceph-osd: allow to use ceph_tcmalloc_max_total_thread_cache for bluestore
TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES is for both bluestore and filestore

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
(cherry picked from commit 41295f0ef6)
2021-04-30 15:03:58 +02:00
Guillaume Abrioux f6d7e84044 ceph-nfs: fix dev repo task
We need to filter with the OS architecture in order to fetch the right
dev repository in shaman

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 8f87754b76)
2021-04-30 13:37:09 +02:00
Francesco Pantano 8ce0000921 Config the monitoring stack components api urls using a VIP
When dashboard_frontend_vip is provided, all the services should be
configured using the related VIP. A new VIP variable is added for
both prometheus and alertmanager: we're already able to properly
config the grafana vip using dashboard_frontend_vip variable.
This change adds the same variable for both prometheus and
alertmanager.

Signed-off-by: Francesco Pantano <fpantano@redhat.com>
(cherry picked from commit 441651638d)
2021-04-16 08:33:16 +02:00
Benoît Knecht 8eec148d15 ceph-rgw-loadbalancer: Fix rgw_ports fact
The `set_fact rgw_ports` task was failing due to a templating error, because
`hostvars[item].rgw_instances` is a list, but it was treated as if it was a
dictionary.

Another issue was the fact that the `unique` filter only applied to the list
being appended to `rgw_ports` instead of the entire list, which means it was
possible to have duplicate items.

Lastly, `rgw_ports` would have been a list of integers, but the `seport` module
expects a list of strings.

This commit fixes all of the issues above, allowing the `ceph-rgw-loadbalancer`
role to work on systems with SELinux enabled.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit c078513475)
2021-04-15 13:20:45 +02:00
Guillaume Abrioux 5f6050bed1 container/systemd: ensure /var/log/ceph exists
This adds a `ExecStartPre=-/usr/bin/mkdir -p /var/log/ceph` in all
systemd service templates for all ceph daemon.
This is specific to RHCS after a Leapp upgrade is done. Indeed, the
`/var/log/ceph` seems to be removed after the upgrade.
In order to work around this issue let's ensure the directory is present
before trying to start the containers with podman.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1949489

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit bab403b603)
2021-04-14 20:04:54 +02:00
Guillaume Abrioux 64815ce7ac rbdmirror: add retries/until when configuring mirroring
`configure_mirroring.yml` is called right after the daemon is started.
Sometimes, it can happen the first task in `configure_mirroring.yml` is
run while the daemon isn't yet ready, adding a retries/until on that
task should help to avoid causing the playbook to fail.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1944996

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b1e7e1ad0f)
2021-04-14 16:12:38 +02:00
Guillaume Abrioux 174df8c18e nfs: remove legacy task
This fact is never used, let's remove the task.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0772b3d28d)
2021-04-12 15:32:22 +02:00
Guillaume Abrioux 71a6e6ec33 nfs: rename two tasks
set the name of those tasks accordingly with the fact name being set.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit d3d3d01528)
2021-04-12 15:32:22 +02:00
Guillaume Abrioux e84c42b33f docker2podman: skip some role imports from handler
when running docker-to-podman playbook, there's no need to call
`ceph-config` and `ceph-rgw` from the role `ceph-handler`.
It can even have side effects when coming from a baremetal cluster that
was previously migrated using the switch-to-containers playbook. Indeed
it might complain about missing .target systemd unit since they are
removed during that migration.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1944999

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 70f19be367)
2021-04-12 13:30:09 +02:00
Guillaume Abrioux 69c3d6ea83 common: selinux tasks related refactor
This moves some task from the `ceph-nfs` role in `ceph-common` since
some of them are needed in `ceph-rgwloadbalancer` role.
This avoids duplicated tasks.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit d0442d81b9)
2021-04-06 15:08:38 +02:00
Guillaume Abrioux cc6a10bd02 rgw-loadbalancers: add all rgw_ports to http_port_t type
This adds all rgw ports to the http_port_t selinux type so it
allows haproxy to connect to those ports in order to avoid AVC.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1923890

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 6bbb90198b)
2021-04-06 15:08:38 +02:00
kalebskeithley ef99ac623e rgw-loadbalancer: Update haproxy.cfg.j2
haproxy gets an AVC when configured to connect to port 8081

This commit adds a snippet regarding haproxy in a selinux environment

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1923890

Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
(cherry picked from commit 9e7f22a071)
2021-04-06 15:08:38 +02:00
Dimitri Savineau 21fa7f31b4 container/registry: use password from stdin
Pass the password variable via stdin for the registry login
authentication.
This allows to remove the no_log statement and see the task output
without displaying the password value.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit a0e1a450d3)
2021-04-02 09:46:01 +02:00
Guillaume Abrioux 5210bd16df rgw: supports pg_autoscale_mode option for pool creation
Support enabling/disabling the pg autoscaler for rgw pools.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 9f03a527ba)
2021-04-01 15:32:40 +02:00
Guillaume Abrioux 66235feca3 dashboard: support prometheus storage.tsdb.retention.time parameter
This commit adds the parameter `--storage.tsdb.retention.time` to the
prometheus systemd unit template.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1928000

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b60c61ce45)
2021-04-01 14:52:37 +02:00
Guillaume Abrioux efddbdf909 nfs: set idmap config for Ceph-NFS
Currently NFS Ganesha (ceph-nfs) consumes /etc/idmapd.conf, which
controls mapping of user/owner identities under NFSv4+. With
containerized service deployment, this file is an immutable part of the
container image and cannot be modified.

Here we provide group variables, and a taskk and templates for the
ceph-nfs role, to set the path of the idmap configuration file and
to make the most common adjustment to the contents of that file --
namely to set the 'Domain'. We default the path to /etc/ganesha/idmap.conf
so that we will not conflict with /etc/idmapd.conf on the controller nodes
where ganesha runs. NFSv4 clients, as used for example by the Cinder NFS
driver, consume /etc/idmapd.conf and may require different settings than
what is wanted for NFS Ganesha. Additionally, because we already bind
/etc/ganesha from the host into the ceph-nfs container, the file NFS
Ganesha consumes will no longer be an immutable part of the container.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1925646

Signed-off-by: Tom Barron tpb@dyncloud.net
Co-Authored-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 2db2208e40)
2021-04-01 14:52:12 +02:00
Guillaume Abrioux 94d227149d defaults: add a comment about `igw_network`
This add a quick documentation in ceph-defaults about `igw_network`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit c5728bdc63)
2021-03-29 11:23:49 +02:00
VasishtaShastry 4a10f6ee72 Peer addition won't be skipped if remote is not in peer
rbd-mirroring is not configured as adding peer is getting skipped.
Peer addition should not get skipped if its not added already

Closes - https://bugzilla.redhat.com/show_bug.cgi?id=1942444

Signed-off-by: VasishtaShastry <vipin.indiasmg@gmail.com>
(cherry picked from commit 006998e804)
2021-03-26 21:24:59 +01:00
Guillaume Abrioux 2b19dfdaae dashboard: support igw nodes with dedicated subnet
This adds the possibility to deploy the dashboard with igw nodes using
a dedicated subnet.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1926170

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit c33de174f1)
2021-03-26 19:38:37 +01:00
Guillaume Abrioux 8a4fd99db7 convert some missed `ansible_*`` calls to `ansible_facts['*']`
This converts some missed calls to `ansible_*` that were missed in
initial PR #6312

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0163ecc924)
2021-03-26 00:04:49 +01:00
Alex Schultz 181924db7b Disable facts by default in ansible.cfg
As a continuation of a7f2fa73e6, this
change switches fact injection to off by default in the provided
ansible.cfg.

Signed-off-by: Alex Schultz <aschultz@redhat.com>
(cherry picked from commit db031a4993)
2021-03-26 00:04:49 +01:00
Alex Schultz 56aac327dd Use ansible_facts
It has come to our attention that using ansible_* vars that are
populated with INJECT_FACTS_AS_VARS=True is not very performant.  In
order to be able to support setting that to off, we need to update the
references to use ansible_facts[<thing>] instead of ansible_<thing>.

Related: ansible#73654
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1935406
Signed-off-by: Alex Schultz <aschultz@redhat.com>
(cherry picked from commit a7f2fa73e6)
2021-03-26 00:04:49 +01:00
Guillaume Abrioux 65d1cfd634 iscsi: fetch right repo from shaman
due to recent changes in shaman, we must fetch the right repo by
filtering on the desired architecture.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5801171b37)
2021-03-24 21:36:24 +01:00
Guillaume Abrioux 05ab3a7d50 validate: update `ceph_repository_community` check
this updates the `ceph_repository_community` check in `ceph-validate`
with the right ceph release expected.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 47b9b75ace)
2021-03-18 08:54:33 +01:00
Guillaume Abrioux 01939808b0 nfs: bump nfs-ganesha version
This commit updates the default version of nfs-ganesha to V3.5 which is the
latest version available upstream.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit c78388e580)
2021-03-18 08:54:33 +01:00
Guillaume Abrioux 688e432c32 facts: fix nfs/external cluster scenario
These tasks shouldn't be run when at least 1 monitor isn't present in
the inventory.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1937997

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit ccd1cbb732)
2021-03-18 06:40:33 +01:00
Guillaume Abrioux d65c7b4035 config: reset num_osds
When collocating OSDs with other daemon, `num_osds` is incorrectly calculated
because `ceph-config` is called multiple times.

Indeed, the following code:
```
num_osds: "{{ lvm_list.stdout | default('{}') | from_json | length | int + num_osds | default(0) | int }}"
```

makes `num_osds` be incremented each time `ceph-config` is called.

We have to reset it in order to get the correct number of expected OSDs.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 31a0f2653d)
2021-03-17 17:35:19 +01:00
Matthew Vernon 6deb88d8fb ceph-osd: add prepare_osd tag to lvm-batch scenario
Sometimes it's useful to be able to skip the OSD creation step when
running ceph-ansible (cf #1777). The lvm scenario has a prepare_osd
tag on the relevant play. This commit adds the same tag to the
lvm-batch scenario.

Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
(cherry picked from commit 88d119e95a)
2021-03-12 09:19:55 +01:00
Matthew Vernon 1a67f59789 Fix typo and broken link for documenting RGW frontends
http://docs.ceph.com/docs/nautilus/radosgw/frontends/ 404s so replace
it with a working "pacific" docs link, and correct the spelling of
"additional" while I'm at it.

Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
(cherry picked from commit 847611048e)
2021-03-03 14:17:31 +01:00
Guillaume Abrioux f42ed8e1e0 dashboard: add missing parameter in `ceph_cmd`
the `ceph_cmd` fact is missing the `--net=host` parameter.

Some tasks consuming this fact can fail like following:

```
Error: error configuring network namespace for container b8ec913db1fb694ae683faf202680de7a59c714a004e533aba87e8503d29261f: Missing CNI default network
```

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1931365

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit f143b1a647)
2021-03-03 14:12:46 +01:00
Guillaume Abrioux 7dd4a8a059 tests: use shaman to test against ceph pacific
Given there's no pacific packages available at
https://download.ceph.com, let's use shaman in order to test against
Ceph Pacific

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-12 09:15:24 +01:00
Dimitri Savineau e4dd0067c6 ceph-common: enable rhcs tools repo for monitoring
The monitoring node running grafana needs the rhcs tools repostory
enabled in non containerized deployment to be able to install the
ceph-grafana-dashboards rpm package.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1918650

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-02-10 08:17:11 +01:00
Guillaume Abrioux 931b87e830 rgw: fix a typo in multisite
if `rgw_zonegroupmaster` is not defined at the rgw instance level in
`rgw_instances` it will fallback to a wrong variable (`rgw_zonemaster`).

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1925247

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-10 07:01:21 +01:00
Guillaume Abrioux 39649f0ce8 common: ensure shaman returns right repo
Due to recent changes in shaman, there's a chance it returns the wrong
repository from architecture point of view.
We can query shaman and ask for the correct architecture to get around
this.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-06 00:34:14 +01:00
Guillaume Abrioux c1f627c465 validate: fix a typo
fixes a typo

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-02-06 00:34:14 +01:00
Dimitri Savineau b1f37c4b3d ceph-defaults: use https for download.ceph.com
There's no reason to still use http on download.ceph.com instead of
https.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-02-03 07:14:37 +01:00
Dimitri Savineau 7208a39e57 ceph-facts: set rgw_instances_all fact once
There's no need to set the rgw_instances_all fact for each node. We can
rely on run_once for that one.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-02-01 13:49:12 +01:00
Dimitri Savineau 3749d297c7 ceph-mon: add ExecStartPre docker stop to systemd
We already do that in the other systemd templates (mgr, mds, etc..)
and would present to add workaround in other orchestration tool.
This change is for containerized deployment only.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1882724

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-29 09:03:34 +01:00
Guillaume Abrioux 8617081664 rgw: avoid useless call to ceph-rgw
since `ceph-rgw` may be called from `ceph-handler` in some contexts we
should avoid rerunning it unnecessarily.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-28 14:37:14 -05:00
Guillaume Abrioux 71a5e666e3 rgw: multisite refact
Add the possibility to deploy rgw multisite configuration with a mix of
secondary and primary zones on a same rgw node.
Before that, on a same node, all instances were either primary
zones *OR* secondary.

Now you can define a rgw instance like following:

```
rgw_instances:
  - instance_name: 'rgw0'
    rgw_zonemaster: false
    rgw_zonesecondary: true
    rgw_zonegroupmaster: false
    rgw_realm: 'france'
    rgw_zonegroup: 'zonegroup-france'
    rgw_zone: paris-00
    radosgw_address: "{{ _radosgw_address }}"
    radosgw_frontend_port: 8080
    rgw_zone_user: jacques.chirac
    rgw_zone_user_display_name: "Jacques Chirac"
    system_access_key: P9Eb6S8XNyo4dtZZUUMy
    system_secret_key: qqHCUtfdNnpHq3PZRHW5un9l0bEBM812Uhow0XfB
    endpoint: http://192.168.101.12:8080
```

Basically it's now possible to define `rgw_zonemaster`,
`rgw_zonesecondary` and `rgw_zonegroupmaster` at the intsance
level instead of the whole node level.

Also, this commit adds an option `deploy_secondary_zones` (default True)
which can be set to `False` in order to explicitly ask the playbook to
not deploy secondary zones in case where the corresponding endpoint are
not deployed yet.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1915478

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-27 15:46:43 +01:00
Dimitri Savineau bbcad9609c grafana: update container tag to 6.7.4
This update the grafana container tag to 6.7.4.
The RHCS version is now based on the RHCS 5 container image which is
also based on 6.7.4.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-27 15:08:31 +01:00
Dimitri Savineau 7d56771975 ceph-defaults: change default ceph container tag
The "latest" ceph container tag references the latest stable release
(octopus at the moment). "latest" is an alias on "latest-octopus".
On the devel branch we should use "latest-master" tag instead.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-22 21:12:34 +01:00
Guillaume Abrioux 4af0845702 mon: fix cephx disabled deployment
Due to missing condition on `cephx` variable, cephx disabled deployments
are broken.
This commit fixes this.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1910151

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-18 11:30:02 -05:00
Guillaume Abrioux ae196bf946 validate: check virtual_ips variable
This commit checks the length of `virtual_ips` doesn't exceed the length
of `groups[rgwloadbalancer_group_name]`.
It also ensure this variable is defined when
`groups[rgwloadbalancer_group_name]` contains at least one node.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-12 11:03:12 +01:00
Benoît Knecht 3116f46422 ceph-rgw-loadbalancer: Fix keepalived master selection
While 2ca33641 fixed a bug in the way the `keepalived.conf.j2` template matched
hostnames to set the VRRP `MASTER`/`BACKUP` states, it also introduced a
regression in the case where `virtual_ips` is a list of more than one IP
address.

The previous behavior would result in each host in the `rgwloadbalancers` group
to be `MASTER` for one of the `virtual_ips`, but the new behavior caused the
first host to be `MASTER` for all the IP address in `virtual_ips`.

This commit restores the original behavior.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
2021-01-12 11:03:12 +01:00
Dimitri Savineau 3f64ced36b ceph-osd: replace sysctl command task by slurp
Instead of using the command module for retrieving a sysctl value then
we can use the slurp module and read the value directly from /proc.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-01-11 13:24:23 +01:00
Guillaume Abrioux ef975ef5ea dashboard: configure passwords via stdin
Due to recent changes in ceph, the few dashboard passwors
must be passed via `-i`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-07 17:09:46 -05:00
Mike Currin 4cbc9a48c9 Path for ceph config missing in crash template
The path where ceph.conf is located (/etc/ceph) missing in the Docker container bind mounts, this throws errors

Signed-off-by: Mike Currin <currin@gmail.com>
2021-01-06 16:50:18 +01:00
Guillaume Abrioux 513c8cfe55 rgw: support switching from single-site to multisite
When collocating rgw with either a mon, mgr or osd, switching from
single site to a multisite rgw setup failed because of the handlers
triggered between the ansible play of the collocated daemon and the play
of the rgw. Since the multisite changes are not yet applied the handlers
fail.
The idea here is to ensure we run the multisite configuration from the
ceph-handler role before the restart happens, this way it won't complain
because of non existing multisite configuration.

(Note: this is also valid when simply changing a multisite configuration)

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1888630

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-01-06 09:58:45 -05:00
Fabien Brachere 4026ba9da1 library: add missing `target_size_ratio` parameter support in ceph_pool module
When creating a new pool, target_size_ratio was ignored by ansible module ceph_pool.py.
target_size_ratio is now used when pg_autoscale_mode is on.
Tests added to library tests.
This adds too the use in the role ceph-rgw.

Signed-off-by: Fabien Brachere <fabien.brachere@celeste.fr>
2020-12-16 15:10:27 +01:00
Dimitri Savineau 827b23353f ceph-config: fix ceph-volume lvm batch report
Since the major ceph-volume lvm batch refactoring, the report value
is different.
Before the refact, the report was a dict with the OSDs list to be created
under the "osds" key.
After the refact, the report is a list of dict.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-15 21:19:04 +01:00
Karl-Heinz Preuß 6ce34ef59f fix broken ceph-fetch-keys role
set fetch_directory variable in default/main.yml instead of using the
defaults jinja filter in tasks/main.yml.

Fixes: #6072

Signed-off-by: Karl-Heinz Preuß <karl-heinz.preuss@cms.hu-berlin.de>
2020-12-14 17:36:17 +01:00
Seena Fallah 5e9444fa5c ceph-osd: use global crush_device_class in lvm_volumes
Use global crush_device_class variable if it's not set per OSD

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2020-12-12 06:56:53 +01:00
Dimitri Savineau aa6e1f20ea Revert "config: Always use osd_memory_target if set"
This reverts commit 4d1fdd2b05.

This breaks the backward compatibility with previous osd_memory_target
calculation and we could have a value lower than the minimum value allowed
(896M) which causes some ceph commands to fail (like ceph assimilate-conf).

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-12 06:56:32 +01:00
Dimitri Savineau 5a41026347 monitoring: use config_template module for config
The alertmanager, grafana and prometheus configuration file are
generated with the template module which doesn't allow for using
config overrides.
Instead we could use the config_template plugin action and add a
new variable for overrides (one for each component).

With this patch, one should be able to add configuration to
prometheus with the following:

---
alertmanager_conf_overrides:
  global:
    smtp_smarthost: 'localhost:25'
...

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1902999

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-12 06:55:27 +01:00
Dimitri Savineau d82249a8c0 ceph-rgw: add cluster parameter on ceph_ec_profile
81233dd introduced a regression with the ceph_ec_profile module call in
the ceph-rgw role due the missing cluster module parameter.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-12 06:54:46 +01:00
Dimitri Savineau 2aeab882f3 ceph-facts: fix grafana group conversion
The conversion fact task was only executed when the grafana_server_group_name
variable was explicitly set in the user configuration. If an user was using
the default value then the conversion wasn't executed.

This also adds back the default grafana_server_group_name value in case user
was using the default value and to avoid undefined variable error.

Instead of hardcoding the "monitoring" group name then we can reuse the
monitoring_group_name variable.

There's no need to override the monitoring_group_name variable, it's either
using the default value or the one defined by the user.

Finally removing the delegate_to statement on the add_host task since it's
always executed on the ansible controller.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1903732

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-10 16:51:16 +01:00
Jukka Nousiainen eb7473491b ceph-mon: No become during gen mon initial keyring
Since the backing generate_secret() just hands out urandom output,
running as privileged doesn't seem to be required. It's not
desireable to provide sudo in some Ansible runner environments.

Signed-off-by: Jukka Nousiainen <jukka.nousiainen@csc.fi>
2020-12-03 10:04:21 +01:00
Guillaume Abrioux 86a8889ee3 common: do not use pipefail when not needed
Let's discard the ansible lint error 306 and add a "# noqa 306" on tasks
where we don't need `set -o pipefail`

Fixes: #6090

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-12-01 15:07:09 -05:00
Dimitri Savineau cf7345f143 consume ceph_volume module when possible
We should always use the ceph_volume ansible module when possible.
This patch replace the ceph-volume inventory and lvm {list,zap} commands
called via the command/shell modules by the corresponding call with the
ceph_volume module.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-12-01 17:54:10 +01:00