Commit Graph

5420 Commits (31ad2d9338f226e559df17fffad0209dcab17437)
 

Author SHA1 Message Date
Guillaume Abrioux 31ad2d9338 workflows: add signed-off check
This adds a github workflow for checking the signed off line in commit
messages.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 8c09497567)
2021-06-30 09:52:44 +02:00
Guillaume Abrioux e899b84f6f workflow: add group_vars/defaults checks
let's use github workflow for checking defaults values.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit d71db816c6)
2021-06-30 09:52:44 +02:00
Guillaume Abrioux d22406bb6d workflow: add syntax check
This adds the ansible --syntax-check test in the ansible-lint workflow

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5ed423ad88)
2021-06-30 09:52:44 +02:00
Dimitri Savineau 695954023e workflow/lint: pin ruamel.yaml.clib to 0.2.2
Recent release is broken with py27.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-06-30 09:32:34 +02:00
Dimitri Savineau 6f3dc45175 tox: add ceph_stable_release to switch2container
We need to set the ceph_stable_release variable during the switch2container
playbook.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-06-30 09:32:34 +02:00
Dimitri Savineau 25ea12d31d switch2container: run ceph-validate role
This adds the ceph-validate role before starting the switch to a containerized
deployment.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1968177

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit fc160b3be1)
2021-06-30 09:32:34 +02:00
Guillaume Abrioux 0fa1cf0cdf nfs: do no copy client.bootstrap-rgw when using mds
There's no need to copy this keyring when using nfs with mds

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 8dbee99882)
2021-06-17 08:15:53 +02:00
VasishtaShastry 7bc9e391cb Container: Fixing service name lvm2-lvmetad
Playbook failing saying:

msg: 'Could not find the requested service lvmetad: host'

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1955040
Signed-off-by: VasishtaShastry <vipin.indiasmg@gmail.com>
(cherry picked from commit e49c38f8b7)
2021-06-17 08:14:54 +02:00
Guillaume Abrioux 98eb93db3e multisite: fix bug during switch2containers
When running the switch-to-containers playbook with multisite enabled,
the fact "rgw_instances" is only set for the node being processed
(serial: 1), the consequence of that is that the set_fact of
'rgw_instances_all' can't iterate over all rgw node in order to look up
each 'rgw_instances_host'.

Adding a condition checking whether hostvars[item]["rgw_instances_host"]
is defined fixes this issue.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1967926

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 8279d14d32)
2021-06-17 07:20:37 +02:00
Guillaume Abrioux 611494b88f rolling_update: fix mon+rgw/multisite collocation
When monitors and rgw are collocated with multisite enabled, the
rolling_update playbook fails because during the workflow, we run some
radosgw-admin commands very early on the first mon even though this is
the monitor being upgraded, it means the container doesn't exist since
it was stopped.

This block is relevant only for scaling out rgw daemons or initial
deployment. In rolling_update workflow, it is not needed so let's skip
it.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1970232

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit f7166cccbf)
2021-06-14 13:59:16 +02:00
Guillaume Abrioux 71764c3440 tests: disable test_mgr_dashboard_is_listening
Due to a recent commit that has introduced a regression in ceph, this
test is failing.
Temporarily disabling it to unblock the CI.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 2e19d1705e)
2021-06-07 15:12:43 +02:00
Guillaume Abrioux d00e4b3d9e dashboard: set cookie_secure in grafana
When using grafana behind https `cookie_secure` should be set to `true`.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1966880

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 4daed1f137)
2021-06-07 15:12:43 +02:00
Guillaume Abrioux f3b992328a dashboard: fix rgw user creation
When deploying dashboard in a cluster with rgw multisite deployed.
Due to the last rgw multisite refactor, we now expect the variable
`rgw_zonemaster` to be defined in the dict `rgw_instances`.
The idea here is to create that user on the cluster as soon as we have 1
`rgw_zonemaster` set to `true` in `rgw_instances`.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1964995

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-06-03 17:07:02 +02:00
Guillaume Abrioux d2bd0961d4 crash: fix --limit deployments (containers)
ceph-crash deployments is broken when ceph-ansible playbook is called
with --limit in containerized contexts since we don't set
`container_exec_cmd` on the first monitor.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1964835

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 10ed26f14d)
2021-05-26 18:51:48 +02:00
Guillaume Abrioux a391dad8e1 dashboard: fix typo introduced during backport
during backport of c8b92deba1 the pattern
should have been s/monitoring_group_name/grafana_server_group_name/

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1964907

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit ac0a5c1e68)
2021-05-26 18:51:18 +02:00
Guillaume Abrioux 8b4eb0f108 prometheus: enforce osd nodes in templates
When osd nodes are collocated in the clients group (HCI context for
instance), the current logic will exclude osd nodes since they are
present in the client group.

The best fix would be to exclude clients node only when they are not
member of another group but for now, as a workaround, we can enforce
the addition of osd nodes to fix this specific case.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1947695

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 664dae0564)
2021-05-26 10:57:51 +02:00
Guillaume Abrioux 0eed858952 fs2bs: use match filter in selectattr()
0990ae4109 changed the filter in
selectattr() from 'match' to 'equalto' but due to an incompatibility with
the Jinja2 version for python 2.7 on el7 we must stick to using 'match'
filter.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit d6745e9cd9)
2021-05-26 09:58:19 +02:00
Guillaume Abrioux abebf9b23e fs2bs: fix wrong filter when setting osd_ids
using 'match' filter in that task will lead to bad behavior if I have
the following node names for instance:

- node1
- node11
- node111

with `selectattr('name', 'match', inventory_hostname)` it will match
'node1' along with 'node11' and 'node111'.

using 'equalto' filter will make sure we only match the target node.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1963066

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0990ae4109)
2021-05-26 09:58:19 +02:00
Guillaume Abrioux a27761855b container: conditionnally disable lvmetad
Enabling lvmetad in containerized deployments on el7 based OS might
cause issues.
This commit make it possible to disable this service if needed.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1955040

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-05-25 16:51:04 +02:00
Brad Hubbard 5d3a46e6fd Make sure the repo url contains the correct arch
We can end up with an arm only repo unless we are specific about the
architecture we require. Brings the deb code in line with the rpm
equivalent.

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 267cce9e83)
2021-05-17 08:57:19 +02:00
Guillaume Abrioux 6999118fb6 validate: check virtual_ips variable
This commit checks the length of `virtual_ips` doesn't exceed the length
of `groups[rgwloadbalancer_group_name]`.
It also ensure this variable is defined when
`groups[rgwloadbalancer_group_name]` contains at least one node.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3b63e0649c)
2021-05-05 09:56:42 +02:00
Benoît Knecht 97066a1ebc ceph-rgw-loadbalancer: Fix keepalived master selection
While 2ca33641 fixed a bug in the way the `keepalived.conf.j2` template matched
hostnames to set the VRRP `MASTER`/`BACKUP` states, it also introduced a
regression in the case where `virtual_ips` is a list of more than one IP
address.

The previous behavior would result in each host in the `rgwloadbalancers` group
to be `MASTER` for one of the `virtual_ips`, but the new behavior caused the
first host to be `MASTER` for all the IP address in `virtual_ips`.

This commit restores the original behavior.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 2bede4762e)
2021-05-05 09:56:42 +02:00
Guillaume Abrioux 2d59f4579b update: fix ceph-crash stop task
This is a workaround for an issue in ansible.
When trying to stop/mask/disable this service in one task, the stop
didn't actually happen, the task doesn't fail but for some reason the
container is still present and running.
Then the task starting the service in the role ceph-crash fails because
it can't start the container since it's already running with the same
name.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1955393

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3db1ea7ec4)
2021-05-05 09:47:32 +02:00
Seena Fallah 4f22dc477e ceph-osd: allow to use ceph_tcmalloc_max_total_thread_cache for bluestore
TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES is for both bluestore and filestore

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
(cherry picked from commit 41295f0ef6)
2021-04-29 07:34:55 +02:00
Benoît Knecht 104ba407f2 ceph-mon: Fix check mode for deploy monitor tasks
Skip the `get initial keyring when it already exists` task when both commands
whose `stdout` output it requires have been skipped (e.g. when running in check
mode).

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit e98d9b70bd)
2021-04-28 10:00:30 +02:00
Guillaume Abrioux 650964a8c7 fs2bs: add a final play
This removes the fact `skipped_nodes` which is useless when we run with
`--limit` since it gets reset when a new iteration is made.

Instead, let's print within a final play which node has been skipped
reusing the `skip_this_node` fact.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3d4267051f)
2021-04-28 08:55:34 +02:00
Francesco Pantano 86091f5ba3 Config the monitoring stack components api urls using a VIP
When dashboard_frontend_vip is provided, all the services should be
configured using the related VIP. A new VIP variable is added for
both prometheus and alertmanager: we're already able to properly
config the grafana vip using dashboard_frontend_vip variable.
This change adds the same variable for both prometheus and
alertmanager.

Signed-off-by: Francesco Pantano <fpantano@redhat.com>
(cherry picked from commit 441651638d)
2021-04-28 08:54:21 +02:00
Guillaume Abrioux 871ad6d71a osd: always allow setting target_size_ratio
We shouldn't prevent from setting target_size_ratio when the autoscaler
is set to 'warn'.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1906305

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-04-15 14:47:38 +02:00
Benoît Knecht 9a286e273a ceph-rgw-loadbalancer: Fix rgw_ports fact
The `set_fact rgw_ports` task was failing due to a templating error, because
`hostvars[item].rgw_instances` is a list, but it was treated as if it was a
dictionary.

Another issue was the fact that the `unique` filter only applied to the list
being appended to `rgw_ports` instead of the entire list, which means it was
possible to have duplicate items.

Lastly, `rgw_ports` would have been a list of integers, but the `seport` module
expects a list of strings.

This commit fixes all of the issues above, allowing the `ceph-rgw-loadbalancer`
role to work on systems with SELinux enabled.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit c078513475)
2021-04-15 13:21:07 +02:00
Guillaume Abrioux b4340b71d9 switch-to-containers: only chown corresponding files
When collocating daemons, if we chown all files under `/var/lib/ceph` it
can cause issues for the collocated daemons that wouldn't have been
migrated yet.

This commit makes the playbook chown only the files corresponding to the
daemon being migrated.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit ddbc11c4a9)
2021-04-15 05:24:44 +02:00
Guillaume Abrioux ddd7c42c2b container/systemd: ensure /var/log/ceph exists
This adds a `ExecStartPre=-/usr/bin/mkdir -p /var/log/ceph` in all
systemd service templates for all ceph daemon.
This is specific to RHCS after a Leapp upgrade is done. Indeed, the
`/var/log/ceph` seems to be removed after the upgrade.
In order to work around this issue let's ensure the directory is present
before trying to start the containers with podman.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1949489

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit bab403b603)
2021-04-14 20:46:09 +02:00
Guillaume Abrioux 4db92dae59 rbdmirror: add retries/until when configuring mirroring
`configure_mirroring.yml` is called right after the daemon is started.
Sometimes, it can happen the first task in `configure_mirroring.yml` is
run while the daemon isn't yet ready, adding a retries/until on that
task should help to avoid causing the playbook to fail.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1944996

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b1e7e1ad0f)
2021-04-14 16:13:01 +02:00
Guillaume Abrioux 3ef9690cd1 docker2podman: skip some role imports from handler
when running docker-to-podman playbook, there's no need to call
`ceph-config` and `ceph-rgw` from the role `ceph-handler`.
It can even have side effects when coming from a baremetal cluster that
was previously migrated using the switch-to-containers playbook. Indeed
it might complain about missing .target systemd unit since they are
removed during that migration.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1944999

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 70f19be367)
2021-04-12 13:30:31 +02:00
Guillaume Abrioux c0c90c6747 docker2podman: add documentation/header
this adds a small documentation in the header of the playbook in order
to explain what is the goal of this playbook.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 36b4227dcd)
2021-04-12 09:45:20 +02:00
Guillaume Abrioux 3bf2c45123 switch_to_containers: support iscsigws migration
This adds the iscsigws migration to containers.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=<bz-number>

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 2c74c27321)
2021-04-09 15:28:27 +02:00
Guillaume Abrioux f47da73a8a common: selinux tasks related refactor
This moves some task from the `ceph-nfs` role in `ceph-common` since
some of them are needed in `ceph-rgwloadbalancer` role.
This avoids duplicated tasks.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit d0442d81b9)
2021-04-06 15:09:00 +02:00
Guillaume Abrioux 3bfa0772e2 rgw-loadbalancers: add all rgw_ports to http_port_t type
This adds all rgw ports to the http_port_t selinux type so it
allows haproxy to connect to those ports in order to avoid AVC.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1923890

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 6bbb90198b)
2021-04-06 15:09:00 +02:00
kalebskeithley 3ea39b5db3 rgw-loadbalancer: Update haproxy.cfg.j2
haproxy gets an AVC when configured to connect to port 8081

This commit adds a snippet regarding haproxy in a selinux environment

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1923890

Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
(cherry picked from commit 9e7f22a071)
2021-04-06 15:09:00 +02:00
Guillaume Abrioux b79c2070f4 nfs: set idmap config for Ceph-NFS
Currently NFS Ganesha (ceph-nfs) consumes /etc/idmapd.conf, which
controls mapping of user/owner identities under NFSv4+. With
containerized service deployment, this file is an immutable part of the
container image and cannot be modified.

Here we provide group variables, and a taskk and templates for the
ceph-nfs role, to set the path of the idmap configuration file and
to make the most common adjustment to the contents of that file --
namely to set the 'Domain'. We default the path to /etc/ganesha/idmap.conf
so that we will not conflict with /etc/idmapd.conf on the controller nodes
where ganesha runs. NFSv4 clients, as used for example by the Cinder NFS
driver, consume /etc/idmapd.conf and may require different settings than
what is wanted for NFS Ganesha. Additionally, because we already bind
/etc/ganesha from the host into the ceph-nfs container, the file NFS
Ganesha consumes will no longer be an immutable part of the container.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1925646

Signed-off-by: Tom Barron tpb@dyncloud.net
Co-Authored-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 2db2208e40)
2021-04-02 13:18:52 +02:00
Guillaume Abrioux b2cf677b71 dashboard: support prometheus storage.tsdb.retention.time parameter
This commit adds the parameter `--storage.tsdb.retention.time` to the
prometheus systemd unit template.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1928000

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b60c61ce45)
2021-04-02 13:17:59 +02:00
Guillaume Abrioux 5fd299e358 update: followup on 07029e1
Playbook must fail anyway, the `rescue` block has been introduced for
unmasking the unit after the playbook has failed.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit e9ddb972fe)
2021-03-29 15:22:23 +02:00
Guillaume Abrioux 82b934cfc1 rolling_update: unmask monitor service after a failure
if for some reason the playbook fails after the service was
stopped, disabled and masked and before it got restarted, enabled and
unmasked, the playbook leaves the service masked and which can make users
confused and forces them to unmask the unit manually.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1917680

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 07029e1bf1)
2021-03-29 15:22:23 +02:00
Guillaume Abrioux 653d180ec0 defaults: add a comment about `igw_network`
This add a quick documentation in ceph-defaults about `igw_network`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit c5728bdc63)
2021-03-29 11:24:28 +02:00
Guillaume Abrioux fe47a02134 dashboard: support igw nodes with dedicated subnet
This adds the possibility to deploy the dashboard with igw nodes using
a dedicated subnet.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1926170

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit c33de174f1)
2021-03-26 21:26:14 +01:00
VasishtaShastry 58a28656ff Peer addition won't be skipped if remote is not in peer
rbd-mirroring is not configured as adding peer is getting skipped.
Peer addition should not get skipped if its not added already

Closes - https://bugzilla.redhat.com/show_bug.cgi?id=1942444

Signed-off-by: VasishtaShastry <vipin.indiasmg@gmail.com>
(cherry picked from commit 006998e804)
2021-03-26 19:14:35 +01:00
Guillaume Abrioux a8420d41c6 update: stop ceph-crash service before upgrading
This adds the missing service stop task for ceph-crash upgrade workflow.

It should have been added through commit
`15872e3db1e342238636bc9c8e1aef6bd1d3dcd8` in stable-4.0 but at the time
we backported this patch ceph-crash wasn't implemented yet so the
ceph-crash related content in this patch was removed. Then, ceph-crash
has been implemented later so we are still missing this part of the patch in
stable-4.0.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1943471

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-03-26 16:18:50 +01:00
Guillaume Abrioux a65968a9d1 tests: pin ruamel.yaml version
0.17.0 which was released today (03/26/2021) breaks ansible-lint execution with
py2.7.

From https://pypi.org/project/ruamel.yaml we can read:

> The 0.16.13 release was the last that will tested to be working on Python 2.7.

Let's enforce the version on 0.16.13 when running with py2.7

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-03-26 14:42:27 +01:00
Ali Maredia ba6fa5c959 docs: rgw multisite docs with new rgw_instances config
Docs reflect that each instance of `rgw_instances`
can now take rgw_zonemaster, rgw_zonesecondary,
rgw_zonegroupmaster, rgw_multisite_proto.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
(cherry picked from commit a59bc2da3b)
2021-03-26 07:43:02 +01:00
Guillaume Abrioux 9780490b2f convert some missed `ansible_*`` calls to `ansible_facts['*']`
This converts some missed calls to `ansible_*` that were missed in
initial PR #6312

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0163ecc924)
2021-03-26 00:16:58 +01:00
Alex Schultz 6229b3bdba Disable facts by default in ansible.cfg
As a continuation of a7f2fa73e6, this
change switches fact injection to off by default in the provided
ansible.cfg.

Signed-off-by: Alex Schultz <aschultz@redhat.com>
(cherry picked from commit db031a4993)
2021-03-26 00:16:58 +01:00