Commit Graph

5535 Commits (6485e1a69ed90b446c2cf2d9fdbf703cd8105d6d)
 

Author SHA1 Message Date
Guillaume Abrioux 6485e1a69e purge: remove ceph directories on client nodes
Otherwise any ceph directories are left over on client nodes
after the purge.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2024815

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 20035852a4)
(cherry picked from commit 346d4a1e1d)
2022-05-19 18:00:13 +02:00
Guillaume Abrioux 36fceeaaff update: speed up client play
there's no need to run the roles ceph-facts, ceph-config and ceph-client
altogether on client nodes in rolling update playbook.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2019831

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 817c03bc0e)
(cherry picked from commit c0da98b1d6)
2022-05-19 17:58:01 +02:00
Guillaume Abrioux 8d9ebf2d0b container: align systemd units with rpm
Update `After=` and `Wants=` parameters in container systemd units
and make them be aligned with the systemd units that come
from the packaging.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2027440

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit f01536ea19)
(cherry picked from commit 690c879aef)
2022-05-19 17:55:17 +02:00
Guillaume Abrioux f4bd5c7d91 switch2containers: fail if less than 3 monitors
This playbook doesn't support less than 3 monitors present in the inventory.
Just like the rolling_update playbook, let's fail if less than
3 monitors are present.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2049132

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit f08129edf2)
(cherry picked from commit b970ab6691)
2022-05-19 17:53:53 +02:00
Guillaume Abrioux fe4e04f779 dashboard: allow collecting stats from the host
This commit makes podman bindmount `/:/rootfs:ro` so the container can
collect data from the host.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2028775

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0f34cd16d8)
(cherry picked from commit 2e2d893d28)
2022-05-09 13:46:05 +02:00
Guillaume Abrioux dbe940f1a7 purge: ceph-crash purge fixes
This fixes the service file removal and makes the playbook
call `systemctl reset-failed` on the service because in Ceph
Nautilus, ceph-crash doesn't handle `SIGTERM` signal.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2055992

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 2f11982590)
(cherry picked from commit 7a570c719e)
2022-05-09 13:45:16 +02:00
Guillaume Abrioux 26b396cb6c common: support setting pg autoscaler to off
The current implementation doesn't allow to disable the pg autoscaler
on created pools. This allows only 'on' or 'warn'.

With this commit, this is now possible to disable it.

Valid values would be ['on', 'yes', 'true', 'off', 'no', 'false']

```
openstack_glance_pool:
  name: "images"
  pg_num: 128
  pgp_num: 128
  rule_name: "replicated_rule"
  type: 1
  application: "rbd"
  size: 3
  pg_autoscale_mode: off
```

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2062621

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 9d1ff8f236)
2022-05-09 13:44:31 +02:00
Teoman ONAY b22e1b87d1 Turn off SELinux separation for containers MON and RGW
Initially MONs and RGW binded /etc/pki/ca-trust/extracted using the :z flag
(introduced to solve an OSP TripleO issue on RHEL - #3638) but using
this flag prevents local services (like sssd) running on the host from accessing
the certificates/files in that folder.

Signed-off-by: Teoman ONAY <tonay@redhat.com>
(cherry picked from commit 7e8ce2567e)
(cherry picked from commit cf44ad76f6)
2022-05-09 13:44:04 +02:00
Guillaume Abrioux 45b3e50e25 facts: follow up on aa0cc93
when these variables are defined in the inventory host file,
all tasks are skipped then because the node being played isn't
aware about the values from the rgw nodes.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2063029

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 328bd7c975)
2022-04-21 11:34:18 +02:00
Guillaume Abrioux 8d39897f38 facts: fix mon/mgr collocation
`service dump` hangs when no active mgr is available.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 617dce5e10)
2022-04-20 06:47:23 +02:00
Guillaume Abrioux 1453915ed9 dashboard: fix regression
introduced by ceph/ceph-ansible/pull/7150

when no rgw is present, it fails.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2076192

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 1a56fd6a21)
2022-04-20 06:47:16 +02:00
Guillaume Abrioux c4ec2c4e54 dashboard: support --limit execution with rgw
When the following conditions are met:

- rgw is deployed,
- dashboard is deployed,
- playbook is called with --limit,
- a node being processed is collocated on either a mon or mgr.

The playbook fails because `rgw_instances` is undefined.
The idea here is to make sure this variable is always defined.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2063029

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit aa0cc9381d)
2022-04-14 08:50:33 +02:00
Guillaume Abrioux 49b8e0d89c dashboard: always set `dashboard_server_addr`
When running the playbook with `--limit`, if the play targeted doesn't match
hosts present in the mgr group the playbook can fail.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2063029

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 72e4654aae)
(cherry picked from commit d1e4b83106)
2022-03-28 10:40:36 +02:00
Guillaume Abrioux 3c05ac6c46 dashboard: fix radosgw system user creation
The radosgw system user creation will fail when `rgw_instances`
is set at the host_var level because this variable won't bet set
on monitor nodes, given that this is where the tasks is delegated, it fails.

The idea here is to check over all rgw instances that are defined and set a
boolean fact in order to check if at least one instance has `rgw_zonemaster` set
to `True`

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2034595

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2022-01-03 10:22:09 +01:00
Guillaume Abrioux de14c6aeb2 validate: fix bug when using vault
since a variable encrypted with vault is no longer a string but a
encrypted object we can't use the filter | length, we have to convert it
to a string before.

Fixes: #6991

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 6ad7e52869)
2021-11-29 13:42:43 +01:00
Guillaume Abrioux f5dd0a8c37 mgr: append balancer module to ceph_mgr_modules
otherwise the osd play in rolling_update can fail when it tries to
disable it before upgrading osd nodes.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 45a1d634d8)
2021-11-10 14:10:30 +01:00
Guillaume Abrioux 48a8b1cc34 update: move a set_fact
ceph-facts roles makes decisions based on the fact `rolling_update` so
it must be called before we run this role.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2014304

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit e5edcc4214)
2021-11-03 11:50:50 +01:00
Guillaume Abrioux a9a7c35a74 update: support --limit on monitor nodes
Change needed in order to support --limit on mon nodes.
Otherwise, a call to `hostvars[groups[mon_group_name][0]]['_current_monitor_address']`
throws an error:

```
"The error was: 'ansible.vars.hostvars.HostVarsVars object' has no attribute '_current_monitor_address'"
```

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2014304#c28

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 82eee4303b)
2021-10-29 01:41:13 +02:00
Guillaume Abrioux b8db1166c5 nfs/rgw: support enforcing keys
if one sets `ceph_nfs_rgw_access_key` and/or `ceph_nfs_rgw_secret_key`,
the nfs/rgw user creation won't take those variables into account and it
will generate a user with automatically generated credentials.
It ends up with a mismatch between what will be set in ganesha.conf and
the created user.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2010754

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-10-26 16:39:58 +02:00
Per Abildgaard Toft c5e4851a3f shrink-osd: fix regression because of a wrong regex
968891f449 introduced a regression.
The regex is wrong because it doesn't allow to shrink osds with id
greater than 9

Fixes: #6950

Signed-off-by: Per Abildgaard Toft <per@minfejl.dk>
(cherry picked from commit 84118a3063)
2021-10-26 16:39:33 +02:00
Guillaume Abrioux 3f4abb09b4 shrink-osd: check osd id format
This adds a check early in order to ensure the format of osd ids passed
is correct.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2005734

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 968891f449)
2021-10-26 16:39:33 +02:00
Guillaume Abrioux 2c9fc7f517 rolling_update: modify default health_osd_check_*
let's do more retries with a shorter delay.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 50a21d695e)
2021-10-26 16:39:09 +02:00
Guillaume Abrioux a15448e5cc tests: add new scenario subset_update
new scenario in order to test the subset upgrade approach using tags.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit fb8a66149b)
2021-10-25 23:22:35 +02:00
Guillaume Abrioux 3dd96da652 rolling_update: fix pre and post osd upgrade play
when using --limit osds, the play before and after osd upgrade are
skipped because we use `hosts: "{{ mon_group_name | default('mons') }}[0]"`
using `hosts: "{{ osds_group_name | default('osds') }}" with
`delegate_to` to the first monitor addresses this issue.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit fc9f87c45f)
2021-10-25 23:22:35 +02:00
Guillaume Abrioux dc1a4c29ea update: support upgrading a subset of nodes
It can be useful in a large cluster deployment to split the upgrade and
only upgrade a group of nodes at a time.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2014304

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit e5cf9db2b0)
2021-10-25 23:22:35 +02:00
Guillaume Abrioux c5a8343417 tests: ensure ca-certificates is up to date
otherwise the `rpm_key` module fails because it can't verify the
certificate.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2021-10-02 15:48:31 +02:00
Guillaume Abrioux a890d6a043 tests: remove all references to ceph_stable_release
this is legacy and not needed anymore.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit f277a39dfe)
2021-10-02 15:48:31 +02:00
Seena Fallah 7495de3120 ceph-defaults: set ceph_stable_release default to the stable branch release
ceph_stable_release is a legacy from the time where a single branch of ceph-ansible supported more than one release of ceph

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
(cherry picked from commit fb99626987)
2021-10-02 15:48:31 +02:00
Guillaume Abrioux a8026b3516 tests: set rgw_instances in collect-logs.yml
in order to gather rgw logs, we need rgw_instances to be set.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit c2e46fe5a5)
2021-09-30 17:54:31 +02:00
Guillaume Abrioux dda2368892 tests: update collect-logs.yml playbook
- change `ceph -s` output to json-pretty.
- gather rgw logs
- add `health detail` command

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b2ccc7234a)
2021-09-30 17:49:29 +02:00
Guillaume Abrioux 6dd85f3344 tests: move collect-logs.yml to ceph-ansible repo
related ceph-build PR: ceph/ceph-build#1914

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 702564518b)
2021-09-29 16:41:40 +02:00
Seena Fallah 0a93de938b purge: add remove_docker tag
This can help to skip docker removal tasks

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
(cherry picked from commit ff39c8d70b)
2021-09-14 20:50:06 +02:00
Seena Fallah 0ede37b2ec purge: add container_binary needed for zap osds
`container_binary` isn't set anymore in the purge osd play because of a
regression introduced by 60aa70a.
The CI didn't catch it because the play purging node-exporter sets this
variable for all nodes before we run the purge osd play.

This commit fixes this regression.

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
(cherry picked from commit a51ce767ca)
2021-09-09 14:40:53 +02:00
Dimitri Savineau ce80ed946c ceph-defaults: set quay.io as the default registry
Because the ceph container images are now only pushed to the quay.io
registry then this updates the default registry value.
The docker.io registry can still be used but doesn't receive updated
container images.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit e7b43c1fc6)
2021-09-09 13:48:14 +02:00
Seena Fallah e7b0af31c3 ceph-container-engine: allow override container_package_name and container_service_name
Only include specific variables when they are undefined

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
(cherry picked from commit 95bce32270)
2021-09-09 13:25:00 +02:00
Dimitri Savineau 0d670c7942 purge-dashboard: remove cid files
This adds the service cid file cleanup as supported in the classic purge
playbook since b9dd253

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1786691

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit cddc23f511)
2021-09-08 12:05:33 -04:00
Dimitri Savineau 3aba6c8e4e tests/rgw: use json format output for user info
If the radosgw user already exists then we need to have the output in json
format because we are expecting to load the output with json.loads()
Otherwise we have pytest failure like:

```console
self = <json.decoder.JSONDecoder object at 0x7fa2f00a5fd0>, s = '', idx = 0

    def raw_decode(self, s, idx=0):
        """Decode a JSON document from ``s`` (a ``str`` beginning with
        a JSON document) and return a 2-tuple of the Python
        representation and the index in ``s`` where the document ended.

        This can be used to decode a JSON document from a string that may
        have extraneous data at the end.

        """
        try:
            obj, end = self.scan_once(s, idx)
        except StopIteration as err:
>           raise JSONDecodeError("Expecting value", s, err.value) from None
E           json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit f2bd8ae70f)
2021-08-27 14:40:37 -04:00
Dimitri Savineau e1dd35f6d6 tests/rgw: add timeout 5s to radosgw-admin command
If the radosgw daemons aren't up and running correctly (like not registered
in the servicemap or the OSD are down) then the radosgw-admin will hang
forever.
Jenkins will kill the jobs after 3h but we don't want to wait until this global
timeout.
Adding the timeout 5 command to the radosgw-admin commands (which is already
present on other ceph calls) allows the job to fail earlier.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit f01ae82eec)
2021-08-27 14:40:37 -04:00
Dimitri Savineau 8f0b0d1285 container: explicitly pull monitoring images
We don't pull the monitoring container images (alertmanager, prometheus,
node-exporter and grafana) in a dedicated task like we're doing for the
ceph container image.
This means that the container image pull is done during the start of the
systemd service.
By doing this, pulling the image behind a proxy isn't working with podman.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1995574

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 5bb7240f87)
2021-08-23 16:21:19 -04:00
Guillaume Abrioux c7cd688f2e iscsi: don't set default value for trusted_ip_list
It restricts access to the iSCSI API.
It can be left empty if the API isn't going to be access from outside the
gateway node

Even though this seems to be a limited use case, it's better to leave it
empty by default than having a meaningless default value.

We could make this variable mandatory but that would be a breaking
change. Let's just add a logic in the template in order to set this
variable in the configuration file only if it was specified by users.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1994930

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
Co-authored-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 6802b8dddd)
2021-08-19 12:06:58 -04:00
Guillaume Abrioux 20583e83dd containers: introduce target systemd unit
This adds ceph-*.target systemd unit files support for containerized
deployments.
This also fixes a regression introduced by PR #6719 (rgw and nfs systemd
units not getting purged)

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1962748

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 09ef465f62)
2021-08-18 13:43:01 -04:00
Guillaume Abrioux 6ebbda8cef roles: remove leftover from pr #4319
pr #4319 introduced some uesless `become: true` on systemd tasks.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 1db8fa8989)
2021-08-18 11:08:39 -04:00
Guillaume Abrioux 4f76dedd9e Vagrantfile: fallback on 'varant_variables.yml.sample'
When using a vagrant command from the root directory of the repo, it
throws an error if no 'vagrant_variables.yml' file is present.

```
Message: Errno::ENOENT: No such file or directory @ rb_sysopen - /home/guits/workspaces/ceph-ansible/vagrant_variables.yml
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3d27f9e7dc)
2021-08-18 11:08:13 -04:00
Guillaume Abrioux 2d38d8266b update: gather facts only one time
this play doesn't need to gather facts from localhost

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit c14e9114ba)
2021-08-17 15:47:38 -04:00
Dimitri Savineau ffe01c7ff5 ceph-mon: do not log monitor keyring
We don't want to display the keyring in the ansible log.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit e44075abd6)
2021-08-12 13:31:12 +02:00
Guillaume Abrioux c55c87d3c5 common: do not log keyring secret
let's not display any keyring secret by default in ansible log.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1980744

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 7511195738)
2021-08-11 17:01:22 -04:00
Benoît Knecht a8346af4f7 ceph-rgw: Work around Jinja2 < 2.8 missng eq test
EL7 ships with Jinja2 version 2.7, which is missing the `eq` test.

Work around this by using `match` instead.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
2021-08-11 13:53:44 +02:00
Benoît Knecht 66426e1316 ceph-rgw: Set pg_num on RGW pool if required
If the `pg_num` value specified in `rgw_create_pools` is different from the
actual value in the cluster, apply it with `ceph osd pool set`.

This corresponds to the behavior of the `ceph_pool` module used in Ceph Ansible
5.0 onward.

Also avoid setting the pool application if it's already done.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
2021-08-11 13:53:44 +02:00
Dimitri Savineau 712a9c4403 switch2container: fix mon quorum check
This was reverted by 7ddbe74

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1990733

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2021-08-10 10:03:02 +02:00
Dimitri Savineau 6897153ab7 ceph-dashboard: fix TLS cert openssl generation
With OpenSSL version prior 1.1.1 (like CentOS 7 with 1.0.2k), the -addext
doesn't exist.
As a solution, this uses the default openssl.cnf configuration file as a
template and add the subjectAltName in the v3_ca section. This temp openssl
configuration file is removed after the TLS certificate creation.
This patch also move the run_once statement at the block level.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1978869

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 5e0ace7e54)
2021-08-09 15:14:48 -04:00