Commit Graph

5327 Commits (d19e6033b227c621b6c794db4f571151e5bbf9c4)
 

Author SHA1 Message Date
Kevin Coakley d19e6033b2 Remove ceph-radosgw.target when switching to containerize daemons
The task "remove old systemd unit file" under "switching from
non-containerized to containerized ceph rgw" only removes
the ceph-radosgw@.service file. The task should also remove
the ceph-radosgw.target file, like the "remove old systemd unit
files" tasks for the mons, mgrs, osds, etc, in order to clean up
all of the unused systemd unit files.

Signed-off-by: Kevin Coakley <kcoakley@sdsc.edu>
2020-08-04 11:08:12 -04:00
Guillaume Abrioux 5df6225ede tests: change subnet in lvm_osds container scenario
This commit changes the subnets in container-lvm_osds scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-04 14:00:05 +02:00
Guillaume Abrioux 0a38d91b5b Revert "tests: add more coverage for test_ceph_key"
This reverts commit 1e46264bc1.
2020-08-04 11:28:42 +02:00
Guillaume Abrioux b15063b20e Revert "ceph_key: refact the code and minor fixes"
This reverts commit 9a950b8f0f.
2020-08-04 11:28:42 +02:00
Guillaume Abrioux 9a950b8f0f ceph_key: refact the code and minor fixes
wip

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-03 18:12:45 +02:00
Guillaume Abrioux 1e46264bc1 tests: add more coverage for test_ceph_key
This commit adds more coverage regarding the testing of ceph_key module

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-03 18:12:45 +02:00
Guillaume Abrioux 0a581a6e60 config: only add related rgw section
there's no need to add each rgw section on all rgw nodes.
With this commit, only related rgw section are rendered.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-03 14:47:27 +02:00
Guillaume Abrioux 8933bfde33 shrink_osd: remove osd data directory
Otherwise it leaves an empty directory.
When shrinking and redeploying multiple OSDs you have no guarantee it
will reuse the same osd id.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-03 14:46:56 +02:00
Guillaume Abrioux 78e4faf077 tox: split shrink_osd scenario
Let's split this scenario with a dedicated tox ini file.

This is for testing in two ways:

1/ shrinking OSDs one by one
2/ shrinking multiple OSDs with a single call of the playbook

ceph-build related PR: ceph/ceph-build#1629

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-03 14:46:56 +02:00
Guillaume Abrioux 7efea219d6 tests: refact shrink_osd scenario
This adds more coverage on the shrink_osd scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-08-03 14:46:56 +02:00
Dimitri Savineau 0d0f1e71df dashboard: allow remote TLS cert/key copy
When using TLS on the ceph dashboard or grafana services, we can provide
the TLS certificate and key.
Those files should be present on the ansible controller and they will be
copyied to the right node(s).
In some situation, the TLS certificate and key could be already present
on the target node and not on the ansible controller.
For this scenario, we just need to copy the files locally (on each remote
host).

This patch adds the dashboard_tls_external variable (with default to
false) to allow users to achieve this scenario when configuring this
variable to true.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1860815

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-08-03 13:39:47 +02:00
Dimitri Savineau ec0a37a74f rolling_update: restart mds after the upgrade
In addition of 155e2a2, the active mds daemons isn't stop/start
correctly as opposed as the other services so that daemon doesn't come
back after the upgrade.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1861688

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-29 16:45:41 -04:00
Dimitri Savineau 891234668e tests: install pyyaml on osd nodes
Due to [1], ceph-volume has now a dependency on pyyaml but it's not
installed by default via the package dependency.
This patch only add the required package on non containerized
deployment and as temporary workaround for the CI.

[1] https://tracker.ceph.com/issues/46759

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-29 12:49:15 -04:00
Dimitri Savineau a6209bd957 rolling_update: refact dashboard workflow
The dashboard upgrade workflow should do the same process than the ceph
upgrade otherwise any systemd unit modification won't be apply on the
monitoring/dashboard stack.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1859173

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-25 09:35:17 +02:00
Dimitri Savineau 155e2a23d5 rolling_update: stop/start instead of restart
During the daemon upgrade we're
  - stopping the service when it's not containerized
  - running the daemon role
  - start the service when it's not containerized
  - restart the service when it's containerized

This implementation has multiple issue.

1/ We don't use the same service workflow when using containers
or baremetal.

2/ The explicity daemon start isn't required since we'are already
doing this in the daemon role.

3/ Any non backward changes in the systemd unit template (for
containerized deployment) won't work due to the restart usage.

This patch refacts the rolling_update playbook by using the same service
stop task for both containerized and baremetal deployment at the start
of the upgrade play.
It removes the explicit service start task because it's already included
in the dedicated role.
The service restart tasks for containerized deployment are also
removed.

Finally, this adds the missing service stop task for ceph crash upgrade
workflow.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1859173

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-25 09:35:17 +02:00
Dimitri Savineau 4e84b4beed ceph-facts: remove mds_name fact
The mds_name fact always gets the ansible_hostname value so we don't
need to have a dedicated fact for this and use the ansible_hostname fact
instead.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-23 17:02:43 +02:00
Dimitri Savineau cbe79428e6 ceph-handler: remove iscsigws restart scripts
The iscsigws restart scripts for tcmu-runner and rbd-target-{api,gw}
services only call the systemctl restart command.
We don't really need to copy a shell script to do it when we can use
the ansible service module instead.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-23 17:02:12 +02:00
Dimitri Savineau 47b7c00287 podman: always remove container on start
In case of failure, the systemd ExecStop isn't executed so the container
isn't removed. After a reboot of a failed node, the container doesn't
start because the old container is still present in created state.
We should always try to remove the container in ExecStartPre for this
situation.
A normal reboot doesn't trigger this issue and this also doesn't affect
nodes running containers via docker.
This behaviour was introduced by d43769d.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1858865

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-23 17:00:38 +02:00
Guillaume Abrioux 44caa062b7 tox: remove ubuntu references
since we've dropped ubuntu testing on PRs and nightlies, we don't need
these references anymore in tox files.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-22 18:57:31 -04:00
Guillaume Abrioux 8ef9fb68bc tests: lvm_setup.yml, add carriage return
This commit adds crlf between each task.
It makes the playbook more readable.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-22 14:25:45 +02:00
Guillaume Abrioux 218f4ae361 tests: (lvm_setup.yml), don't shrink lvol
when rerunning lvm_setup.yml on existing cluster with OSDs already
deployed, it fails like following:

```
fatal: [osd0]: FAILED! => changed=false
  msg: Sorry, no shrinking of data-lv2 to 0 permitted.
```

because we are asking `lvol` module to create a volume on an empty VG
with size extents = `100%FREE`.

The default behavior of `lvol` is to shrink the volume if the LV's current
size is greater than the requested size.

Given the requested size is calculated like this:

`size_requested = size_percent * this_vg['free'] / 100`

in our case, it is similar to:

`size_requested = 100 * 0 / 100` which basically means `0`

So the current LV size is well greater than the requested size which
leads the module to attempt to shrink it to 0 which isn't obviously now
allowed.

Adding `shrink: false` to the module calls fixes this issue.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-22 14:25:45 +02:00
Dimitri Savineau 18e3c7a0a2 ceph-handler: add missing condition on ceph-crash
The ceph-crash tasks present in the ceph-handler role don't need to be
executed on all nodes.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-21 23:26:11 +02:00
Guillaume Abrioux 39bb279a53 crash: rm container in ExecPreStart even with docker
We should ensure the container is removed in `ExecPreStart` even when
`{{ container_binary }}` is docker.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-21 23:23:18 +02:00
Guillaume Abrioux 9d2f2108e1 ceph-crash: introduce new role ceph-crash
This commit introduces a new role `ceph-crash` in order to deploy
everything needed for the ceph-crash daemon.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-21 20:22:12 +02:00
Guillaume Abrioux d490968fc8 defaults: remove legacy
These variables aren't consummed anywhere else than in ceph-nfs role so
there is no need to have them in `ceph-defaults`'s defaults

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-21 09:39:15 +02:00
Dimitri Savineau 5ef965c4dc cephadm: set the command as a fact
Set the cephadm cmd as a fact instead of rewriting the same command
over and over.
This also fix an issue when using docker as container engine because
the --docker cephadm parameter should be use before the subcommand
not after.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-20 16:32:20 -04:00
Guillaume Abrioux f8a951f50c facts: fix broken facts when using --limit
This commit fixes these tasks when --limit is used.

It makes sure the fact is set on right nodes even when the playbook is
run with `--limit`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-20 10:56:10 -04:00
Dimitri Savineau 2b8ebf1457 ceph-dashboard: copy TLS cert/key on monitor
The ceph-dashboard role is executed on the mgr nodes so the TLS cert/key
files are copied to those nodes.
But we are running importing the cert/key files into the ceph
configuration on the monitor.

Closes: #5557

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-20 16:16:35 +02:00
Dimitri Savineau 957903d561 cephadm: add playbook
This adds a new playbook for deploying ceph via cephadm.

This also adds a new dedicated tox file for CI purpose.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-16 11:40:45 -04:00
Dimitri Savineau 9596494911 cephadm-adopt: delegate task for orch apply
This is a partial revert of b38019e because we don't want to execute
the whole play on the monitor otherwise if we have some empty group
like rgws or mdss then the orchestrator commands will still be
executed.
Instead we should keep the real target group name at play level and
delegate the orchestator commands to the monitor. The whole play
will be skipped is the group is empty.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-16 09:44:33 -04:00
Dimitri Savineau 75ae1b7e90 cephadm-adopt: inform users about cephadm
Print a message at the end of the playbook to inform users that they
don't have to user ceph-ansible playbooks anymore as everything else
need to be done via cephadm (day 2 operation).

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-15 17:04:59 -04:00
Dimitri Savineau 7164426456 cephadm-adopt: refresh the service/daemon list
When reporting the orchestrator service/daemon list at the end of the
playbook, we can use the --refresh option otherwise we could have
an outdated output.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-15 17:04:59 -04:00
Dimitri Savineau ceac81cd24 Revert "cephadm-adopt: remove the cephadm script"
This reverts commit c3bbc6b13c.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-15 17:04:59 -04:00
Guillaume Abrioux 9417ecf0c5 ceph_key: fix bug in 'info' feature
Fix 'info' feature from ceph_key.py module

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-15 17:06:17 +02:00
Dimitri Savineau 0c3a2b72ff cephadm-adopt: wait for monitor in quorum
After adopting a monitor we need to wait that monitor to join back
the quorum before moving to the next node.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-13 09:16:11 -04:00
Dimitri Savineau d3b3c8948e cephadm-adopt: add osd flags during adoption
Like rolling_update or switch2container playbooks, we need to set/unset
some osd flags before and after the OSD daemons adoption.
This also adds a task for waiting for clean pgs at then of an OSd node.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-13 09:16:11 -04:00
Dimitri Savineau 9fe2694711 cephadm-adopt: add iscsi support
The iSCSI support has been added recently in cephadm.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-13 09:16:11 -04:00
Dimitri Savineau c3bbc6b13c cephadm-adopt: remove the cephadm script
At the end of the process when don't need the cephadm script.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-13 09:16:11 -04:00
Dimitri Savineau 381201a394 cephadm-adopt: show orchestrator status
At the end of the playbook we can show the orchestrator status like
we do with the ceph status in initial deployment.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-13 09:16:11 -04:00
Dimitri Savineau 91a6c79e41 cephadm-adopt: use placement parameter
It's better to use the --placement parameter when using ceph orch apply
commands to avoid confusion in the parameters.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-10 14:05:15 -04:00
Dimitri Savineau f2d997396e cephadm-adopt: use custom dashboard images
cephadm uses default value for dashboard container images which need to
be customized by ansible for upstream or downstream purpose.
This feature wasn't present when cephadm-adopt.yml has been designed.
Also set the container_image_base variable for upgrade purpose.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-10 16:00:24 +02:00
Dimitri Savineau b38019e3ca cephadm-adopt: run orch apply from monitors
It looks like we can't run the ceph orch apply commands on nodes other
than monitors even if it used to work in the past.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-10 16:00:24 +02:00
Dimitri Savineau 27efcbc0e5 cephadm-adopt: don't fail on systemd reset-failed
If the systemd service exists successfully then we don't need to reset
the failed state.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-10 16:00:24 +02:00
Dimitri Savineau fd36433826 cephadm-adopt: copy client.admin keyring
The ceph config assimilate-conf command requires the client.admin
keyring which isn't present on all nodes most of the time.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-10 16:00:24 +02:00
Dimitri Savineau 14eed63921 tox: add cephadm_adopt scenario
This adds an optional cephadm_adopt scenario which is based on
all_daemons.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-10 16:00:24 +02:00
Guillaume Abrioux 0f3bae09bd play: followup on cc0d969
Remove two other pattern 'iscsigws' in main playbook that have been
missed in cc0d9697c5

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-09 10:01:05 -04:00
Guillaume Abrioux 86edae724f rgw: set container memory limit to 4g
This commit changes the container memory limit for rgw daemons.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1707488

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-09 15:31:10 +02:00
Guillaume Abrioux bcc673f66c facts: refact `ceph_uid` fact
There's no need to set this fact with a `set_fact`
We can achieve this in `ceph-defaults`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-09 13:37:29 +02:00
Guillaume Abrioux f402ab2b87 ceph_volume: fix regression
do not skip zapping if osd_fsid is passed

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-08 09:52:53 -04:00
Guillaume Abrioux 40307f810c tests: add docker hub authentication in jobs
This commit makes all jobs authenticating to docker hub in order to
avoid the rate limit.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-08 09:52:53 -04:00