Commit Graph

46 Commits (a60c74f61eb4b7523ef6d4065624312bfd0aa03a)

Author SHA1 Message Date
Sébastien Han 0205f6d645 rolling_update: nicer way to set osd flags
Prior to this patch, we were applying the osd flags like this:

"
General pre tasks
Set flags
Upgrade OSDs on a host
Unset flags <-- this triggers pending scrub to start
Set flags
Upgrade OSDs on a hosts
Unset flags <-- this triggers pending scrub to start
.
.
.
General post tasks
"

Now instead, we apply the flag once before starting the OSD update and
unset them once the last OSD is finished.

"
General pre tasks
Set flags and wait for any scrubs to finish
Upgrade OSDs on a host
Upgrade OSDs on a host
.
.
.
Unset flags
General post tasks
"

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1450754
Signed-off-by: Sébastien Han <seb@redhat.com>
Co-Authored-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-25 18:21:28 +02:00
Sébastien Han 4a4a20f07d rolling update: skip pg check if num_pgs = 0
In our test case we don't have any pgs, thus the check fails. The check
always returns an empty array, which makes the comparaison failing.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-24 08:50:49 +02:00
Guillaume Abrioux 7a333d05ce Add handlers for containerized deployment
Until now, there is no handlers for containerized deployments.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-02 17:12:20 +02:00
Guillaume Abrioux 5adbf0fdaa Move role dependencies in site.yml/site-docker.yml
This will give us more flexibility and avoid a lot of useless when
skipping all tasks from a non-desired role.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-02 17:12:14 +02:00
Guillaume Abrioux 206c7a16d0 rolling_update: refact code
Refact rolling_update playbook.
Add ceph-client upgrade.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-02 11:10:51 +02:00
Guillaume Abrioux 828f88403e Update: Avoid screen scraping in rolling update
since luminous has revamped the `ceph -s` output, we need to avoid screen
scraping.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-12 15:02:39 +02:00
Guillaume Abrioux 73141118d0 Make the new check PGs working with /bin/sh
The new test in the checks PGs are no longer working on distributions
where /bin/sh isn't linked to /bin/bash.

Fix: #1619
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-06-22 17:59:38 +02:00
Andrew Schoen e2104acb62 rolling_update: set health_mon_check_delay to 15
The old value of 10 did not give enough time for a containerized mon to
pass the health check.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-06-13 08:56:44 -05:00
Guillaume Abrioux 5af9bb432c rewrite check pgs clean tasks
Avoid screen scrapping by rewriting `waiting for clean pgs` tasks like it is
done in 304de48.

Use the json output returned by `ceph -s` instead

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-06-13 09:48:56 +02:00
Sébastien Han fdc7866072 Merge pull request #1469 from ceph/refact_code
Docker: Refact code
2017-06-02 12:40:25 +02:00
Guillaume Abrioux ddfe019342 Refact code
`ceph-docker-common`:
  At the moment there is a lot of duplicated tasks in each
  `./roles/ceph-<role>/tasks/docker/main.yml` that could be refactored in
  `./roles/ceph-docker-common/tasks/main.yml`.

`*_containerized_deployment` variables:
  All `*_containerized_deployment` have been refactored to a single
  variable `containerized_deployment`

duplicate `cephx` variables in `group_vars/* have been removed.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-05-24 15:55:41 +02:00
Sébastien Han 90389864d8 rolling-update: set/unset flags on the right container
Problem: we are delegating the set/unset flag to a monitor node but we
try to call an osd container

Solution: use the right container name.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-05-22 09:38:08 +02:00
Sébastien Han dfd8f4d96e test: add mgr section to the host inventory file
Without this, we don't test the mgr role so we need to add it.

Co-Authored-by: Guillaume Abrioux <gabrioux@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-04-15 00:16:10 +02:00
Sébastien Han c37aaa41f4 playbook: homogenize the way list osd ids
Problem: too many different commands to do the same thing. The 'cut'
command on infrastructure-playbooks/purge-cluster.yml was also wrong.
This sed command from osixia in ceph-docker
https://github.com/ceph/ceph-docker/pull/580/ addresses all the
scenarios.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-03-30 11:51:38 +02:00
Konstantin Shalygin 1662976fc0
Resolve issues when groups names not in default value. 2017-03-27 21:44:30 +07:00
Anthony D'Atri 6c4911276e Enhance clean PG check to catch active+clean+scrubbing and active+clean+scrubbing+deep
Signed-off-by: Anthony D'Atri <anthony.datri@gmail.com>
2017-03-19 00:23:26 -07:00
Sébastien Han 8320c14191 Merge pull request #1317 from ibotty/harmonize-docker-names
harmonize docker names
2017-03-14 18:20:20 +01:00
Andrew Schoen 46f26bec13 rolling-update: do not set group name vars at playbook level
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-03-08 08:57:08 -06:00
yangyimincn 8b36cbac64 Update rolling_update.yml
The task waiting for the monitor to join the quorum... , the result for ceph -s | grep monmap only contain monmap, not included quorum:

# ceph -s --cluster ceph | grep monmap
     monmap e1: 3 mons at {sh-office-ceph-1=10.12.10.34:6789/0,sh-office-ceph-2=10.12.10.35:6789/0,sh-office-ceph-3=10.12.10.36:6789/0}

If want to get monitor, should use this:

# ceph -s --cluster ceph | grep election
            election epoch 80, quorum 0,1 sh-office-ceph-1,sh-office-ceph-2

ceph verison: 10.2.5
2017-02-28 16:56:02 +08:00
Tobias Florek 931027e6f7 harmonize docker names
Created containers now are named more or less in the form of

    <ansible role>-<ansible_hostname>
2017-02-23 09:15:05 +01:00
Andrew Schoen 5622c94e8b rolling-update: do not use upstart to stop mons when using systemd
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-02-21 12:31:26 -06:00
Sébastien Han 8f94bfb498 rolling-update: detect init system properly
Simply use the ansible_service_mgr fact.

Closes: #1286

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-08 08:52:05 +01:00
Guillaume Abrioux 76ddcbc271 Remove support of releases prior to Jewel.
According to #1216, we need to simply the code by removing the
support of anything before Jewel.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-01-31 11:00:54 +01:00
Sébastien Han 829e2b6598 Merge pull request #1077 from font/rolling_update
Support containerized rolling update
2016-11-22 16:56:46 +01:00
Sébastien Han 38e846e542 rolling_update: clarify "serial" usage
Prior to this commit the serial variable was poorly documented. Now we
are making clear that this value should be left untouched as the rolling
update mechanism should happen serially.

Solves: bz-1396742
Signed-off-by: Sébastien Han <seb@redhat.com>
2016-11-21 14:42:46 +01:00
Ivan Font 255e816e28 Rolling update changes for containerized deployments
Separate out systemd restart tasks for containerized and
non-containerized deployments

Signed-off-by: Ivan Font <ifont@redhat.com>
2016-11-17 11:25:25 -08:00
Ivan Font e72f08080d Warn user when upgrading cluster with only one mon
Signed-off-by: Ivan Font <ifont@redhat.com>
2016-11-17 11:25:25 -08:00
Ivan Font 3ff17f1c8f Support containerized rolling update
- Update rolling update playbook to support containerized deployments
  for mons, osds, mdss, and rgws
- Skip checking if existing cluster is running when performing a rolling
  update
- Fixed bug where we were failing to start the mds container because it
  was missing the admin keyring. The admin keyring was missing because
  it was not being pushed from the mon host to the ansible host due to
  the keyring not being available before running the copy_configs.yml
  task include file. Now we forcefully wait for the admin keyring to be
  generated before continuing with the copy_configs.yml task include file
- Skip pre_requisite.yml when running on atomic host. This technically
  no longer requires specifying to skip tasks containing the with_pkg tag
- Add missing variables to all.docker.sample
- Misc. cleanup

Signed-off-by: Ivan Font <ifont@redhat.com>
2016-11-17 11:25:25 -08:00
Alfredo Deza 60ce2311b8 rolling_update: bump retries for osd_check/retries to 20 minutes
Signed-off-by: Alfredo Deza <adeza@redhat.com>

Resolves: rhbz#1395073
2016-11-17 10:43:58 -05:00
Sébastien Han 81a72cb85d Merge pull request #1068 from ceph/v2.2
moving to ansible v2.2 compatibility
2016-11-16 16:33:40 +01:00
Andrew Schoen 5f44b118b8 rolling update: stop RGWs before upgrade and start afterwards
Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Resolves: rhbz#1394929
2016-11-14 14:47:12 -06:00
Andrew Schoen ded9d9dfd3 rolling update: stop MDSs before upgrading and start afterwards
Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Resolves: rhbz#1394929
2016-11-14 14:47:12 -06:00
Andrew Schoen 5429c5f8c5 rolling update: stop MONs before upgrading and start afterwards
Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Resolves: rhbz#1394929
2016-11-14 14:47:12 -06:00
Andrew Schoen 66f09bdac4 rolling update: stop OSDs before upgrading
This avoids a bug where OSDs are sometimes restarted twice on
upgrades which leaves the OSD process running but not marked up.

See:

https://bugzilla.redhat.com/show_bug.cgi?id=1394928
https://bugzilla.redhat.com/show_bug.cgi?id=1391675
https://bugzilla.redhat.com/show_bug.cgi?id=1394929

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Resolves: rhbz#1394929
2016-11-14 14:46:58 -06:00
Sébastien Han 991341f525 rolling_update: add variable to upgrade ceph
My stupid self removed this crucial variable here: 217ce3ca thinking it
was another hard coded variable import where this is actually the
trigger for the upgrade.

Closes: #1071

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-11-04 17:31:02 +01:00
Sébastien Han a2fcd222d2 moving to ansible v2.2 compatibility
Signed-off-by: Sébastien Han <seb@redhat.com>
Co-Authored-By: Julien Francoz julien@francoz.net
2016-11-04 10:09:38 +01:00
Andrew Schoen 8262ce5e40 rolling update: fix restarts of radosgw
Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Resolves: rhbz#1391675
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2016-11-03 14:36:42 -05:00
Eduard Egorov 645b5efebf Fix hard-coded host group names in include tasks for group variables' file paths.
Signed-off-by: Eduard Egorov <eduard.egorov@icl-services.com>
2016-11-01 12:21:40 +00:00
Andrew Schoen 0897c965ff rolling_update: define mon_group_name when upgrading the mons
see: https://bugzilla.redhat.com/show_bug.cgi?id=1389456

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Resolves: rhbz#1389456
2016-10-27 14:17:56 -05:00
Sébastien Han b0989c700f rolling_update: fix wrong indent
Fixing: https://bugzilla.redhat.com/show_bug.cgi?id=1388295
Also add some notes in the README on how to run infrastructure
playbooks.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-10-26 12:51:08 -05:00
Andrew Schoen bebf412c92 infrastructure-playbooks: fix syntax errors in all playbooks
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2016-10-25 16:56:58 -05:00
Sébastien Han f49bf2832c rolling_update: improve variables import
we now have pointer to default role so we don't miss any of the
variables defined.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-10-06 14:08:04 +02:00
Sébastien Han ac2cb9ac2c upgrade: add custom timeout options
This commit introduces the ability to configure delays and retries for
cluster health checks, for both monitors and OSDs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-10-03 11:27:02 +02:00
Sébastien Han 21356c653f rolling updates: remove mon compact command
Users have reported this task to hang. Since this command is not
required to perform the upgrade, we remove it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-09-13 10:09:07 +02:00
Rachana Patel ad5805f03e rolling_update.yml will not work if cluster name is not 'ceph'. Adding --cluster will solve this problem
Fixes issue #969

Signed-off-by: Rachana Patel <rachana83.patel@gmail.com>
2016-09-07 15:38:58 -04:00
Sébastien Han fde819d1a8 create a directory for infrastructure playbooks
Since we have a couple of infrastructure related playbooks
(additionnally to the roles we are using to deploy Ceph), it makes sense
to have them located in a separate directory.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-08-17 11:53:34 +02:00