rolling_update: fail early if cluster state is not OK

starting an upgrade if the cluster isn't HEALTH_OK isn't a good idea.
Let's check for the cluster status before trying to upgrade.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3a100cfa52)

infrastructure-playbooks/rolling_update.yml

@@ -104,6 +104,23 @@
     - import_role:
         name: ceph-facts
 
+    - block:
+        - name: get ceph cluster status
+          command: "{{ container_exec_cmd }} ceph --cluster {{ cluster }} -s -f json"
+          register: check_cluster_status
+          delegate_to: "{{ mon_host }}"
+
+        - block:
+            - name: display ceph health detail
+              command: "{{ container_exec_cmd }} ceph --cluster {{ cluster }} health detail"
+              delegate_to: "{{ mon_host }}"
+
+            - name: fail if cluster isn't in an acceptable state
+              fail:
+                msg: "cluster is not in an acceptable state!"
+          when: not (check_cluster_status.stdout | from_json).health.status == 'HEALTH_OK'
+      when: inventory_hostname == groups[mon_group_name] | first
+
     - name: ensure /var/lib/ceph/bootstrap-rbd-mirror is present
       file:
         path: /var/lib/ceph/bootstrap-rbd-mirror