rolling_update: create rbd and rbd-mirror keyrings

During an upgrade ceph won't create keys that were not existing on the
previous version. So after the upgrade of let's Jewel to Luminous, once
all the monitors have the new version they should get or create the
keys. It's ok to have the task fails, especially for the rbd-mirror
key, which only appears in Nautilus.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1650572
Signed-off-by: Sébastien Han <seb@redhat.com>

infrastructure-playbooks/rolling_update.yml

@@ -197,6 +197,23 @@
       when:
         - containerized_deployment
 
+    - name: create potentially missing keys (rbd and rbd-mirror)
+      ceph_key:
+        name: "client.{{ item.0 }}"
+        state: present
+        dest: "/var/lib/ceph/{{ item.0 }}/"
+        caps:
+          mon: "allow profile {{ item.0 }}"
+        cluster: "{{ cluster }}"
+        containerized: "{{ 'docker exec ceph-mon-' + hostvars[groups[mon_host]]['ansible_hostname'] if containerized_deployment else None }}"
+      when:
+        - cephx
+      delegate_to: "{{ mon_host }}"
+      ignore_errors: True # this might fail for upgrade from J to L on rbd-mirror and also on partially updated clusters
+      with_nested:
+        - ['bootstrap-rbd', 'bootstrap-rbd-mirror']
+        - "{{ groups[mon_group_name] }}" # so the key goes on all the nodes
+
     - name: set osd flags
       command: ceph --cluster {{ cluster }} osd set {{ item }}
       with_items: