Commit Graph

4399 Commits (4e3301d3614bb9e9ee40d531fcd4421ccf5ea18b)
 

Author SHA1 Message Date
Dimitri Savineau 4e3301d361 ceph-osd: exit gracefully when no data partition
When using collocated or non-collocated osd_scenarios (ceph-disk) and
trying to deterime the OSD_DEVICE from the OSD_ID passed to the systemd
unit then we can be in a situation where the OSD hasn't been activated
but the OSD ID exists.
This means the data partition isn't in activate state and the ceph-disk
list command won't show the OSD ID on the data partition.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1850377

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-07-07 18:18:14 +02:00
Guillaume Abrioux 90f3f61548 infra: introduce docker to podman playbook
This isn't backported from master because there are too many changes
between stable-3.2 and other newer branches.

NOTE:
This playbook  *doesn't* add podman support in stable-3.2 at all.
This is a tripleO dedicated playbook which is intended to be run
early during FFU workflow in order to prepare the OS upgrade.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1853457

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-07-07 12:11:09 -04:00
Guillaume Abrioux 6daa2c9d22 doc: add a note about deprecated branches
This commit adds a note about `stable-3.0` `stable-3.1` branches which
are deprecated and not maintained anymore.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit bbe30bcc69)
2020-07-03 14:45:05 +02:00
Guillaume Abrioux 15f03e66b2 doc: add a note about containerized deployments
This commit updates the documentation to add a note about containerized
deployments.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit e61488507b)
2020-07-03 14:45:05 +02:00
Guillaume Abrioux 2b6528561d doc: fix warning treated as an error
Typical error:

```
Warning, treated as error:
/home/jenkins-build/build/workspace/ceph-ansible-docs-pull-requests/docs/source/day-2/upgrade.rst:2:Title underline too short.
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5c254861bd)
2020-07-03 09:45:39 +02:00
Guillaume Abrioux 8b8fa74db7 switch_to_containers: don't set noup flag
We shouldn't set this flag when running switch_to_containers playbook.
Otherwise the playbook fails waiting for pgs to be clean.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1843569

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b91d60d384)
2020-06-29 15:25:01 +02:00
Guillaume Abrioux b2e1dcc0f4 switch-to-containers: set and unset osd flags
The workflow in this playbook should be the same than in rolling_update,
we should first set noout and nodeep-scrub flags before migrating the
first osd and unset osd flags after the last osd is migrated.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 2cfaa056e0)
2020-06-29 15:25:01 +02:00
Guillaume Abrioux 1cf3a57a6c Revert "switch-to-containers: set and unset osd flags"
This reverts commit 5a4134098a.

We need to provide a tag for RHCS 3.3z6 without this commit.
2020-06-25 17:08:10 +02:00
Guillaume Abrioux 693e534ee9 Revert "switch_to_containers: don't set noup flag"
This reverts commit b7ec4a995b.

We need to provide a tag for RHCS 3.3z6 without this commit.
2020-06-25 17:07:25 +02:00
Dimitri Savineau a2556f084d docker: Add Requires on docker service
When using docker container engine then the systemd unit scripts only
use a dependency on the docker daemon via the After parameter.
But if docker is restarted on a live system then the ceph systemd units
should wait for the docker daemon to be fully restarted.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1846830

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit bd22f1d1ec)
2020-06-22 21:08:13 -04:00
Dimitri Savineau 9a9ef7bc97 docs: Add upgrade operation.
This commit adds a chapter about the ceph upgrade process.

Closes: #5393

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit e41487dbce)
2020-06-18 18:02:09 +02:00
Guillaume Abrioux b7ec4a995b switch_to_containers: don't set noup flag
We shouldn't set this flag when running switch_to_containers playbook.
Otherwise the playbook fails waiting for pgs to be clean.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1843569

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit b91d60d384)
2020-06-18 09:56:28 +02:00
Guillaume Abrioux 5a4134098a switch-to-containers: set and unset osd flags
The workflow in this playbook should be the same than in rolling_update,
we should first set noout and nodeep-scrub flags before migrating the
first osd and unset osd flags after the last osd is migrated.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 2cfaa056e0)
2020-06-18 09:56:28 +02:00
Guillaume Abrioux 6f3d696742 clients: move dummy container creation
This commit moves the dummy container creation task right before the
cephx keys creation task so it can't be run out of time.

Also, this commit makes the dummy container running for ever.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1828105

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-04-27 13:31:52 -04:00
ianwatsonrh 2666c54b3a typo: updating type check on rc
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1827271
Signed-off-by: ianwatsonrh <ianwatson@redhat.com>
(cherry picked from commit ccf6a7f153)
2020-04-23 11:36:59 -04:00
Guillaume Abrioux 8c9be9c179 doc: add day-2 operations documentation
This commit is the first of a serie in order to describe all day-2 operations
that are possible via ceph-ansible using a set of playbook provided in
`infrastructure-playbooks` directory.

Fixes: #5061

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 7e800303e9)
2020-04-23 13:29:32 +02:00
Rishabh Dave 34cf0e5301 library/ceph_volume: look for error messages in stderr
Error message were moved to from stdout in stderr here -
b8d6dcbe9f (diff-20f7c578a4e69ec61a5869d706567a24R137).

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1793542
Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 4249d1e02d)
2020-04-20 13:36:57 -04:00
Dimitri Savineau 65b0e9bb5d ceph-validate: update RHEL requirement for RHCS
We were not testing the right ansible_distribution fact value for RHEL
distribution.
This commit also updates the minial RHEL version supported by RHCS.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 5de74fe512)
2020-04-14 11:27:21 -04:00
Guillaume Abrioux a51331beb9 add-osd: refact the playbook
There's no need to have two plays anymore since we now set/unset osd
flags in `ceph-osd` role.

Also, this commit makes the role `ceph-facts` to be called after
`ceph-defaults`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-04-07 11:19:53 -04:00
Guillaume Abrioux 724620ed3d add-osd: fix fact gathering in add-osd
This commit makes this playbook gathering facts from all other nodes but
clients.
When collocating OSDs on other nodes it can fail like following:

```
fatal: [vm252-11]: FAILED! => {
    "msg": "'ansible.vars.hostvars.HostVarsVars object' has no attribute 'ansible_hostname'"
}
```

In that case, a fact from a RGW node is called when rendering the
`ceph.conf.j2` but it fails because facts are gathered only from mon and
osd nodes.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1806765

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-04-07 11:19:53 -04:00
Guillaume Abrioux 8ccf91c1f0 add-osd: unset noup flag after last osd is deployed
this commit fixes a bug when using `add-osd.yml` playbook.
`noup` flag is set early but it never got unset before the "wait for pgs
clean" check, so the playbook always fails because OSDs aren't never
seen UP.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1816023

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-04-07 11:19:53 -04:00
Guillaume Abrioux a8f5e43624 ceph_key: fetch key when needed
Fetch the key when it is present in the cluster but not on the node.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit ccfa249919)
2020-04-03 16:19:03 -04:00
Guillaume Abrioux 323d4f8f0b ceph_key: fix idempotency when no secret is passed
553584cbd0 introduced a regression when no
secret is passed, it overwrites the secret each time the task is run.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 003defec03)
2020-04-03 16:19:03 -04:00
Guillaume Abrioux b107dcf80b ceph_key: remove 'update' state
With this change, the state `present` is enough to update a keyring.
If the keyring already exist, it will be updated if caps or secret
passed to the module are different.
If the keyring doen't exist, it will be created.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1808367

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 553584cbd0)
2020-04-03 16:19:03 -04:00
Dimitri Savineau edfeb98593 tests: add mgr nodes to shrink_mon inventory
Since 306ce82 we explicitly fail when there's no mgr node preent in the
inventory.

fatal: [mon0]: FAILED! => {
    "changed": false
}

MSG:

Please add a mgr host to your inventory.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-04-02 22:02:35 +02:00
Guillaume Abrioux d4ffe21225 osd: support changing default rule even when osd_crush_location isn't defined
Creating crush rules even with no crush hierarchy configuration is a
valid scenario so we shouldn't be bound to the first task result (which
configure crush hierarchy) to be able to add new crush rules.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1816989

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 5b0476385c)
2020-03-31 23:04:03 +02:00
Dimitri Savineau 586c6e8afe Add site-container.yml symlink
This adds a symlink to the site-docker.yml.sample playbook.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-03-31 23:00:49 +02:00
Guillaume Abrioux 3b1794a0fd switch_to_containers: exclude clients nodes from facts gathering
just like site.yml and rolling_update, let's exclude clients node from
the fact gathering.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 332c39376b)
(cherry picked from commit 5c3ba0787c)
2020-03-30 11:10:29 -04:00
Guillaume Abrioux cfe77bc51f main: exclude client nodes from facts gathering when delegate_facts_host
This commit excludes client nodes from facts gathering, they are not
needed and can speed up this task.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 865d2eac9b)
2020-03-30 11:10:29 -04:00
John Fulton 658d9cadfd The _filtered_clients list should intersect with ansible_play_batch
Client configuration with --limit fails without this patch
because certain tasks are only done to the first host in the
_filtered_clients list and it's likely that first host will
not be included in what's sepcified with --limit. To fix this
the _filtered_clients list should be built from all clients
in the inventory that are also in the running play.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1798781

Signed-off-by: John Fulton <fulton@redhat.com>
(cherry picked from commit e4bf4857f5)
2020-03-30 11:10:29 -04:00
Guillaume Abrioux 6006985466 defaults: remove legacy comment
This is no longer true, let's remove this comment given that this option
is not ignored in containerized deployments.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit e551b5ba1a)
2020-03-26 12:08:31 -04:00
Guillaume Abrioux c60967f045 docker-common: remove legacy tasks for ntp configuration
Those tasks aren't needed in docker-common since the introduction of
`ceph-infra` role. They are duplicated tasks.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1810376

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit cd0195c562)
2020-03-25 13:53:25 -04:00
Guillaume Abrioux a0f01db800 tests: add inventory host for 4.0 upgrade job
This inventory is intended to be used in the upgrade scenario in
stable-4.0 branch.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-03-04 23:18:43 +01:00
Guillaume Abrioux d2d241f21d
tests: modify add-osd job
This commit modifies the way we test add-osd scenario given that the
playbook add-osd.yml is broken at the moment.

As a workaround we can use main playbook with `--limit` to achieve this
operation.

Note: This commit is intended to be reverted once we get a fix.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-03-03 11:08:22 +01:00
Dimitri Savineau 2d2cec99fc tests: pg num should be a power of two number
This patch changes the pg_num value of the rgw pools foo and bar to be
a power of two number.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-02-17 14:52:09 -05:00
Benoît Knecht 87034b1fb6 ceph-rgw: Fix customize pool size "when" condition
In 3c31b19ab3, I fixed the `customize pool
size` task by replacing `item.size` with `item.value.size`. However, I
missed the same issue in the `when` condition.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 3842aa1a30)
2020-02-17 11:53:58 -05:00
Benoît Knecht 874c94c59e ceph-rgw: Fix custom pool size setting
RadosGW pools can be created by setting

```yaml
rgw_create_pools:
  .rgw.root:
    pg_num: 512
    size: 2
```

for instance. However, doing so would create pools of size
`osd_pool_default_size` regardless of the `size` value. This was due to
the fact that the Ansible task used

```
{{ item.size | default(osd_pool_default_size) }}
```

as the pool size value, but `item.size` is always undefined; the
correct variable is `item.value.size`.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 3c31b19ab3)
2020-02-17 11:53:58 -05:00
Dimitri Savineau db8902d444 ceph-{mon,osd}: move default crush variables
Since ed36a11 we move the crush rules creation code from the ceph-mon to
the ceph-osd role.
To keep the backward compatibility we kept the possibility to set the
crush variables on the mons side but we didn't move the default values.
As a result, when using crush_rule_config set to true and wanted to use
the default values for crush_rules then the crush rule ansible task
creation will fail.

"msg": "'ansible.vars.hostvars.HostVarsVars object' has no attribute
'crush_rules'"

This patch move the default crush variables from ceph-mon to ceph-osd
role but also use those default values when nothing is defined on the
mons side.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1798864

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 1fc6b33714)
2020-02-17 16:23:33 +01:00
Dimitri Savineau 306ce82358 ceph-validate: fail if no mgr host is present
We already stop the upgrade playbook (rolling_update.yml) if there's
no mgr node present so we should also do the same for initial
deployment.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1788644

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-02-11 13:27:10 -05:00
Dimitri Savineau 553fb1ed1e ceph-mon: use interactive session with aliases
When using ceph aliases with commands that require manual intervention
to stop then the command will keep running inside the container (like
using Ctrl+c).
For handling this, we should use the interactive session option (-it)
with the docker commands.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1797874

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-02-05 15:29:51 +01:00
Mike Christie c2a9397474 iscsi: Fix crashes during rolling update
During a rolling update we will run the ceph iscsigw tasks that start
the daemons then run the configure_iscsi.yml tasks which can create
iscsi objects like targets, disks, clients, etc. The problem is that
once the daemons are started they will accept confifguration requests,
or may want to update the system themself. Those operations can then
conflict with the configure_iscsi.yml tasks that setup objects and we
can end up in crashes due to the kernel being in a unsupported state.

This could also happen during creation, but is less likely due to no
objects being setup yet, so there are no watchers or users accessing the
gws yet. The fix in this patch works for both update and initial setup.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1795806

Signed-off-by: Mike Christie <mchristi@redhat.com>
(cherry picked from commit 77f3b5d51b)
2020-02-03 15:15:53 +01:00
Guillaume Abrioux b7a21d94d3 tests: retry to fire up VMs on vagrant failure
Add a script to retry several times to fire up VMs to avoid vagrant
failures.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
Co-authored-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 1ecb3a9352)
2020-02-03 10:20:19 +01:00
Guillaume Abrioux d437593e85 config: fix external client scenario
When no monitor group is present in the inventory, this task fails.
This affects only non-containerized deployments.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit e7bc079405)
2020-02-03 10:20:19 +01:00
Guillaume Abrioux 523a93b0e1 tests: add external_clients scenario
This commit adds a new 'external ceph clients' scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 641729357e)
2020-02-03 10:20:19 +01:00
Guillaume Abrioux b6744fd82a validate: allow running ceph-ansible 3.2 against ansible 2.7
This commit allows ceph-ansible 3.2 to be run against ansible 2.7

However, note that running stable-3.2 against ansible 2.7 doesn't get
any testing upstream this might break the playbook, only ansible 2.6 is
officially supported.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1781635

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-01-31 10:07:48 -05:00
Guillaume Abrioux ce7503a3a6 tests: add 'all_in_one' scenario
Add new scenario 'all_in_one' in order to catch more collocated related
issues.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 3e7dbb4b16)
2020-01-31 11:26:40 +01:00
Guillaume Abrioux cf748e729f update: remove legacy tasks
These tasks should have been removed with backport #4756

Note:
This should have been backported from master but it's not possible
because of too many change between master and stable-3.2

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1740463

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2020-01-29 09:25:15 -05:00
Dimitri Savineau 13e0f7d341 ceph-defaults: remove rgw from ceph_conf_overrides
The [rgw] section in the ceph.conf file or via the ceph_conf_overrides
variable doesn't exist and has no effect.
To apply overrides to all radosgw instances we should use either the
[global] or [client] sections.
Overrides per radosgw instance should still use the
[client.rgw.{instance-name}] section.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1794552

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
(cherry picked from commit 2f07b85131)
2020-01-29 14:19:17 +01:00
Guillaume Abrioux 726b3f220b defaults: change monitor|radosgw_address default values
To avoid confusion, let's change the default value from `0.0.0.0` to
`x.x.x.x`.
Users might think setting `0.0.0.0` will make the daemon binding on all
interfaces.

Fixes: #4827

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit fc02fc98eb)
2020-01-14 17:22:35 +01:00
Dimitri Savineau 071b950325 tox: allow copy admin key for purge scenario
This is enabled in the group_vars/clients file but it's overrided in
extra vars by tox.
Let's do it like that for now.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2020-01-13 14:50:29 -05:00