Commit Graph

291 Commits (0930f149154151e19dfc85917b1d8acf19234d33)

Author SHA1 Message Date
Sébastien Han 0930f14915 osd: do not use dm when osd_auto_discovery
The current code will also return lvm devices such as /dev/dm-2, this
kind of device type is not supported by ceph-disk at the moment. Now we
just ignore them.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-11-08 11:33:10 +11:00
Sébastien Han d4ed9a2064 osd: enhance backward compatibility
During the initial implementation of this 'old' thing we were falling
into this issue without noticing
https://github.com/moby/moby/issues/30341 and where blindly using --rm,
now this is fixed the prepare container disappears and thus activation
fail.
I'm fixing this for old jewel images.

Also this fixes the machine reboot case where the docker logs are
purgend. In the old scenario, we now store the log locally in the same
directory as the ceph-osd-run.sh script.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-11-03 11:15:23 +01:00
Sébastien Han faccd0acf0 Merge pull request #2100 from ceph/lvm-bluestore
ceph-volume lvm bluestore support
2017-10-27 17:36:16 +02:00
Alfredo Deza 517a2b3feb ceph-osd skip lvm creation if they are already in use
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-27 11:33:54 -04:00
Sébastien Han 5a10b048b0 Merge pull request #2105 from major/really-fix-always-run
Really fix always run
2017-10-27 09:33:47 +02:00
Sébastien Han 07e2a783f8 Merge pull request #2084 from ceph/backward-osd-2.4
osd: bring backward compatibility with old Jewel images
2017-10-25 17:33:49 +02:00
Major Hayden f73232caa4
Use check_mode instead of always_run
This patch changes the `always_run: yes` task option to
`check_mode: no` to avoid Ansible warnings.
2017-10-25 09:53:34 -05:00
Major Hayden c2b5118c1b
Revert "Avoid deprecated always_run"
This reverts commit 620fb37dd4.
2017-10-25 09:48:09 -05:00
Alfredo Deza d3b427e169 ceph-osd lvm scnearios are no longer limited to filestore
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-25 08:23:45 -04:00
Alfredo Deza df05e63c10 ceph-osd use --cluster in ceph-volume calls
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-25 08:23:45 -04:00
Alfredo Deza 628d98a92c ceph-osd add the CEPH_VOLUME_DEBUG env var to all ceph-volume commands
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-25 06:50:22 -04:00
Alfredo Deza bbc3672253 ceph-osd: lvm support for bluestore
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-10-25 06:46:39 -04:00
John Fulton 7a7ddab6c2 Require osd_scenario parameter to be provided in containerized deploy
Fixes: #2095
2017-10-23 15:16:03 +00:00
Sébastien Han 968ef04324 osd: bring backward compatibility with old Jewel images
There was a huge resync from luminous to jewel in ceph-docker:
https://github.com/ceph/ceph-docker/pull/797

This change brought a new handy function to discover partitions tight to
an OSD. This function doesn't exist in the old image so the
ceph-osd-run.sh script breaks when trying to deploy Jewel OSD with that
old Jewel image version.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-20 16:26:41 +02:00
Sébastien Han a53aa9e8b4 ci: new osd scenarios
This commit add new osd scenarios, it aims to simplify the CI setup and
brings a better coverage on the OSD scenarios.
We decided to differentiate between filestore and bluestore, thinking
ahead when filestore won't be supported anymore.
So we now have two classes of tests:

* Filestore
* Bluestore

In each of those classes we have container and non-container.
Then for each we test the following:

* collocated
* collocated dmcrypt
* non-collocated
* non-collocated dmcrypt
* auto discovery collocated
* auto discovery collocated dmcrypt

This gives us a nice coverage and also reduces the footprint on the CI.
We are now up to 4 scenarios, each containing 6 OSD VMs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-18 09:26:06 +02:00
Major Hayden c01851325e
Remove jinja2 delimiters from `when` keys
This patch changes the `when:` keys so that they have no jinja2
delimiters. This avoids Ansible warnings which could turn into
errors in a future Ansible release.
2017-10-12 11:27:42 -05:00
Major Hayden 620fb37dd4
Avoid deprecated always_run
The `always_run` key is deprecated and being removed in Ansible 2.4.
Using it causes a warning to be displayed:

    [DEPRECATION WARNING]: always_run is deprecated.

This patch changes all instances of `always_run` to use the `always`
tag, which causes the task to run each time the playbook runs.
2017-10-12 08:29:44 -05:00
Sébastien Han c693e95cbf purge-docker: rework device detection
we don't need "devices" and other device variable anymore, the playbook
detects that for us.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-07 03:39:04 +02:00
Guillaume Abrioux 6b027557e6 osd: fix `set_fact build dedicated_devices`
Use an intermediate variable to build the final `dedicated_devices` list
to avoid duplicate entry in that array. (We need a 1:1 relation between
`dedicated_devices` and `devices` since we are using a `with_together`
later.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-06 15:00:32 +02:00
Sébastien Han 29888649e5 osd: do not do unique on dedicated_devices
This is needed later, if we do unique, only the first OSD will get a
journal.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-05 18:20:18 +02:00
Michel Rode b462b68e65 Fixing path to osd_fragment.yml 2017-10-05 14:42:10 +02:00
Guillaume Abrioux 70e2787fe2 docker: fix keyrings copied on all nodes
All keyring are getting copied to all nodes.
This commit fixes a leftover from a previous code refactor.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1498583

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-05 09:23:22 +02:00
Guillaume Abrioux 784cc73da0 set docker_exec_cmd fact early in each role
This is to ensure `docker_exec_cmd` fact is set with the correct value
in case of daemons collocation

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-04 11:31:09 +02:00
Sébastien Han 3bd341f6c0 osd: container use id instead of dev name
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1494127
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-03 14:44:00 +02:00
Sébastien Han ba42894516 osd: do not copy admin key on collocated scenario
ceph-disk used to have a bug requiring the admin key to store the
encrypted key in the mon kv store. This was reported in:
http://tracker.ceph.com/issues/17849

Fixed and backported here: https://github.com/ceph/ceph/pull/11996

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-03 14:44:00 +02:00
Guillaume Abrioux 466f6f35b7 Use systemd module instead of service.
Using systemd module allows us to do in one task what we did in three
tasks:

- enable unit file,
- issue a `daemon-reload`,
- start the service

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-29 14:54:00 +02:00
Guillaume Abrioux 913ad53709 docker: add condition to run selinux tasks only on rhel os family
This fixes the error :

```
The conditional check 'sestatus.stdout != 'Disabled'' failed.
```

that occurs when running on non rhel based system since the
`sestatus` fact is registered only on rhel based distribution.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-29 02:35:07 +02:00
Sébastien Han cb05172605 docker: we don't need to copy the ceph.conf on all the nodes
We generate the ceph.conf on all the nodes through the
ceph-docker-common so there is no need to push it to the Ansible file.

Also this is breaking the ceph.conf template generation since we only
generate sections based on the host the ansible task is running on.

For example, what's typically happening, we bootstrap the monitor, we
get a ceph.conf generated for a mon only, we go on an osd, we generate
the ceph.conf with osd section (done by ceph-docker-common) but this
gets overwritten by the copy_config task of the ceph-osd role.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-20 16:33:29 +02:00
Sébastien Han d100b4e596 name includes and set_fact for clarity
When Ansible is not run with verbose options it's difficult to see which
include and/or set_fact does what. So adding a name for each clarifies.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-18 23:39:46 +02:00
Sébastien Han 66d41f342d Merge pull request #1889 from ceph/client-containers
client: ability to create keys and pool with no ceph binaries
2017-09-18 17:27:32 +02:00
Sébastien Han 660893e70e osd: add meaningful message for journal_size
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-13 23:49:15 -06:00
Sébastien Han ef8d37dd0d Merge pull request #1800 from ceph/wip-osd-start-fix
ceph-osd: Fix osd start sequence
2017-09-13 17:20:10 -06:00
Sébastien Han f67b47d056 Merge pull request #1882 from ceph/multi-journal
osd: drop support for device partition
2017-09-13 11:43:48 -06:00
Sébastien Han ac62437609 Merge pull request #1883 from ceph/quick_refact
osd: refact include of `activate_osds.yml`
2017-09-12 22:11:31 -06:00
Sébastien Han fdf924401f osd: drop support for device partition
We have been struggling with this, it's still broken and breaking other
things too now.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1490283
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-12 17:42:07 -06:00
Guillaume Abrioux 49ad8528e5 osd: refact include of `activate_osds.yml`
remove duplicate code.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-12 16:53:11 -06:00
Guillaume Abrioux 0f506f4f0a Docker: split the task 'copy ceph configs&keys'
All keys are copied to all nodes.
This commit split that task in each roles so keys are copied to their
respective nodes.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1488999

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-11 21:14:13 +02:00
Sébastien Han 3753e6cfa7 ceph-osd: fix autodetection activation
Prior to this patch this activation sequence for autodetection was
always skipped because we were asking to activate on device without
partitions, which doesn't make sense.

We also fix the way we lookup for a device, since the data partition is
always numbered 1, we take the min element of the dict.

Closes: https://github.com/ceph/ceph-ansible/issues/1782
Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-07 17:47:37 +02:00
Sébastien Han 1dd976d28e ceph-osd: do not re-prepare if alreadyy prepared
I forgot to re-add the partition check while refactoring the osd

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-05 09:51:57 +02:00
Andrew Schoen fcba9d17f0 ceph-osd: add support for --journal vg/lv for lvm osds
This also updates the tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-30 15:55:16 -05:00
Sébastien Han e0a264c7e9 osd: allow multi dedicated journals for containers
Fix: https://bugzilla.redhat.com/show_bug.cgi?id=1475820
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-30 12:34:06 +02:00
Boris Ranto 5f1b8fcd75 ceph-osd: Fix osd start sequence
The script can fail to get the osd id because the osds are activated by
udev and it can take a while for them to activate. This commit fixes
that by trying to get all the osds per node in a loop.

This commit also makes the osd services enabled so that they are
available after reboot.

Signed-off-by: Boris Ranto <branto@redhat.com>
2017-08-25 13:40:04 +02:00
Andrew Schoen 758c31b1cd ceph-osd: ceph-volume requires --data to be in vg/lv format
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-23 13:43:31 -05:00
Andrew Schoen 594d5e017a ceph-osd: restructure lvm_volumes variable for more flexiblity
The lvm_volumes variable is now a list of dictionaries that represent
each OSD you'd like to deploy using ceph-volume. Each dictionary must
have the following keys: data, journal and data_vg. Each dictionary also
can optionaly provide a journal_vg key.

The 'data' key represents the lv name used for the OSD and the 'data_vg'
key is the vg name that the given lv resides on. The 'journal' key is
either an lv, device or partition. The 'journal_vg' key is optional and
must be the vg name for the journal lv if given. This key is mainly used
for purging of the journal lv if purge-cluster.yml is run.

For example:

  lvm_volumes:
    - data: data_lv1
      journal: journal_lv1
      data_vg: vg1
      journal_vg: vg2
    - data: data_lv2
      journal: /dev/sdc
      data_vg: vg1

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-23 10:14:14 -05:00
Sébastien Han 07821d9bb1 Merge pull request #1786 from ceph/re-arrange-skipped
mon, osd: fix skipped condition
2017-08-22 19:44:48 +02:00
Sébastien Han a359fc35b4 mon, osd: fix skipped condition
To be properly evaluated the "skipped" conditions must always have the
first place on the list of condition, otherwise the other conditions are
evaluated before and make the task fail.

Closes: https://github.com/ceph/ceph-ansible/issues/1733
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-22 18:34:51 +02:00
Andy McCrae 4671b9e74e Allow ceph service systemd overrides to be specified
ceph services can fail to start under certain circumstances (for
example, when running in a container) because the default systemd
service configuration causes namespace issues.

To work around this we can override the system service settings by
placing an overrides file in the ceph-<service>@.service.d directory.
This can be generic so as to allow any potential changes required to
the ceph-<service> service files.

The overrides file is only setup when the
"ceph_<service>_systemd_overrides" config_template override variable is
specified.

The available service systemd override files are as follows:
ceph_mds_systemd_overrides
ceph_mgr_systemd_overrides
ceph_mon_systemd_overrides
ceph_osd_systemd_overrides
ceph_rbd_mirror_systemd_overrides
ceph_rgw_systemd_overrides
2017-08-16 17:57:06 +01:00
Andrew Schoen 1d5f876729 ceph-osd: devices is not required when osd_scenario == lvm
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:38:37 -05:00
Andrew Schoen e597628be9 lvm: update scenario for new osd_scenario variable
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:38:36 -05:00
Andrew Schoen 3b5a06bb3c lvm-osds: reorder mandatory vars checks
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:13:10 -05:00