Commit Graph

1528 Commits (acbdb7f4fe0aea6fcbaf639f485a96d0bfdde413)

Author SHA1 Message Date
Sébastien Han c527515502 Merge pull request #2000 from ceph/merge-osd-scenarios
[skip ci] ci: new osd scenarios
2017-10-19 09:18:02 +02:00
Guillaume Abrioux ff228e2d88 mgr: fix broken task on jewel
3a58757 introduced an issue for Jewel deployments, since this role is
skipped, `enabled_ceph_mgr_modules.stdout` doesn't exist, therefore, it
ends up with an attribute error.

Uses `.get()` to retrieve `stdout` with a default value so it won't fail
if this attribute doesn't exist (jewel).

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-18 14:11:46 +02:00
Sébastien Han 1579f1c5b1 Merge pull request #2073 from ceph/fix_rbd_handler
[skip ci] rbd: fix restart script for jewel
2017-10-18 11:12:05 +02:00
Guillaume Abrioux c2850b11be rbd: fix restart script for jewel
In Jewel, we don't use bootstrap-rbd keyring for rbd-mirror nodes, it
results with a socket path/name different according to which ceph
release you are deploying.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-18 11:10:49 +02:00
Sébastien Han 2936d152c9 Merge pull request #2053 from Fbrachere/mgr-modules
Add ability to enable ceph mgr modules.
2017-10-18 10:27:31 +02:00
Sébastien Han a53aa9e8b4 ci: new osd scenarios
This commit add new osd scenarios, it aims to simplify the CI setup and
brings a better coverage on the OSD scenarios.
We decided to differentiate between filestore and bluestore, thinking
ahead when filestore won't be supported anymore.
So we now have two classes of tests:

* Filestore
* Bluestore

In each of those classes we have container and non-container.
Then for each we test the following:

* collocated
* collocated dmcrypt
* non-collocated
* non-collocated dmcrypt
* auto discovery collocated
* auto discovery collocated dmcrypt

This gives us a nice coverage and also reduces the footprint on the CI.
We are now up to 4 scenarios, each containing 6 OSD VMs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-18 09:26:06 +02:00
Sébastien Han 90b75185d5 defaults: fix handlers for collocation
When doing collocation the condition "inventory_hostname in play_hosts"
is breaking the restart workflow.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-17 19:23:16 +02:00
Guillaume Abrioux 2aa53fb0f5 Merge pull request #2055 from ceph/update-mirror-nfs
upgrade: support for rbd mirror and nfs
2017-10-17 14:51:39 +02:00
Christian Berendt 4c380c9ef8 Cleanup readme files in roles directories
The contents of the README files are no longer up to date.
Documentation for all roles is located below the docs directory.
2017-10-17 11:22:06 +02:00
Sébastien Han d920d4839d upgrade: support for rbd mirror and nfs
- Add upgrade support for rbd mirror and nfs daemons.
- Only works with systemd (remove sysvinit and upstart occurence)
- A bit of cleanup

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-17 10:54:47 +02:00
Christian Berendt cf901f0171 In docker start scripts replace \u00a0 with \u0020
This will solve the following issue when starting docker containers on ubuntu:

invalid argument "1\u00a0" for --cpus=1 : failed to parse 1  as a rational number

Closes-bug: #2056
2017-10-16 15:16:48 +02:00
Fabien Brachere 3a587575d7 Add ability to enable ceph mgr modules. 2017-10-16 15:04:23 +02:00
Guillaume Abrioux 7ee9aa94b5 Merge pull request #1963 from ceph/pull-in-para
site-docker.yml try to fetch images in //
2017-10-13 19:35:11 +02:00
Sébastien Han 71d819620c mds: fix fs pool creation
1. add the variables to docker_collocation
2. trigger the check when a MDS is part of the inventory file, not when
we run on an MDS...

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-13 16:03:04 +02:00
Sébastien Han b34a04ea41 site-docker.yml try to fetch images in //
The container deployment is serialized, adding this task as a best
effort. If docker is already present we pull the image otherwise we wait
for the role to play.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-13 11:24:40 +02:00
Guillaume Abrioux 7d4b3f9989 Merge pull request #2047 from ceph/enable_ceph-rbd-mirror.target
rbd-mirror: enable ceph-rbd-mirror.target
2017-10-13 10:34:10 +02:00
Sébastien Han f7832e5eb9 Merge pull request #2031 from major/simplify-ntp
Simplify NTP checks/install
2017-10-13 09:16:20 +02:00
Guillaume Abrioux 59ca1065e9 rbd-mirror: enable ceph-rbd-mirror.target
on jewel `ceph-rbd-mirror.target` isn't enabled, therefore, if the node
is rebooted, the service doesn't get started.

from ceph-rbd-mirror unit file:
```
[Install]
WantedBy=ceph-rbd-mirror.target
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-13 08:27:43 +02:00
Sébastien Han b685aceede Merge pull request #2044 from major/avoid-jinja-in-when
Remove jinja2 delimiters from `when` keys
2017-10-12 22:23:06 +02:00
Major Hayden a1c76e834c
Simplify NTP checks/install
This patch simplifies the checks and installation tasks for NTP.

Debian and Red Hat had a check for NTP's presence but would then
install NTP right afterwards anyways. In addition, there were
tasks for atomic that weren't used anywhere else in the role.

This patch also uses a dynamic include to reduce delays from
skipped tasks.
2017-10-12 12:31:07 -05:00
Sébastien Han 9c3d749f7c Merge pull request #2038 from major/fix-cmd-warning
Suppress yum/dnf/rpm command warnings
2017-10-12 18:46:52 +02:00
Major Hayden c01851325e
Remove jinja2 delimiters from `when` keys
This patch changes the `when:` keys so that they have no jinja2
delimiters. This avoids Ansible warnings which could turn into
errors in a future Ansible release.
2017-10-12 11:27:42 -05:00
Guillaume Abrioux 17623a2157 Merge pull request #2036 from ceph/cephfs-pool
mds: precisely define cephfs pool
2017-10-12 17:47:10 +02:00
Sébastien Han b49f9bda21 mds: precisely define cephfs pool
We now have a variable called ceph_pools that is mandatory when
deploying a MDS.
It's a dictionnary that contains a pool name and a PG count. PG count is
mandatory and must be set, the playbook will fail otherwise.

Closes: https://github.com/ceph/ceph-ansible/issues/2017
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-12 15:56:04 +02:00
Major Hayden 33b200d43a
Suppress yum/dnf/rpm command warnings
Ansible throws warnings when using yum/dnf/rpm with the command
module:

    [WARNING]: Consider using yum module rather than running yum

This patch adds the `warn: no` argument to suppress the warnings
in the Ansible output.
2017-10-12 08:38:05 -05:00
Major Hayden 620fb37dd4
Avoid deprecated always_run
The `always_run` key is deprecated and being removed in Ansible 2.4.
Using it causes a warning to be displayed:

    [DEPRECATION WARNING]: always_run is deprecated.

This patch changes all instances of `always_run` to use the `always`
tag, which causes the task to run each time the playbook runs.
2017-10-12 08:29:44 -05:00
Sébastien Han 739a41ae91 Merge pull request #2030 from major/ceph-common-pass-pkgs-as-list
Pass list of packages instead of with_items
2017-10-12 09:15:58 +02:00
Major Hayden 9d62630303
Pass list of packages instead of with_items
Modern versions of Ansible can handle a list of packages passed
directly to the package modules. This patch optimizes the package
install process by passing the list of packages directly to the
module.
2017-10-11 12:18:15 -05:00
Sébastien Han aa70b07ae2 config: proper render ceph.conf when doing collocation
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-11 18:29:34 +02:00
Sébastien Han f50b170a49 Merge pull request #2022 from ceph/fix-purge-iscis
[skip ci] purge-iscsi: fix group name
2017-10-11 14:21:19 +02:00
Sébastien Han d0a9e57bfc osd: rollback bindmount of /run/udev
This is causing unknown issues when trying to start a dmcrypt container.
Basically the container is stuck at mount opening the LUKS device. This
is still unknown why this is causing trouble but we need to move
forward. Also, this doesn't seem to help in any ways to fix the race
condition we've seen.

Here is the log for dmcrypt:

cryptsetup 1.7.4 processing "cryptsetup --debug --verbose --key-file
key luksClose fbf8887d-8694-46ca-b9ff-be79a668e2a9"
Running command close.
Locking memory.
Installing SIGINT/SIGTERM handler.
Unblocking interruption on signal.
Allocating crypt device context by device
fbf8887d-8694-46ca-b9ff-be79a668e2a9.
Initialising device-mapper backend library.
dm version   [ opencount flush ]   [16384] (*1)
dm versions   [ opencount flush ]   [16384] (*1)
Detected dm-crypt version 1.14.1, dm-ioctl version 4.35.0.
Device-mapper backend running with UDEV support enabled.
dm status fbf8887d-8694-46ca-b9ff-be79a668e2a9  [ opencount flush ]
[16384] (*1)
Releasing device-mapper backend.
Trying to open and read device /dev/sdc1 with direct-io.
Allocating crypt device /dev/sdc1 context.
Trying to open and read device /dev/sdc1 with direct-io.
Initialising device-mapper backend library.
dm table fbf8887d-8694-46ca-b9ff-be79a668e2a9  [ opencount flush
securedata ]   [16384] (*1)
Trying to open and read device /dev/sdc1 with direct-io.
Crypto backend (gcrypt 1.5.3) initialized in cryptsetup library
version 1.7.4.
Detected kernel Linux 3.10.0-693.el7.x86_64 x86_64.
Reading LUKS header of size 1024 from device /dev/sdc1
Key length 32, device size 1943016847 sectors, header size 2050
sectors.
Deactivating volume fbf8887d-8694-46ca-b9ff-be79a668e2a9.
dm status fbf8887d-8694-46ca-b9ff-be79a668e2a9  [ opencount flush ]
[16384] (*1)
Udev cookie 0xd4d14e4 (semid 32769) created
Udev cookie 0xd4d14e4 (semid 32769) incremented to 1
Udev cookie 0xd4d14e4 (semid 32769) incremented to 2
Udev cookie 0xd4d14e4 (semid 32769) assigned to REMOVE task(2) with
flags         (0x0)
dm remove fbf8887d-8694-46ca-b9ff-be79a668e2a9  [ opencount flush
retryremove ]   [16384] (*1)
fbf8887d-8694-46ca-b9ff-be79a668e2a9: Stacking NODE_DEL [verify_udev]
Udev cookie 0xd4d14e4 (semid 32769) decremented to 1
Udev cookie 0xd4d14e4 (semid 32769) waiting for zero

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-11 13:21:37 +02:00
Major Hayden 10e1d464e5
Remove duplicate 'package' key
This patch fixes a typo where "package:" was used twice in the same
task.
2017-10-10 15:39:20 -05:00
Sébastien Han f6d1be269f Merge pull request #2015 from ceph/fix_nfs-ganesha-repos
nfs: move repository configuration in ceph-nfs role
2017-10-10 17:15:33 +02:00
Guillaume Abrioux 5dc9c640e8 nfs: add missing condition for debian_rhcs
in addition to c4dcdaa20 this commit adds the missing condition on
install tasks for debian_rhcs deployment. Without them, these tasks are
played on any kind of deployment.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-10 16:27:00 +02:00
Jan Provaznik 87b1da09e7 Ceph-nfs dynamic exports fixes
* DBus on host should include ganesha service file
* to allow ganesha container to respond on DBus it needs to run
  in --privileged mode (ganesha folks contacted to look at this)
* ceph_nfs_include_exports_dir variable replaced with more general
  ceph_nfs_dynamic_exports
2017-10-10 13:59:01 +02:00
Guillaume Abrioux fbd1a57b11 iscsi-gw: move repository configuration to ceph-iscsi-gw
This is something that has nothing to do in `ceph-common`, this
is too specific to `ceph-iscsi-gw` role.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-10 11:36:03 +02:00
Guillaume Abrioux c4dcdaa201 nfs: move repository configuration in ceph-nfs role
This is something that has nothing to do in `ceph-common`, this
is too specific to `ceph-nfs` role.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-10 11:35:58 +02:00
Guillaume Abrioux 9e8204d9e8 nfs: move packages installation to own role
Make role `ceph-nfs` handling itself the installation of nfs
packages.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-09 19:10:15 +02:00
Guillaume Abrioux 3c64abe07d mds: move installation packages in role itself
Make role `ceph-mds` handling itself the installation of `ceph-mds`
package.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-09 17:25:46 +02:00
Sébastien Han 4032f102fe iscsi: move package install to ceph-iscsi-role
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:46 +02:00
Guillaume Abrioux 1581a1c078 mgr: move installation packages in role itself
Make role `ceph-mgr` handling itself the installation of `ceph-mgr`
package because it's complicated to manage it regarding we are going to
install `jewel vs. luminous`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-09 17:25:45 +02:00
Sébastien Han bf99751ce1 osd: bindmount /run/udev
Ensures that "udevadm" is able to check the status of udev's event queue.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:45 +02:00
Sébastien Han 1bd891232c config: do not duplicate sections when doing collocation
Prior to this commit, when collocating a RGW and NFS on the same box the
ceph.conf layout was the following:

[client.rgw.rgw0]
host = mds0
host = rgw0
rgw frontends = civetweb port=192.168.15.50:8080
num_threads=100[client.rgw.mds0]
rgw frontends = civetweb port=192.168.15.70:8080 num_threads=100
rgw frontends = civetweb port=192.168.15.50:8080 num_threads=100
keyring = /var/lib/ceph/radosgw/test-rgw.mds0/keyring
keyring = /var/lib/ceph/radosgw/test-rgw.rgw0/keyring
rgw data = /var/lib/ceph/radosgw/test-rgw.rgw0
log file = /var/log/ceph/test-rgw-mds0.log
log file = /var/log/ceph/test-rgw-rgw0.log

[mds.mds0]
host = mds0

[global]
rgw override bucket index max shards = 16
fsid = 70e1d368-57b3-4978-b746-cbffce6e56b5
rgw bucket default quota max objects = 1638400
osd_pool_default_size = 1
public network = 192.168.15.0/24
mon host = 192.168.15.10,192.168.15.11,192.168.15.12
osd_pool_default_pg_num = 8
cluster network = 192.168.16.0/24

[mds.rgw0]
host = rgw0

[client.rgw.mds0]
host = mds0
rgw data = /var/lib/ceph/radosgw/test-rgw.mds0
keyring = /var/lib/ceph/radosgw/test-rgw.mds0/keyring
rgw frontends = civetweb port=192.168.15.70:8080 num_threads=100
log file = /var/log/ceph/test-rgw-mds0.log

Basically appending all the sections. This commits solves that.
Now the sections appear like this:

-bash-4.2# cat /etc/ceph/test.conf
[client.rgw.rgw0]
log file = /var/log/ceph/test-rgw-rgw0.log
host = rgw0
keyring = /var/lib/ceph/radosgw/test-rgw.rgw0/keyring
rgw frontends = civetweb port=192.168.15.50:8080 num_threads=100

[client.rgw.mds0]
log file = /var/log/ceph/test-rgw-mds0.log
host = mds0
keyring = /var/lib/ceph/radosgw/test-rgw.mds0/keyring
rgw frontends = civetweb port=192.168.15.70:8080 num_threads=100

[global]
cluster network = 192.168.16.0/24
mon host = 192.168.15.10,192.168.15.11,192.168.15.12
osd_pool_default_size = 1
public network = 192.168.15.0/24
rgw bucket default quota max objects = 1638400
osd_pool_default_pg_num = 8
rgw override bucket index max shards = 16
fsid = 77a21980-3033-4174-9264-1abc7185bcb3

[mds.rgw0]
host = rgw0

[mds.mds0]
host = mds0

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:44 +02:00
Sébastien Han 7054abef99 Merge pull request #2009 from ceph/fix-clean-pg
[skip ci] handler: do not test if pgs_num = 0
2017-10-07 03:39:26 +02:00
Sébastien Han 9f1bd3d6dd handler: add serial restart back
We now restart daemons on each machine in a serialized fashion.

Closes: https://github.com/ceph/ceph-ansible/issues/1989
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-07 03:39:10 +02:00
Sébastien Han a4dcef73d4 common: fix debian rhcs installation
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-07 03:39:09 +02:00
Sébastien Han c693e95cbf purge-docker: rework device detection
we don't need "devices" and other device variable anymore, the playbook
detects that for us.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-07 03:39:04 +02:00
Sébastien Han ac29e8f977 Merge pull request #1983 from jprovaznik/suffix
Allow to override systemd service instance id
2017-10-06 22:40:57 +02:00
Sébastien Han 5d39f378da Merge pull request #1984 from jprovaznik/exportdir
Include exports dir in ceph-nfs config file
2017-10-06 22:38:13 +02:00
Ali Maredia 28862a99d9 nfs: missing conditional for setting rgw key permissions
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-10-06 15:21:35 -04:00