Commit Graph

3376 Commits (61b4a1bedcf7259f4ee00af9297754e0edd7bf0c)
 

Author SHA1 Message Date
Major Hayden a1c76e834c
Simplify NTP checks/install
This patch simplifies the checks and installation tasks for NTP.

Debian and Red Hat had a check for NTP's presence but would then
install NTP right afterwards anyways. In addition, there were
tasks for atomic that weren't used anywhere else in the role.

This patch also uses a dynamic include to reduce delays from
skipped tasks.
2017-10-12 12:31:07 -05:00
Sébastien Han 9c3d749f7c Merge pull request #2038 from major/fix-cmd-warning
Suppress yum/dnf/rpm command warnings
2017-10-12 18:46:52 +02:00
Major Hayden c233c874f5
Enable profile_tasks callback plugin
This patch adds the `profile_tasks` callback plugin to the whitelist
so that we can identify the tasks which are taking the longest amount
of time to run.
2017-10-12 11:43:29 -05:00
Major Hayden c01851325e
Remove jinja2 delimiters from `when` keys
This patch changes the `when:` keys so that they have no jinja2
delimiters. This avoids Ansible warnings which could turn into
errors in a future Ansible release.
2017-10-12 11:27:42 -05:00
Guillaume Abrioux 17623a2157 Merge pull request #2036 from ceph/cephfs-pool
mds: precisely define cephfs pool
2017-10-12 17:47:10 +02:00
Sébastien Han 6bd152d555 Merge pull request #2037 from major/remove-always-run
Avoid deprecated always_run
2017-10-12 17:15:28 +02:00
Sébastien Han c6210fa85e Merge pull request #2028 from ceph/fix_testinfra
tests: rbd/rgw adapt testinfra for jewel
2017-10-12 16:19:33 +02:00
Sébastien Han b49f9bda21 mds: precisely define cephfs pool
We now have a variable called ceph_pools that is mandatory when
deploying a MDS.
It's a dictionnary that contains a pool name and a PG count. PG count is
mandatory and must be set, the playbook will fail otherwise.

Closes: https://github.com/ceph/ceph-ansible/issues/2017
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-12 15:56:04 +02:00
Major Hayden 33b200d43a
Suppress yum/dnf/rpm command warnings
Ansible throws warnings when using yum/dnf/rpm with the command
module:

    [WARNING]: Consider using yum module rather than running yum

This patch adds the `warn: no` argument to suppress the warnings
in the Ansible output.
2017-10-12 08:38:05 -05:00
Major Hayden 620fb37dd4
Avoid deprecated always_run
The `always_run` key is deprecated and being removed in Ansible 2.4.
Using it causes a warning to be displayed:

    [DEPRECATION WARNING]: always_run is deprecated.

This patch changes all instances of `always_run` to use the `always`
tag, which causes the task to run each time the playbook runs.
2017-10-12 08:29:44 -05:00
Guillaume Abrioux a179e312fd tests: add missing override for collocation scenario
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-12 14:43:25 +02:00
Sébastien Han 002b0341d0 Merge pull request #2035 from ceph/replace-disk
[skip ci] infra: replace osd playbook
2017-10-12 11:55:23 +02:00
Sébastien Han 13bce287ad infra: replace osd playbook
This playbook can replace failed OSD in containerized and
non-containerized env.
The current limitation is that it won't allow you to choose between
filestore/bluestore and will do collocation as well.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-12 11:53:30 +02:00
Sébastien Han 739a41ae91 Merge pull request #2030 from major/ceph-common-pass-pkgs-as-list
Pass list of packages instead of with_items
2017-10-12 09:15:58 +02:00
Guillaume Abrioux a2880e6345 tests: rbd/rgw adapt testinfra for jewel
- the rbd-mirror unit systemd name is not the same when running jewel vs
luminous.
- servicemap is not available on jewel.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-12 00:06:08 +02:00
Sébastien Han 4afd51c19d Merge pull request #2029 from ceph/rgw-jinja
config: fix rgw section rendering for collocation
2017-10-11 22:22:48 +02:00
Major Hayden 9d62630303
Pass list of packages instead of with_items
Modern versions of Ansible can handle a list of packages passed
directly to the package modules. This patch optimizes the package
install process by passing the list of packages directly to the
module.
2017-10-11 12:18:15 -05:00
Sébastien Han aa70b07ae2 config: proper render ceph.conf when doing collocation
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-11 18:29:34 +02:00
Sébastien Han b561252da8 Merge pull request #2027 from ceph/hub-backport
[skip ci]
2017-10-11 15:20:19 +02:00
Sébastien Han 155dd36269 contrib: backport auto create PR
Fix branch and repo

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-11 15:19:34 +02:00
Sébastien Han 4afc400093 Merge pull request #2026 from ceph/hub-backport
[skip ci]
2017-10-11 15:16:35 +02:00
Sébastien Han 32625c5e2b contrib: backport auto create PR
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-11 14:47:43 +02:00
Sébastien Han f50b170a49 Merge pull request #2022 from ceph/fix-purge-iscis
[skip ci] purge-iscsi: fix group name
2017-10-11 14:21:19 +02:00
Sébastien Han 81c04b649e Merge pull request #2021 from major/remove-duplicate-package-key
Remove duplicate 'package' key
2017-10-11 14:17:21 +02:00
Sébastien Han d0a9e57bfc osd: rollback bindmount of /run/udev
This is causing unknown issues when trying to start a dmcrypt container.
Basically the container is stuck at mount opening the LUKS device. This
is still unknown why this is causing trouble but we need to move
forward. Also, this doesn't seem to help in any ways to fix the race
condition we've seen.

Here is the log for dmcrypt:

cryptsetup 1.7.4 processing "cryptsetup --debug --verbose --key-file
key luksClose fbf8887d-8694-46ca-b9ff-be79a668e2a9"
Running command close.
Locking memory.
Installing SIGINT/SIGTERM handler.
Unblocking interruption on signal.
Allocating crypt device context by device
fbf8887d-8694-46ca-b9ff-be79a668e2a9.
Initialising device-mapper backend library.
dm version   [ opencount flush ]   [16384] (*1)
dm versions   [ opencount flush ]   [16384] (*1)
Detected dm-crypt version 1.14.1, dm-ioctl version 4.35.0.
Device-mapper backend running with UDEV support enabled.
dm status fbf8887d-8694-46ca-b9ff-be79a668e2a9  [ opencount flush ]
[16384] (*1)
Releasing device-mapper backend.
Trying to open and read device /dev/sdc1 with direct-io.
Allocating crypt device /dev/sdc1 context.
Trying to open and read device /dev/sdc1 with direct-io.
Initialising device-mapper backend library.
dm table fbf8887d-8694-46ca-b9ff-be79a668e2a9  [ opencount flush
securedata ]   [16384] (*1)
Trying to open and read device /dev/sdc1 with direct-io.
Crypto backend (gcrypt 1.5.3) initialized in cryptsetup library
version 1.7.4.
Detected kernel Linux 3.10.0-693.el7.x86_64 x86_64.
Reading LUKS header of size 1024 from device /dev/sdc1
Key length 32, device size 1943016847 sectors, header size 2050
sectors.
Deactivating volume fbf8887d-8694-46ca-b9ff-be79a668e2a9.
dm status fbf8887d-8694-46ca-b9ff-be79a668e2a9  [ opencount flush ]
[16384] (*1)
Udev cookie 0xd4d14e4 (semid 32769) created
Udev cookie 0xd4d14e4 (semid 32769) incremented to 1
Udev cookie 0xd4d14e4 (semid 32769) incremented to 2
Udev cookie 0xd4d14e4 (semid 32769) assigned to REMOVE task(2) with
flags         (0x0)
dm remove fbf8887d-8694-46ca-b9ff-be79a668e2a9  [ opencount flush
retryremove ]   [16384] (*1)
fbf8887d-8694-46ca-b9ff-be79a668e2a9: Stacking NODE_DEL [verify_udev]
Udev cookie 0xd4d14e4 (semid 32769) decremented to 1
Udev cookie 0xd4d14e4 (semid 32769) waiting for zero

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-11 13:21:37 +02:00
Sébastien Han 85e13a864c purge-iscsi: fix group name
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1500281
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-11 12:52:12 +02:00
Major Hayden 10e1d464e5
Remove duplicate 'package' key
This patch fixes a typo where "package:" was used twice in the same
task.
2017-10-10 15:39:20 -05:00
Sébastien Han f6d1be269f Merge pull request #2015 from ceph/fix_nfs-ganesha-repos
nfs: move repository configuration in ceph-nfs role
2017-10-10 17:15:33 +02:00
Guillaume Abrioux 5dc9c640e8 nfs: add missing condition for debian_rhcs
in addition to c4dcdaa20 this commit adds the missing condition on
install tasks for debian_rhcs deployment. Without them, these tasks are
played on any kind of deployment.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-10 16:27:00 +02:00
Sébastien Han 90d4ac5db5 Merge pull request #2020 from jprovaznik/dexports
Ceph-nfs dynamic exports fixes
2017-10-10 15:17:28 +02:00
Jan Provaznik 87b1da09e7 Ceph-nfs dynamic exports fixes
* DBus on host should include ganesha service file
* to allow ganesha container to respond on DBus it needs to run
  in --privileged mode (ganesha folks contacted to look at this)
* ceph_nfs_include_exports_dir variable replaced with more general
  ceph_nfs_dynamic_exports
2017-10-10 13:59:01 +02:00
Guillaume Abrioux fbd1a57b11 iscsi-gw: move repository configuration to ceph-iscsi-gw
This is something that has nothing to do in `ceph-common`, this
is too specific to `ceph-iscsi-gw` role.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-10 11:36:03 +02:00
Guillaume Abrioux ea7a8fb807 Docs: fix error `Unknown target name`
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-10 11:36:03 +02:00
Guillaume Abrioux c4dcdaa201 nfs: move repository configuration in ceph-nfs role
This is something that has nothing to do in `ceph-common`, this
is too specific to `ceph-nfs` role.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-10 11:35:58 +02:00
Sébastien Han a10f12954d Merge pull request #2018 from ceph/purge-journal
[skip ci] purge: fix journal purge
2017-10-10 10:01:18 +02:00
Sébastien Han 24b82c2679 purge: fix journal purge
Using a condition when osd_scenario == 'non-collocated' was wrong since
these partitions can be collocated on a single device also. Removing the
check makes the purge of these partitions.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1499871
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-10 09:57:39 +02:00
Guillaume Abrioux f147b119ed Merge pull request #2014 from ceph/fixes-2
infra: use the pg check in the right place
2017-10-09 20:14:06 +02:00
Guillaume Abrioux 9e8204d9e8 nfs: move packages installation to own role
Make role `ceph-nfs` handling itself the installation of nfs
packages.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-09 19:10:15 +02:00
Guillaume Abrioux 3c64abe07d mds: move installation packages in role itself
Make role `ceph-mds` handling itself the installation of `ceph-mds`
package.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-09 17:25:46 +02:00
Sébastien Han 4032f102fe iscsi: move package install to ceph-iscsi-role
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:46 +02:00
Guillaume Abrioux 1581a1c078 mgr: move installation packages in role itself
Make role `ceph-mgr` handling itself the installation of `ceph-mgr`
package because it's complicated to manage it regarding we are going to
install `jewel vs. luminous`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-09 17:25:45 +02:00
Guillaume Abrioux a1ea6e7f59 tests: adapt current testing for collocation scenario
Since we introduced collocation testing scenario, we need to adapt
current tests to this new scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-09 17:25:45 +02:00
Sébastien Han 88a37ca990 ci: disable vnc console
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:45 +02:00
Sébastien Han bf99751ce1 osd: bindmount /run/udev
Ensures that "udevadm" is able to check the status of udev's event queue.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:45 +02:00
Sébastien Han 6d7b73fa91 ci: re-add osd_pool_default_size to 1 with the override
If we don't do this the client will create pools with a replica 3 since
osd_pool_default_size was gone in ceph-override.json. This was making
switch_to_containers failing.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:45 +02:00
Sébastien Han e15302c284 tox: decrease sleep to 2min instead of 5.
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:44 +02:00
Sébastien Han 450108fab9 infra: add independant purge-iscsi-gateways.yml
The current inclusion of purge-iscsi-gateways.yml in purge-cluster.yml
is not working well and blocking the CI too. So removing it from
purge-cluster.yml and re-add the original purge-iscsi-gateways.yml.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:44 +02:00
Sébastien Han 1bd891232c config: do not duplicate sections when doing collocation
Prior to this commit, when collocating a RGW and NFS on the same box the
ceph.conf layout was the following:

[client.rgw.rgw0]
host = mds0
host = rgw0
rgw frontends = civetweb port=192.168.15.50:8080
num_threads=100[client.rgw.mds0]
rgw frontends = civetweb port=192.168.15.70:8080 num_threads=100
rgw frontends = civetweb port=192.168.15.50:8080 num_threads=100
keyring = /var/lib/ceph/radosgw/test-rgw.mds0/keyring
keyring = /var/lib/ceph/radosgw/test-rgw.rgw0/keyring
rgw data = /var/lib/ceph/radosgw/test-rgw.rgw0
log file = /var/log/ceph/test-rgw-mds0.log
log file = /var/log/ceph/test-rgw-rgw0.log

[mds.mds0]
host = mds0

[global]
rgw override bucket index max shards = 16
fsid = 70e1d368-57b3-4978-b746-cbffce6e56b5
rgw bucket default quota max objects = 1638400
osd_pool_default_size = 1
public network = 192.168.15.0/24
mon host = 192.168.15.10,192.168.15.11,192.168.15.12
osd_pool_default_pg_num = 8
cluster network = 192.168.16.0/24

[mds.rgw0]
host = rgw0

[client.rgw.mds0]
host = mds0
rgw data = /var/lib/ceph/radosgw/test-rgw.mds0
keyring = /var/lib/ceph/radosgw/test-rgw.mds0/keyring
rgw frontends = civetweb port=192.168.15.70:8080 num_threads=100
log file = /var/log/ceph/test-rgw-mds0.log

Basically appending all the sections. This commits solves that.
Now the sections appear like this:

-bash-4.2# cat /etc/ceph/test.conf
[client.rgw.rgw0]
log file = /var/log/ceph/test-rgw-rgw0.log
host = rgw0
keyring = /var/lib/ceph/radosgw/test-rgw.rgw0/keyring
rgw frontends = civetweb port=192.168.15.50:8080 num_threads=100

[client.rgw.mds0]
log file = /var/log/ceph/test-rgw-mds0.log
host = mds0
keyring = /var/lib/ceph/radosgw/test-rgw.mds0/keyring
rgw frontends = civetweb port=192.168.15.70:8080 num_threads=100

[global]
cluster network = 192.168.16.0/24
mon host = 192.168.15.10,192.168.15.11,192.168.15.12
osd_pool_default_size = 1
public network = 192.168.15.0/24
rgw bucket default quota max objects = 1638400
osd_pool_default_pg_num = 8
rgw override bucket index max shards = 16
fsid = 77a21980-3033-4174-9264-1abc7185bcb3

[mds.rgw0]
host = rgw0

[mds.mds0]
host = mds0

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:44 +02:00
Sébastien Han 774697ebd8 infra: use the pg check in the right place
Use the pg check before doing the pg check, not on the quorum check.
Also never quote int when doing comparaison.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-09 17:25:41 +02:00
Sébastien Han a3e7bcb13f Merge pull request #2013 from ceph/wip-purge-cluster
A couple of purge cluster fixes
2017-10-09 17:18:30 +02:00