Commit Graph

1417 Commits (69aadbb8ceac02dd85e5d7277a22fc32d3ee99c0)

Author SHA1 Message Date
Sébastien Han d2320e412e osd: docker, refactor ceph-osd-run.sh.j2
Easier to read and enhance.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-06 15:49:14 +02:00
Guillaume Abrioux 7e1df484db Mon: Allow to deploy with custom admin secret
Add a new parameter `admin_secret` that allow to deploy a ceph cluster
with a custom admin secret.

Fix: #1630

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-05 14:57:49 +02:00
Sébastien Han e7ae65b6f9 Merge pull request #1632 from ceph/bluestore-redundant-options
osd: remove redundant options to enable bluestore
2017-07-04 19:33:54 +02:00
Sébastien Han 7d657ac643 osd: ability to set db and wal to bluestore
This commits refactors how we deploy bluestore. We have existing
scenarios that we don't want to change too much. This commits eases the
user experience by now changing the way you use scenarios. Bluestore is
just a different interface to store objects but the scenarios more or
less remain the same.

If you set osd_objectstore == 'bluestore' along with
journal_collocation: true, you will get an OSD running bluestore with DB
and WAL partitions on the same device.

If you set osd_objectstore == 'bluestore' along with
raw_multi_journal: true, you will get an OSD running bluestore with a
dedicated drive for the rocksdb DB, then the remaining
drives (used with 'devices') will have WAL and DATA collocated.

If you set osd_objectstore == 'bluestore' along with
raw_multi_journal: true and declare bluestore_wal_devices you will get
an OSD running bluestore with a dedicated drive for rocksdb db, a
dedicated drive partition for rocksdb WAL and a dedicated drive for
DATA.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-04 19:07:16 +02:00
Sébastien Han adf752f241 Merge pull request #1597 from czunker/openstack_pools_with_rule
Create OpenStack pools with crush rule
2017-07-04 18:34:00 +02:00
Sébastien Han fc0e54c59e osd: remove redundant options to enable bluestore
There is no need for 2 variables to enable bluestore, prior to this
patch one had to do the following to activate bluestore:

osd_objectstore: bluestore
bluestore: true

Now you just need to set `osd_objectstore: bluestore`.

Fixes: https://github.com/ceph/ceph-ansible/issues/1475
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-04 18:22:03 +02:00
Guillaume Abrioux 896d62d78b Refact: remove ceph_mon_docker_interface variable
remove `ceph_mon_docker_interface` and use `monitor_interface` instead
for both containerized and non-containerized deployment.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-04 18:08:59 +02:00
Guillaume Abrioux f16037fac5 Common: monitor_address_block is always defined
monitor_address_block is always defined.
We need to check for true or false instead.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-04 17:50:30 +02:00
Guillaume Abrioux a951b7f957 Docker-common: Add missing variables declaration
Some variables are missing from ceph-docker-common role since the
include of check_mandatory_vars.yml has been re-added in the ceph-mon
role.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-04 17:50:23 +02:00
Sébastien Han 1c2d3a0b79 Merge pull request #1594 from ceph/ipv6_support
Common: Add ipv6 support
2017-07-04 15:08:47 +02:00
Guillaume Abrioux 0ab9be9a7a Mon: Readd the include of check_mandatory_vars.yml
The check regarding the networking scenario configuration has been
moved from ceph-common to ceph-mon in 1de8176 but the include was not re-added
in 189f4fe

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-04 10:57:27 +02:00
Guillaume Abrioux 675087d01c Doc: Add a note to clarify how to setup network
Explain how to setup networking in ceph-ansible by adding a note in
README.md.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-04 10:57:27 +02:00
Guillaume Abrioux 88df105d0b Common: Add ipv6 support
e8187f6 does not fix the ipv6 as expected since `ansible_default_*` are
filled with the IP address carried by the network interface used by the
default gateway route. By the way, it assumes that the MON_IP address will
be this IP address which is not always the case.

We need to keep using the previous fact but add some intelligence in the
template to determine how to retrieve the ipv4|ipv6 address since the path
to the fact in `hostvars` is not the same according to ipv4 vs ipv6 case.

Fix: 1569

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-04 10:57:26 +02:00
Christian Zunker 0050f8e6ef Create OpenStack pools with crush rule
Add an extra variable to the openstack pools, which creates them with
defined rules. This will allow to place different pools on e.g.
different type of disks.

This commit will also set a new default rule when defined and move
the rbd pool to the new rule.
2017-07-03 15:30:38 +02:00
Sébastien Han 78d95aeb63 Merge pull request #1636 from fghaas/fix-mon-directory-perms
Ensure that ceph-mon applies mon directory ownership recursively
2017-07-03 11:19:53 +02:00
Florian Haas 18e6c35dee Ensure that ceph-mon applies mon directory ownership recursively
Fixes #1635.

Signed-off-by: Florian Haas <florian@hastexo.com>
2017-06-30 10:18:51 +02:00
Douglas Fuller 6915dfcf81 ansible: fail if user selects OSD auto detection and raw devices are mounted
Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-06-29 17:02:17 +00:00
Sébastien Han d6ba1d1d4e mon: fix openstack key creation
Somehow the shell module will return an error if the command line is not
next to it.
Plus fixed the import with the right path.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-06-27 11:47:02 +02:00
Guillaume Abrioux 3dfeffab43 Fix followup on refact code (1469)
In addition to 7bb04a5, these lines are no longer needed and can even
cause playbook failures.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-06-26 15:53:41 +02:00
Sébastien Han 0c58257004 Merge pull request #1621 from ceph/openstack-key
mon: rework openstack keys creation
2017-06-26 14:39:05 +02:00
Sébastien Han 010897d1a5 Merge pull request #1624 from ceph/chose-ceph-config-location
Choose ceph config location
2017-06-26 14:38:54 +02:00
Sébastien Han 670d7a2495 mon: rework openstack keys creation
We now allow a user to pass a key secret.

Fixes: https://github.com/ceph/ceph-ansible/issues/1617
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-06-26 13:39:22 +02:00
Sébastien Han 0c21fb3f66 docker: ability to change ceph config directory
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-06-26 13:21:56 +02:00
Sébastien Han 7bb04a5970 docker: refactor followup
Followup on https://github.com/ceph/ceph-ansible/pull/1469 where we
merged most of the container code from roles/ceph-*/task/docker/*.yml
into roles/ceph-docker-common/tasks/

It seems that we forgot to remove the original files.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-06-26 13:21:36 +02:00
Guillaume Abrioux 73141118d0 Make the new check PGs working with /bin/sh
The new test in the checks PGs are no longer working on distributions
where /bin/sh isn't linked to /bin/bash.

Fix: #1619
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-06-22 17:59:38 +02:00
John Fulton 9619ef33d0 Add OpenStack metrics pool
OpenStack's Gnocchi service expects to have a pool called "metrics".
This change addess "metrics" to the list of `openstack_pools` and
creates a corresponding key. It is only run if the user sets
`openstack_config: false`.
2017-06-19 14:25:59 -04:00
Sébastien Han 62192df644 Merge pull request #1595 from czunker/restart_all_osds
Restart all OSDs and do not stop after the first one.
2017-06-16 11:15:47 +02:00
Christian Zunker bf8347e149 Restart all OSDs and do not stop after the first one.
The current handler only restarts one OSD on each OSD server. After
the first one the handler stops, not matter what results the checks had.

Co-Authored-By: Gaudenz Steinlin (@gaudenz)
2017-06-14 09:38:07 +00:00
Peter Jenkins 804b0eef24 Bluestore: Omit "osd mkfs type" etc from ceph.conf
Remove "osd mkfs type" and the other pre-Bluestore parameters from the
generated ceph.conf so that disk activation on OSDs will work. The
current default xfs config results in a failed deployment and
incorrect partition metadata.
2017-06-14 10:44:13 +03:00
Sébastien Han 497924795d ceph-mon: fix get rbd size hanging
For newly created cluster the command: ceph --cluster {{ cluster }} osd
pool get rbd size does not respond properly.
We only want to check if the rbd pool exists, so we know use an ls |
grep approach.

Closes: https://github.com/ceph/ceph-ansible/issues/1547
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-06-12 14:39:39 +02:00
Guillaume Abrioux 304de4833f Common: Rewrite check_pgs
Rewrite the check_pgs by using json parsing instead of complex regexp to
parse the `ceph -s` output.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-06-12 10:59:16 +02:00
Guillaume Abrioux a09ce92d51 Common: Add a default for ceph_docker_on_openstack
Add a default value for `ceph_docker_on_openstack` to avoid a
conditional check error for the task `pause after docker install before starting` in
`roles/ceph-docker-common/tasks/pre_requisites/prerequisites.yml`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-06-06 16:49:04 +02:00
Andrew Schoen e8187f6a0f ceph-mon: fix support for ipv6 on containerized mons
The fact ['ansible_$interface']['ipv4'] is a dictionary where
['ansible_$interface']['ipv6'] is a list. If we use
ansible_default_ipv6|ipv4 is is always a dictionary which allows us to
get the ipv6 and ipv4 address without adding more complexity to the
template.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-06-05 10:51:47 -05:00
Sébastien Han fdc7866072 Merge pull request #1469 from ceph/refact_code
Docker: Refact code
2017-06-02 12:40:25 +02:00
Sébastien Han bd4a7dd6c8 Merge pull request #1580 from ceph/fix_check_pgs
Common: Improve check pgs
2017-06-02 12:11:05 +02:00
Guillaume Abrioux 0542a95b68 Common: Improve check pgs
For some reason we changed the check of pgs but it appears it could be
dangerous because the current check might satisfied as long as 1 PG is
active+clean.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-06-01 20:12:36 +02:00
Guillaume Abrioux 0a2048a577 Docker: Remove duplicate var passed to docker-run
since `-e CEPH_DAEMON=OSD_CEPH_DISK_ACTIVATE` is already hardcoded in
`eph-osd-run.sh.j2` there is no need to add `-e
CEPH_DAEMON=OSD_CEPH_DISK_ACTIVATE` as a default value in defaults vars.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-06-01 14:31:17 +02:00
Guillaume Abrioux ddfe019342 Refact code
`ceph-docker-common`:
  At the moment there is a lot of duplicated tasks in each
  `./roles/ceph-<role>/tasks/docker/main.yml` that could be refactored in
  `./roles/ceph-docker-common/tasks/main.yml`.

`*_containerized_deployment` variables:
  All `*_containerized_deployment` have been refactored to a single
  variable `containerized_deployment`

duplicate `cephx` variables in `group_vars/* have been removed.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-05-24 15:55:41 +02:00
Guillaume Abrioux f0adecf482 Clean osds.yml.sample
Remove duplicate lines in osds.yml default vars file.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-05-24 15:55:41 +02:00
Andrew Schoen 2326c5ac63 Merge pull request #1557 from ceph/install-condition
common: fix installation condition
2017-05-24 06:39:36 -05:00
Sébastien Han 468dc06bcd common: remove useless check
We only check for everything expect 'distro' because that
is a valid way of deploying RHCS, with preprepared repos
present on the nodes.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-05-24 11:52:22 +02:00
Austin Workman 22033bd1bf Fixing partition detection regex for FusionIO devices. 2017-05-23 14:39:39 -05:00
Sébastien Han f7e9585a2c common: fix installation condition
Problem: we could end up in situation where we would install a package
on a machine that does not have the right repo enabled. Because the
condition was set to OR we weren't pinning a particular host but just a
condition. Let's say someone sets 'ceph_origin == "distro"', this would
try to install OSD packages on Monitors.

Solution: use a AND condition to first pin to the group_name (which
identifies a set of hosts) AND then after this one of the installation
condition.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1453119
Co-Authored-By: https://github.com/zhsj
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-05-23 11:50:58 +02:00
Sébastien Han 8ad503b248 common: explicitly set rhel os version support
Clarify in the error message that only RHEL version >= 7.3 are
supported.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1452431
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-05-19 10:38:20 +02:00
Sébastien Han 6bdadc4363 Revert "docker: Retry OSD disk prepare to workaround race condition" 2017-05-18 16:03:16 +02:00
Sébastien Han c7aae7f965 mon: do not hardcode ipv4
Problem: fail to deploy a containerized Ceph cluster with ipv6
Solution: do not hardcode ipv4 when bootstrapping the container.

Now use ip_version: ipv6 to get a containerized cluster deployed with
ipv6.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1451786
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-05-18 11:56:55 +02:00
Andrew Schoen 58618aa778 Merge pull request #1531 from ceph/wip-1495
docker: Retry OSD disk prepare to workaround race condition
2017-05-17 09:36:07 -05:00
Sébastien Han 1da3a3106f Merge pull request #1513 from flokli/monitor_interface
monitor_interface: document need to use monitor_address when using IPv6
2017-05-17 15:12:38 +02:00
Sébastien Han 4555f3c04a Merge pull request #1527 from ceph/piwi3910-master
added support for Sandisk FusionIO devices
2017-05-17 15:11:32 +02:00
Sébastien Han cf25ebb17c Merge pull request #1500 from yanyixing/master
ignore error when key is not exist
2017-05-17 15:04:20 +02:00
Guillaume Abrioux 1e7010de7f Docker: rm container before retry of ceph osd prepare
In addition to `196fa7e` this commit check if a container has been
already launched and delete it before retrying the ceph osd prepare
process.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-05-17 10:10:49 +02:00
Pascal Watteel e4ef8bb87f added support for Sandisk FusionIO devices
Signed-off-by: Pascal Watteel <pascal.watteel@emc.com>
2017-05-16 12:00:21 +02:00
Sébastien Han 213d8acedf Merge pull request #1518 from ceph/pgs-error-message
ceph-common: improve error message on restart osd daemon script
2017-05-15 13:58:35 +02:00
Alfredo Deza b04d18988e ceph-common: improve error message on restart osd daemon script
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-05-12 11:37:38 -04:00
Sébastien Han f1651cea9b mon: do not create mgr key on jewel
The CI on Docker is reporting the following error:

STDERR:
Error EINVAL: bad entity name

This is due to the fact that this auth entity name does not exist on
Jewel so we should not create that key when running Jewel containers.

Fixes: https://github.com/ceph/ceph-ansible/issues/1514

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-05-12 17:22:24 +02:00
David Galloway 196fa7ef39 docker: Retry osd disk prep to workaround race condition
Fixes: https://github.com/ceph/ceph-ansible/issues/1495

Signed-off-by: David Galloway <dgallowa@redhat.com>
2017-05-11 16:19:11 -04:00
Florian Klink 10b91661ce monitor_interface: document need to use monitor_address when using IPv6
Already documented in the Red Hat Ceph Storage 2 Installation Guide
for Red Hat Enterprise Linux, but not here

Signed-off-by: Florian Klink <flokli@flokli.de>
2017-05-11 11:17:10 +02:00
Ali Maredia 2aeb3a4957 rgw: move default bucket quota conf vars to global
"rgw override bucket index max shards" and
"rgw bucket default quota max objects" were in the
client section of the ceph.conf and not being
applied, this commit moves them to global

Resolves: bz#1391500

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-05-10 13:06:48 -04:00
Sébastien Han 31267ab48a Merge pull request #1460 from albertomurillo/clearlinux
Add clearlinux to supported platforms
2017-05-09 18:15:55 -04:00
Andrew Schoen b38b69b603 ceph-osd: fix typo in containerized OSD systemd unit
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-05-08 15:30:45 -05:00
yanyx 027f9aaf09 ignore error when key is not exist 2017-05-05 16:44:58 +08:00
Guillaume Abrioux 48bd807508 Common: Do not install ceph-mgr packages on jewel
ceph-mgr tasks has to be skipped on jewel.

Fix: #1494

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-05-04 18:38:59 +02:00
Andrew Schoen 471cdb0c39 ceph-common: add luminous to ceph_release_num
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-05-03 09:11:52 -05:00
Andrew Schoen 598fe0cada ceph-common: remove debian_ceph_packages
We shouldn't need this anymore as the upgrade bug that
debian_ceph_packages was used to workaround should have
been fixed as of jewel.

See https://github.com/ceph/ceph-ansible/issues/1481 for more
detailed information.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-04-28 10:51:51 -05:00
Guillaume Abrioux b61deacfc0 Common: Fix condition on osd restart handler
Fix the condition on the osd restart handler.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-04-27 16:24:21 +02:00
Ali Maredia 5192e3cd6d rgw: set tuning parameters
Change civetweb_num_thread default to 100

Add capability to override number of pgs for
rgw pools.

Add ceph.conf vars to enable default bucket
object quota at users choosing into the ceph.conf.j2
template

Resolves: rhbz#1437173
Resolves: rhbz#1391500

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-04-25 16:01:03 -04:00
Andrew Schoen ae351efeca Merge pull request #1466 from ceph/upgrade-calamari
mon: upgrade calamari when running rolling_update playbook
2017-04-24 07:34:41 -07:00
Alberto Murillo 5218df5ef3 Add clearlinux to supported platforms
Signed-off-by: Alberto Murillo Silva <alberto.murillo.silva@intel.com>
2017-04-24 09:34:23 -05:00
Andrew Schoen b28424334a Merge pull request #1455 from ceph/restart_daemons
Common: Restore check_socket
2017-04-24 06:54:07 -07:00
Guillaume Abrioux 800b439667 Common: Restore check_socket
Restore the check_socket that was removed by `5bec62b`.
This commit also improves the logging in `restart_*_daemon.sh` scripts

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-04-24 12:31:49 +02:00
Sébastien Han 84d96be197 mon: upgrade calamari when running rolling_update playbook
Prior to this change, ansible was only checking for the existence of the
package, now if upgrade_ceph_packages is true this means we are
performing an upgrade.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1442016

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-04-24 11:05:13 +02:00
Sébastien Han 58e7d39bcc Merge pull request #1461 from ceph/wip-remove-osd-directory-scenario
remove osd directory scenario
2017-04-24 10:54:54 +02:00
Gregory Meno eb0c83db5f remove osd directory scenario
Proof-of-concept clusters or actual production clusters will never want to use this. We also do not test it anywhere for this same reason.

Signed-off-by: Gregory Meno <gmeno@redhat.com>
2017-04-21 15:50:32 -07:00
John Spray 3a0d03656f mon: update mgr key capabilities
This is to allow ceph-mgr daemons to remote control
osd and mds daemons with MCommand messages.

Fixes: http://tracker.ceph.com/issues/19713
Signed-off-by: John Spray <john.spray@redhat.com>
2017-04-21 12:15:25 +01:00
John McEleney f1388dc2c2 Apparmor on Ubuntu Xenial will not permit containers to mount devices, even with CAP SYS_ADMIN. 2017-04-19 19:22:02 +01:00
Andrew Schoen be3e0d772c ceph-common: install ceph-mgr for rhcs on debian
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-04-17 08:40:24 -05:00
Andrew Schoen 30098f4e34 ceph-common: install ceph-mgr for rhcs on rhel
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-04-17 08:37:34 -05:00
Sébastien Han dfd8f4d96e test: add mgr section to the host inventory file
Without this, we don't test the mgr role so we need to add it.

Co-Authored-by: Guillaume Abrioux <gabrioux@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-04-15 00:16:10 +02:00
Sébastien Han 804aa529bb Merge pull request #1442 from ceph/fix-hung-command
ceph-mon: change command to see if rbd exists
2017-04-13 15:21:26 +02:00
Andrew Schoen ebed16e9a5 Merge pull request #1439 from ceph/rhcs-tests
tests: fix ceph_rhcs setup
2017-04-13 07:23:07 -05:00
Andrew Schoen 9ce0c2808e ceph-mon: change command to see if rbd exists
The previous command was hanging, see this issue:

https://github.com/ceph/ceph-ansible/issues/1440

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-04-12 15:07:10 -05:00
Andrew Schoen 37d38b122b ceph-common: is ceph_rchs is True do not include install_on_redhat.yml
That task includes logic for upstream installs that we do not want to
run when deploying RHCS.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-04-12 07:55:03 -05:00
Matthew Vernon e3a6429e73 Only include cluster.conf and osd.conf when building ceph.conf
This is the same fix as bc846b7da6
applied to the other part of the code-base that builds ceph.conf (I'd
missed that 349b9ab3e7 had duplicated
this code).

Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
2017-04-11 14:33:21 +01:00
Matthew Vernon bc846b7da6 Only assemble {{ cluster }}.conf and osd.conf
Ansible's assemble module by default will put all files in the src
directory together into dest. We only want to put {{ cluster }}.conf
and osd.conf together, not anything that might have found its way into
/etc/ceph/ceph.d (e.g. files left by the sysadmin taking backups
before an ansible run). So specify a regexp that matches only those
two files.

Signed-off-by: Matthew Vernon <mv3@sanger.ac.uk>
2017-04-11 13:27:19 +01:00
Sébastien Han 05331a2634 mon: fix with_items being evaluated before when
Ansible evaluates the 'with_items' before the 'when' so if the inventory
does not have the group declared it'll fail. To fix this, we set an
empty array to make the with_items happy and then evaluate with the
'when'.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-04-11 12:10:55 +02:00
Sébastien Han 186a392656 Merge pull request #1425 from ceph/bump-kraken
common: bump ceph version to kraken
2017-04-10 19:03:39 +02:00
Sébastien Han e48c31c671 common: bump ceph version to kraken
Kraken has been out for a couple of weeks now and the CI can test both
Kraken and Jewel.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-04-10 18:05:19 +02:00
Sébastien Han d113cf854c Merge pull request #1377 from ceph/ceph-mgr
mgr: add new role for ceph-mgr
2017-04-10 17:50:24 +02:00
Sébastien Han 12b2aa2e55 docker: fix monitors name
Prior to this change we were deploying a monitor using tis fqdn name but
we were checking its state and performing actions on it using its
shortname.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-04-10 15:13:12 +02:00
Sébastien Han 2aa5286544 mgr: add new role for ceph-mgr
The Ceph Manager daemon (ceph-mgr) runs alongside monitor daemons, to
provide additional monitoring and interfaces to external monitoring and
management systems.

Only works as of the Kraken release.

Co-Authored-By: Guillaume Abrioux <gabrioux@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-04-10 15:13:09 +02:00
Konstantin Shalygin 334d4cb885
ceph-common: disable_transparent_hugepage now persist 2017-04-08 13:06:18 +07:00
Sébastien Han e2c8f1a404 Merge pull request #1412 from guits/fix_handlers
Common: Fix handlers that are not properly triggered.
2017-04-07 02:36:27 +02:00
WingkaiHo 6ad8288c91 enable ceph-mon.target service after monitor start
ceph-create-keys unit file was removed here:

* 8bcb4646b6
* dc5fe8d415

As a consequence the systemctl preset command now fails to run since the
unit does not exist anymore. Due to the redirection in /dev/null we
don't know what's happening.

Ultimately the mon unit doesn't get enabled and the mon service won't
start after reboot.
Removing the old/non-existent unit makes the command succeed now.

ceph fix: https://github.com/ceph/ceph/pull/14226

Signed-off-by: WingkaiHo <sanguosfiang@163.com>
Co-Authored-By: Sébastien Han <seb@redhat.com>
2017-04-06 16:58:32 +02:00
Guillaume Abrioux 5bec62ba7c Common: Fix handlers that are not properly triggered.
Until now, only the first task were executed.
The idea here is to use `listen` statement to be able to notify multiple
handler and regroup all of them in `./handlers/main.yml` as notifying an
included handler task is not possible.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-04-06 16:19:58 +02:00
Daniel Horak ce06dc1460 osd autodiscovery mode: fix holders detection
Small fix for (probably copy&paste) issue from 42ffe6301.

Signed-off-by: Daniel Horak <dahorak@redhat.com>
2017-04-06 09:11:32 +02:00
Sébastien Han 42ffe63017 osd: autodiscovery mode, use holders to detect device
As reported in
https://github.com/ceph/ceph-ansible/issues/1403 when devices are held
by lvm and `osd_auto_discovery` is set to true, it's not enough to check
for a partition count = 0 since Ansible does not report.
This patch also looks for 'holders' which in a case of lvm corresponds
to the name of the pv. Now we also look for holders = 0.

Fixes: #1403

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-04-04 10:37:14 +02:00
Sébastien Han c37aaa41f4 playbook: homogenize the way list osd ids
Problem: too many different commands to do the same thing. The 'cut'
command on infrastructure-playbooks/purge-cluster.yml was also wrong.
This sed command from osixia in ceph-docker
https://github.com/ceph/ceph-docker/pull/580/ addresses all the
scenarios.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-03-30 11:51:38 +02:00
Guillaume Abrioux 4e62627202 Common: Do not install ntp when ntp_service_enabled is false
ntp is still installed even if ntp_service_enabled is set to false.
That could be a problem if the time synchronization is managed by
something else than ceph-ansible or if you want to use different NTP
implementation as suggested in #1354.

Fixes: #1354

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
Signed-off-by: Guits <gabrioux@redhat.com>
2017-03-28 19:51:34 +02:00
Sébastien Han 6a9a32de82 Merge pull request #1391 from k0ste/newfeature
Resolve issues when groups names not in default value.
2017-03-27 18:48:33 +02:00
Sébastien Han 6e3c0971a9 Merge pull request #1396 from guits/fix_1374
BugFix: Fix #1374
2017-03-27 18:13:51 +02:00
Konstantin Shalygin 1662976fc0
Resolve issues when groups names not in default value. 2017-03-27 21:44:30 +07:00
Guillaume Abrioux a99e04a9b7 BugFix: Fix #1374
If a group of hosts is empty, (for instance 'mdss', in case of a
deployment without any mds node), the playbook will fails when trying
to restart service with `"'dict object' has no attribute u'XXX'"` error.

The idea here is to force the `with_items` statements in all included handler tasks
to get at least an empty array.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-03-27 16:24:07 +02:00
Concubidated 0454d540b0 ceph-common: update sysctl file location
systctl tuning should be in the sysctl.d directory. This creates
a seperation from what values were set specific to ceph, and what
values were set by the operator.

Signed-off-by: Tyler Brekke <tbrekke@redhat.com>
2017-03-24 12:59:34 -07:00
Guillaume Abrioux 589d6812ca ceph-docker: fix bootstrap directories permissions
Make bootstrap directories permissions work for both RedHat and Debian
os families.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
Fix: #1338
2017-03-22 11:18:56 +01:00
Sébastien Han 350d2f88c7 Merge pull request #1381 from joke-lee/master
the $? of pgrep ceph-rest-api command return is 1 when ceph-rest-api is already run
2017-03-21 09:53:01 +01:00
Sébastien Han 833d16c068 Merge pull request #1379 from D3N14L/fix_1299
Use ansible uri module instead of shell module with curl
2017-03-21 09:43:10 +01:00
Sébastien Han 839e0fdf4d Merge pull request #1376 from czunker/fix_osd_crush_location
Fix osd_crush_location to prevent systemd error message
2017-03-21 09:41:10 +01:00
Sébastien Han 28042eb032 Merge pull request #1365 from czunker/fix_osd_fragments
Make ceph-common aware of osd config fragments
2017-03-21 09:40:25 +01:00
yuliyang aa90fe6417 the $? of pgrep ceph-rest-api command return is 1 when ceph-rest-api already run. use ps instead 2017-03-18 21:41:53 +08:00
Christian Zunker 09646041ee Fix osd_crush_location to prevent systemd error message
With ' in osd_crush_location, systemd will show this error:
ceph-osd-prestart.sh[2931]: Invalid command:  invalid chars ' in 'root=

Signed-off-by: Christian Zunker <christian.zunker@codecentric.de>
2017-03-17 07:26:40 +01:00
Daniel Marks 2ed94d914c Use ansible uri module instead of shell module with curl
This fixes issue #1299. According to @ktdreyer s comment in the ticket,
he fixed the web server config so also older (non-SNI) python clients
can use the uri module here.
2017-03-16 23:16:30 +01:00
Andrew Schoen ec5ce81884 ceph-mon: always call ceph-create-keys
After the jewel release the mon startup does not generate keys, but it's
still harmless to call ceph-create-keys with jewel because this task has
a 'creates' argument that will cause it not to run if the keys already
exist.

Removing this when condition also allows the downstream CI tests to
install kraken or luminous without resetting ceph_stable_release, which does not
pertain to rhcs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-03-16 16:31:25 -05:00
Sébastien Han 777913e9da docker: change centos extra repo var name
This is not only for monitors, but also mds, rgw and rbd mirror so
making the var name more generic:
ceph_docker_enable_centos_extra_repo

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-03-16 15:16:34 +01:00
Sébastien Han 0609786ce6 Merge pull request #1331 from guits/enh_1321
ceph-mon: create openstack pools and keys even for containerized deployments
2017-03-16 13:28:45 +01:00
Sébastien Han 8b463d0bb8 Merge pull request #1373 from ceph/docker-mon-wait
mon: increase wait for monitor socket timeout
2017-03-16 12:03:15 +01:00
Sébastien Han 60a56b5d4a mon: increase wait for monitor socket timeout
Sometimes the socket appears during the 5th attempt and sometimes not so
increasing the timeout a little bit.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-03-16 11:16:49 +01:00
Guillaume Abrioux ea68fbaaae ceph-mon: Create openstack pools and keys even for containerized deployments
Add the possibility to create openstack pools and keys even for containerized deployments

Fix: #1321
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-03-16 10:53:53 +01:00
Sébastien Han 8c6079e23e rbd-mirror: fix spelling
It is RBD and NOT RDB.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-03-16 10:18:31 +01:00
Christian Zunker 349b9ab3e7 Make ceph-common aware off osd config fragments
This removes the implicit order requirement when using OSD fragments.
When you use OSD fragments and ceph-osd role is not the last one,
the fragments get removed from ceph.conf by ceph-common.

It is not nice to have this code at two locations, but this is
necessary to prevent problems, when ceph-osd is the last role as
ceph-common gets executed before ceph-osd.

This could be prevented when ceph-common would be explicitly called
at the end of the playbook.

Signed-off-by: Christian Zunker <christian.zunker@codecentric.de>
2017-03-16 09:50:08 +01:00
Sébastien Han 38ab6de602 Merge pull request #1336 from WingkaiHo/master
Load a variable file for devices partition
2017-03-15 11:55:26 +01:00
Sébastien Han 8567609b7e kv scenario: add KV_PORT option
This option was missing for rrgw, mds, rbd mirror and nfs making these
daemon impossible to run on a kv deployment with containers.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-03-14 22:46:09 +01:00
Sébastien Han 8320c14191 Merge pull request #1317 from ibotty/harmonize-docker-names
harmonize docker names
2017-03-14 18:20:20 +01:00
Sébastien Han 2fcffafb93 Merge pull request #1358 from ceph/nfs-ganesha-fsal-deb
ceph-common: install nfs-ganesha FSALs on Debian
2017-03-14 16:03:14 +01:00
Ken Dreyer bf57efaf3c ceph-common: install nfs-ganesha FSALs on Debian
Prior to this change, ceph-ansible would install the main NFS Ganesha
server daemon on Ubuntu, but it would skip the Ceph FSALs.

Running "apt-get install nfs-ganesha" will only install the main NFS Ganesha
server. It does *not* pull in the RGW FSAL
(/usr/lib/x86_64-linux-gnu/ganesha/libfsalrgw.so)

Running "apt-get install nfs-ganesha-fsal" will install the RGW FSAL as
well as the main NFS Ganesha server package.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
2017-03-14 08:37:45 -06:00
Boris Ranto 3c6a7a60eb calamari: Add debugging option
This patch introduces calamari_debug option which will turn on debugging
for calamari before initializing and running it.

Signed-off-by: Boris Ranto <branto@redhat.com>
2017-03-14 10:53:05 +01:00
Sébastien Han c9e333f750 Merge pull request #1347 from ceph/no-vfs-cache-pressure
avoid setting vfs_cache_pressure
2017-03-08 18:02:06 +01:00
Sébastien Han b55a0702c2 Merge pull request #1335 from batrick/mon-fs-cmd-1mon
send fs commands to one mon
2017-03-07 16:45:16 +01:00
Sébastien Han 703e82f277 Merge pull request #1339 from ceph/rgw-remove-sudo
rgw: remove sudoers file
2017-03-07 16:44:28 +01:00
Ken Dreyer a77cd4be5e avoid setting vfs_cache_pressure
From Josh Durgin, "I'd recommend not setting vfs_cache_pressure in
ceph-ansible. The syncfs issue is still there, and has caused real
problems in the past, whereas there hasn't been good data showing lower
vfs_cache_pressure is very helpful - the only cases I'm aware of have
shown it makes little difference to performance."

https://bugzilla.redhat.com/show_bug.cgi?id=1395451
2017-03-03 11:20:05 -07:00
Ken Dreyer f9a317ef30 check_system: improve RHCS support message and version comparison
"red hat storage" -> "red hat ceph storage"

"RHEL 7.1" -> "RHEL 7"

and make the version number check verify the very latest version

https://bugzilla.redhat.com/show_bug.cgi?id=1354059
2017-03-03 09:57:16 -07:00
Guillaume Abrioux 66b59ea9c6 docker: Fix #1303
Install package from official repos rather than pip when using RHEL.

This commit fix https://bugzilla.redhat.com/show_bug.cgi?id=1420855

Also this commit Refact all `roles/ceph-*/tasks/docker/pre_requisite.yml`
to avoid a lot of duplicated code.

Fix: #1303
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-03-03 10:49:13 +01:00
Sébastien Han 88c6296fa2 rgw: remove sudoers file
This was needed for Hammer and older version, not needed anymore since
we have a 'ceph' user to run ceph processes.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-03-02 14:59:21 +01:00
WingKai Ho 029eb2a6d3 Update ceph_keys.yml 2017-03-02 14:09:08 +08:00
WingKai Ho 62892fbdf6 Update ceph_keys.yml 2017-03-02 14:07:52 +08:00
Patrick Donnelly 56d8367339
send fs commands to one mon
Add-on to #1329.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-03-01 11:05:56 -05:00
WingKai Ho 997fe5b198 Update ceph_keys.yml 2017-03-01 17:39:53 +08:00
Guillaume Abrioux e00b745a6c ceph-mon: Check if ceph filesystem exists
Check if ceph filesystem already exists before creating it.
If the ceph filesystem doesn't exist, execute the task only on one node.

Fix: #1314
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-02-24 09:11:52 +01:00
Tobias Florek 931027e6f7 harmonize docker names
Created containers now are named more or less in the form of

    <ansible role>-<ansible_hostname>
2017-02-23 09:15:05 +01:00
Sébastien Han 458a9ad5c3 mon: docker, ability to enable centos extra repo
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-21 15:56:09 -05:00
Sébastien Han 72b17d2480 docker: osd, clarify variable usage for scenarii
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-21 15:56:09 -05:00
Sébastien Han b91d227b99 docker: make ceph docker osd script path
Since distro will not allow /usr/share to be writable (e.g: atomic) so
we let the operator decide where to put that script.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-21 15:56:09 -05:00
Sébastien Han 73cf0378c2 docker: osd, do not use priviledged container anymore
Oh yeah! This patch adds more fine grained control on how we run the
activation osd container. We now use --device to give a read, write and
mknodaccess to a specific device to be consumed by Ceph. We also use
SYS_ADMIN cap to allow mount operations, ceph-disk needs to temporary
mount the osd data directory during the activation sequence.

This patch also enables the support of dedicated journal devices when
deploying ceph-docker with ceph-ansible.

Depends on https://github.com/ceph/ceph-docker/pull/478

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-21 15:54:36 -05:00
Sébastien Han dd548c6034 docker: osd, do not skip on failure
If the systemd unit file can not be generated we should fail, same for
systemd enable and reload.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-21 15:54:36 -05:00
Andrew Schoen 6cf842eb39 ceph-common: remove infernalis comment on radosgw_civetweb_port
As of Infernalis, the Ceph daemons run as an unprivileged "ceph" UID,
and this is by design.

Commit f19b765 altered the default
civetweb port from 80 to 8080 with a comment in the commit log about
"until this gets solved"

Remove the comment about permissions on Infernalis, because this is
always going to be the case on the Ceph versions we support, and it
is just confusing.

If users want to expose civetweb to s3 clients using privileged TCP
ports, they can redirect traffic with iptables, or use a reverse proxy
application like HAproxy.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-02-21 12:35:00 -06:00
Andrew Schoen 1579642e3f ceph-common: do not get current fsid when performing a rolling_update
This avoids a situation where during a rolling_update we try to talk to
a mon to get the fsid and if that mon is down the playbook hangs
indefinitely.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-02-21 12:31:26 -06:00
Andrew Schoen 920bd9cf2d ceph-common: use yum_repository when adding the ceph_stable repo
This gives us more flexibility than installing the ceph-release package
as we can easily use different mirrors. Also, I noticed an issue when
upgrading from jewel -> kraken as the ceph-release package for those
releases both have the same version number and yum doesn't know to
update anything.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-02-21 12:31:26 -06:00
Sébastien Han effefe91d5 common: add doc for rgw on ipv6
See: https://bugzilla.redhat.com/show_bug.cgi?id=1424799

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-21 12:00:37 -05:00
WingKai Ho 421d1a2853 Update ceph_keys.yml
jewel version need to build the {{ cluster }}.client.admin.keyring exists key
2017-02-21 09:49:52 +08:00
Sébastien Han 7c79e09424 common: fix "disable transparent hugepage"
To configure kernel the task is using "command" module which is not
respect operator ">". So this task just print to "stdout": "never >
/sys/kernel/mm/transparent_hugepage/enabled"

fix: #1319

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-20 17:07:53 -05:00
Shengjing Zhu 32923fd217 fix grep match pattern for osd ids
Some playbooks use [0-9]*, others use \d+$
The latter is more correct since cluster name may contain numbers.

Signed-off-by: Shengjing Zhu <zsj950618@gmail.com>
2017-02-20 16:35:56 +08:00
Sébastien Han cc769464d0 docker: homogenise systemd file location
So unit files were stored in /var/lib/ceph some where in
/etc/systemd/system. Now they are all under /etc/systemd/system.

closes: #1296

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-16 18:31:32 +01:00
Sébastien Han 978093d843 Merge pull request #1304 from guits/fix_1300
Skip facts_mon_fsid.yml if cephx is disabled
2017-02-11 13:14:07 +01:00
Andrew Schoen c5a5658e09 Merge pull request #1301 from guits/fix_1294
Check if ceph_conf_overrides.global is defined before calling it
2017-02-10 10:20:14 -06:00
Andrew Schoen 94ac749918 Merge pull request #1302 from ceph/docker-pull
docker: use a better method to pull images
2017-02-10 10:18:37 -06:00
Guillaume Abrioux 11bae8b252 Skip facts_mon_fsid.yml if cephx is disabled
If cephx is disabled it is not necessary to include `facts_mon_fsid.yml`
in `roles/ceph-common/tasks/facts.yml`.

Fix: #1300
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-02-10 17:04:32 +01:00
Guillaume Abrioux e63631a4ab bugfix: Add missing conditions for kraken release
76ddcbc introduced an issue by removing some needed conditions on tasks
that need to be run only on release >= kraken.
2017-02-10 15:14:54 +01:00
Guillaume Abrioux 4295d427b9 Check if ceph_conf_overrides.global is defined before calling it
Expand the fix in #1291 to all the playbook in order to get a full coverage.

Fix: #1294
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-02-09 20:28:58 +01:00
Sébastien Han c2f1dca823 docker: use a better method to pull images
We changed the way we declare image.
Prior to this patch we must have a "user/image:tag"
format, which is incompatible with non docker-hub registry where you
usually don't have a "user". On the docker hub a "user" is also
identified as a namespace, so for Ceph the user was "ceph".

Variables have been simplified with only:

* ceph_docker_image
* ceph_docker_image_tag

1. For docker hub images: ceph_docker_name: "ceph/daemon" will give
you the 'daemon' image of the 'ceph' user.

2. For non docker hub images: ceph_docker_name: "daemon" will simply
give you the "daemon" image.

Infrastructure playbooks have been modified as well.
The file group_vars/all.docker.yml.sample has been removed as well.
It is hard to maintain since we have to generate it manually. If
you want to configure specific variables for a specific daemon simply
edit group_vars/$DAEMON.yml

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1420207
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-09 17:57:18 +01:00
Guits df162a61b1 ceph-mon: Fix bug #1242
We shouldn't test directly the value of
`ceph_conf_overrides.global.osd_pool_default_pg_num` because this can
cause the playbook to fail if the key `global` is not present in
`ceph_conf_overrides`. Therefore we have to use the facts that have been
defined earlier.

Fix: #1242

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-02-08 10:46:46 +01:00
Shengjing Zhu 7e10b0325b add ceph_mirror variable
Closes: #1282

Signed-off-by: Shengjing Zhu <zsj950618@gmail.com>
2017-02-08 13:13:31 +08:00
Andrew Schoen 9580fd974a ceph-common: set ms bind ipv6 = true in ceph.conf when using ipv6
This fixes an issue with parsing the ceph.conf file when ip_version is set to
ipv6.

See: https://bugzilla.redhat.com/show_bug.cgi?id=1419814

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-02-07 12:34:29 -06:00
Sébastien Han 55abf69481 Merge pull request #1267 from ceph/container-systemd
Container systemd
2017-02-03 14:02:53 +01:00
Sébastien Han 2ed93875a9 docker: fix monitor addition on kraken
Probably not the best fix but useful for testing. We will remove this
once we find a better workaround.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-03 11:35:54 +01:00
Sébastien Han 064c57babb Merge pull request #1264 from ceph/clarify-osd-logs
osd: clarify osd scenario prepare sequence
2017-02-02 14:03:12 +01:00
Sébastien Han c4331d87b3 common: ability to disable handler osd health check
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-02 11:22:55 +01:00
Sébastien Han 6d5afdfb99 docker: fix restapi key creation with cluster name
Add support for key creation when the cluster name is different than
'ceph'.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-02 09:32:12 +01:00
Sébastien Han cb499a689b docker: fix mkdir path for all the distros
On ubntu systems mkdir is in /bin where on atomic it is /usr/bin/.
We use the shell built-in function "command" to find its right location.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-01 17:19:07 +01:00
Sébastien Han f446b1c82f docker: fail if systemd is not present
Since we treat containers as services using systemd, systemd is an
absolute requirement.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-01 17:18:17 +01:00
Sébastien Han 40709c8336 docker: use systemd to manage container
Since we now only support systemd has an init system we can finally
treat containers as processes using systemd and this for all the
distros.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-01 17:12:46 +01:00
Andrew Schoen 873c044c53 ceph-common: remove absolute path to handler include files
This breaks if the roles don't live in the same place as the playbook.

See: https://github.com/ceph/ceph-ansible/issues/1265

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-02-01 09:22:26 -06:00
Sébastien Han 5578b9bc7b osd: clarify osd scenario prepare sequence
we now use the name of the scenario in the prepare task.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-02-01 13:59:35 +01:00
Sébastien Han 40a2df5bbf common: serialise host restart
This commits allows us to restart Ceph daemon machine by machine instead
of restarting all the daemons in a single shot.

Rework the structure of the handler for clarity as well.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-01-31 17:39:51 +01:00
Guillaume Abrioux 76ddcbc271 Remove support of releases prior to Jewel.
According to #1216, we need to simply the code by removing the
support of anything before Jewel.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-01-31 11:00:54 +01:00
Sébastien Han f888cc4e06 Merge pull request #1254 from ceph/osd-dir-exist
Osd dir exist and purge fixes
2017-01-31 08:46:22 +01:00
Andrew Schoen 7dc9c4b5b6 ceph-common: update apt cache in it's own task
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-01-30 17:15:08 -06:00
Sébastien Han 837ca9eaff Merge pull request #1257 from clwells/rgw-restart-handler
Use ansible_hostname instead of ansible_host in handler
2017-01-30 14:38:35 +01:00
Sébastien Han 1149825f8f common: create ceph initial directories
Some users purge their environments and leave it in a non-optimal state.
e.g: packages are still installed but /etc/ceph and /var/lib/ceph don't
exist anymore. This will result in multiple failures across the play,
sometimes hard to detect. Populating these directories "just in case"
should help us solving these problems.

Closes: #1253
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-01-30 14:31:56 +01:00
Sébastien Han 6f53774ee9 osd: make sure osd directory exists
Sometimes users for testing, tend to delete the whole /var/lib/ceph and
then run ansible again, OSD will never come up if we do not create their
directory.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-01-30 14:31:56 +01:00
Chris Wells 84c322550a
Use hostname instead of host (host can be an IP) and hostname matches the default nomenclature in ceph.conf 2017-01-28 13:09:03 -05:00
Chris Wells 8b2dace29b
Using ini_file with ansible_hostname to ensure each INI block gets the rgw_zone setting in a multi-RGW setup. Also, ansible_hostname better matches what ceph-common does for the actual hostname (ansible_host != ansible_hostname under all conditions). 2017-01-28 12:30:27 -05:00
Sébastien Han e35070f6ce mon: make sure osd_pool_default_size is honoured
This patch makes sure we set the proper pool size on the rbd pool.
Usually during bootstrap the rbd pool size is not honoured so we need to
add this workaround.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-01-25 22:53:17 +01:00
Andrew Schoen 03cb803bd1 Adds ip_version configuration option
This allows the user to set ip_version to either ipv4 or ipv6. This
resolves a bug where monitor_address is set to an ipv6 address, but the
template fails to render because it's hardcoded to look for an 'ipv4'
key in the ansible facts.

See: https://bugzilla.redhat.com/show_bug.cgi?id=1416010

Signed-off-by: Andrew Schoen <aschoen@redhat.com>

Resolves: bz#1416010
2017-01-24 09:48:55 -06:00
tynorth-cisco 6400989cf2 add unique filter to openstack pool names
could have scenario where different openstack components would
use the same pool, but the logic would create the same pool
more than once

add unique filter to account for this
2017-01-23 11:58:28 -08:00
Andrew Schoen c4161df38b Merge pull request #1146 from ceph/client-pool
mon: pool creation and pgs
2017-01-23 10:17:21 -06:00
Logan V cd50cd8f18 RGW: Add resolve cname configuration parameter
Adds a switch to enable bucket resolution using DNS CNAME provided
by the client's HTTP headers.
2017-01-19 11:15:18 -06:00
Logan V 4941de838c RGW: Add static website hosting configuration 2017-01-19 11:15:18 -06:00
Logan V 6f3a98919c RGW: Allow configurable rgw frontends setting
Allow for more operator flexibility in the `rgw frontends` setting
while maintaining backwards compatibility with the old vars. This
allows an operator to, for example, use the civetweb settings for
implementing SSL ports.

For available civetweb configuration parameters, see:
https://github.com/civetweb/civetweb/blob/master/docs/UserManual.md
2017-01-19 11:15:18 -06:00
Logan V 8edc2663d9 RGW: Add support for usage log configuration
Adds support for configuring the RadosGW usage log described at
http://docs.ceph.com/docs/jewel/man/8/radosgw/#usage-logging.
2017-01-19 11:15:18 -06:00
Andrew Schoen e2a18af5b3 Merge pull request #1227 from Logan2211/resync-group-vars-sample
Remove libcephfs1 from group_vars sample
2017-01-19 09:57:54 -06:00
Sébastien Han 90648e7518 mon: fix mds pool creation
It is not enough to check for the mds to exists, it actually always does
because we declare the variable. So we need to make sure that there is a
mds host.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-01-19 14:49:09 +01:00
Sébastien Han ddac3a1fb5 mon: pool creation and pgs
Since we introduced config_overrides we removed a lot of options from
the default template. In some cases, like mds pool, openstack pools etc
we need to know the amount of PGs required. The idea here is to skip the
task if ceph_conf_overrides.global.osd_pool_default_pg_num is not define
in your `group_vars/all.yml`.

Closes: #1145

Signed-off-by: Sébastien Han <seb@redhat.com>
Co-Authored-By: Guillaume Abrioux <gabrioux@redhat.com>
2017-01-19 11:30:39 +01:00
Andrew Schoen 2e8cf582f2 ceph-docker-common: adds a README file
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-01-18 11:06:25 -06:00
Andrew Schoen 88a2ddb240 ceph-docker-common: include a meta/main.yml so we can use galaxy
This role needs a meta/main.yml before we can upload it to ansible
galaxy.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-01-18 10:46:48 -06:00
Sébastien Han c6728ca21d Merge pull request #1229 from ceph/docker-fixes
adds a ceph-docker-common role and enables custom registry usage
2017-01-18 14:54:12 +01:00
Musee Ullah 7636d09f92
mon: run ceph-create-keys after startup for kraken release 2017-01-17 22:51:39 +09:00
Andrew Schoen 0c55a35963 ceph-osd: use ceph_docker_registry when preparing OSDs
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-01-16 11:39:13 -06:00
Andrew Schoen 655b8449ae use ceph_docker_registry when starting containers
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-01-16 11:17:41 -06:00
Andrew Schoen 3713824b79 ceph-docker-common: add symlink to ceph.ceph-docker-common
This allows for the role to be used with ansible-galaxy and to fix the
include in all the meta/main.yml files in the roles.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-01-16 10:42:42 -06:00
Andrew Schoen 9449dbf083 use ceph_docker_registry in all the roles instead of docker.io
This allows for ceph-ansible to use other docker registries.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-01-16 10:42:42 -06:00
Andrew Schoen 25277587fa ceph-common: include ceph_docker_registry when fetching the image
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-01-16 09:51:20 -06:00
Andrew Schoen c07b7ddbaa use ceph-docker-common in roles that support docker deployments
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-01-16 09:30:53 -06:00
Andrew Schoen f770780dda ceph-docker-common: a new role to share things common to docker
We can use this to share common variables and tasks needed for every
containerized deployment.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-01-16 09:28:40 -06:00
Logan V c78a3707f1 Remove libcephfs1 from group_vars sample
The libcephfs1 package was removed from ceph-common in
cb1c06901e, however it was not synced
to group_vars/all.yml.sample using the `generate_group_vars_sample.sh`
script. This fixes up the comment formatting in the ceph-common
defaults and brings the group_vars sample back into sync.
2017-01-16 09:15:57 -06:00
Daniel Marks fefaa8ed13 Set empty list as default for osd_directories
As described in issue #1224 leaving this variable undefined may
cause a problem during execution of the ceph-osd role.
2017-01-13 15:27:16 +01:00
Sébastien Han 6ebea0cf42 Merge pull request #1208 from clwells/multisite-defined-endpoint
[rtr] Granular Multi-site PULL Variables
2017-01-10 16:12:04 -05:00
Chris Wells d609dca7a6
Updated pull[port|proto] to pull_[port|proto] and regenerated base group vars 2017-01-07 09:33:24 -05:00
Ken Dreyer 63e5b5c406 ceph-common: always include release.yml
Prior to this change, a playbook run with '--tags' or '--skip-tags'
would fail, because the ceph-common role would not include the
release.yml task, and this file defines critical things like
ceph_release.

Thanks Andrew Schoen <aschoen@redhat.com> for help with the fix.
2017-01-05 14:40:39 -07:00
Sébastien Han cb1c06901e Merge pull request #1171 from cbodley/wip-libcephfs2
bump package version to libcephfs2
2017-01-03 10:48:56 +01:00
Sébastien Han 676d8812f9 Merge pull request #1212 from ceph/cluster-name-info
common: document "cluster" variable usage
2017-01-03 10:18:41 +01:00
Sébastien Han f1308b5f71 Merge pull request #1213 from ceph/cephx-initial-mon
mon: only put mon initial key in mon kv when cephx is enabled
2017-01-02 13:43:22 +01:00
Sébastien Han 468afa2d5b mon: only put mon initial key in mon kv when cephx is enabled
Task put initial mon keyring in mon kv store from
ceph-mon/tasks/ceph_keys.yml is failing when cephx is disabled. The root
cause is that variable monitor_keyring is not populated by any task from
deploy_monitors.yml.

Fixes: #1211

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-01-02 11:16:03 +01:00
Sébastien Han b76d0aceca common: document "cluster" variable usage
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1408848

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-01-02 09:20:43 +01:00
Shengjing Zhu 93a8b69a57 fix check nmapexist on debian
command is a shell-buitin, so `command -v nmap` must use shell module

Signed-off-by: Shengjing Zhu <zsj950618@gmail.com>
2016-12-28 16:21:23 +08:00
Chris Wells 5f37ae9d1f
Allowing granular control of the pull host/port/protocol for installs where SSL/443 is used and direct civetweb access isn't necessarily available. 2016-12-25 21:54:13 -05:00
Sébastien Han d44927de03 common: rename rh_storage to rhcs to match product name
Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-20 13:49:11 +01:00
Sébastien Han 775d61ed09 common: enable tool repo for mds install of rhcs
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1405985

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-20 13:42:24 +01:00
tynorth-cisco 704b49ba33 use monitor name, not host ansible host in ceph.conf
in ceph-common there is a fact check for monitor name
that will set it to ansible_host or ansible_fqdn
2016-12-19 12:33:38 -08:00
Sébastien Han fd4fb8d5b2 Merge pull request #1199 from Logan2211/fix-mon-fsid-block
Workaround for broken 'block' syntax
2016-12-19 11:30:58 +01:00
Andrew Schoen 99d66e09d9 Merge pull request #1153 from ceph/cluster-name-test
test: add cluster name support test scenario
2016-12-16 13:10:52 -06:00
Sébastien Han 2d8ac4a586 docker: only use systemd to manage containers
Prior to this patch we had several ways to runs containers, we could use
ansible's docker module on some distro and on containers distros we were
using systemd. We strongly believe threating containers as services with
systemd is the right approach so this patch generalizes to all the
distros. These days most of the distros are running systemd so it's fair
assumption.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-16 19:37:05 +01:00
Logan V ffc89ee95f Workaround for broken 'block' syntax
The block syntax in ansible is broken on 2.1 so we work around the
issue introduced in 471be5e by using a task file include.
2016-12-16 09:24:48 -06:00
Sébastien Han ce7431a227 docker: add support for cluster name
We need to honour the cluster name that was chosen by ceph-ansible and
pass it to ceph-docker.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-16 14:31:21 +01:00
Sébastien Han dcd94f6c52 Merge pull request #1179 from Logan2211/rgw-keystone-v3
RGW/Keystone integration improvements
2016-12-16 13:52:52 +01:00
Sébastien Han 608b008a95 mon: add the initial mon keyring to the mon store
Once we have our first monitor up and running we need to add it to the
monitor store as a safety measure. Just in case the local file gets
deleted and you need to add a new monitor. Now you can retrieve this key
like this:

ceph config-key get initial_mon_keyring > initial_mon_keyring.txt

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-16 11:54:02 +01:00
Sébastien Han 6c71cf5b10 common: do not become root on local task
There is no need to become root on local_action. This will event trigger
an error on some systems as it will try to run a sudo command. If the
current user does not have passwordless sudo, Ansible will fail. Anyway
using the current user is perfectly fine and no elevation privilege is
needed.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-16 11:42:07 +01:00
Logan V cccdb2ab13 Add support for Keystone v3 API
The Keystone v2 APIs are deprecated and scheduled to be removed in
Q release of Openstack. This adds support for configuring RGW to
use the current Keystone v3 API.
2016-12-15 17:17:41 -06:00
Logan V aa0bfaa89a Add a switch to disable nss PKI database initialization
The PKI keys are used to decrypt the Keystone revocation list when
PKI tokens are used. When UUID or Fernet token providers are used in
Keystone, PKI certs may not exist, so we now accommodate this scenario
by allowing the operator to disable the PKI tasks.
2016-12-15 17:17:41 -06:00
Logan V 12f8b5c38e Add support for Keystone user authentication with RGW
Jewel added support for user/pass authentication with Keystone,
allowing deployers to disable Keystone admin token as required
for production deployments.

This implements configuration for the new RGW Keystone user/pass
authentication feature added in Jewel.

See docs here: http://docs.ceph.com/docs/master/radosgw/keystone/
2016-12-15 17:17:37 -06:00
Sébastien Han faabfdcefe Merge pull request #1178 from zhsj/dev-partition
Add prepare osd with partition devices in raw_multi_journal
2016-12-15 22:50:23 +01:00
Sébastien Han fbeacb325c Merge pull request #1192 from davidebelloni/master
Adding Proxy support
2016-12-15 17:38:38 +01:00
Davide Belloni 87373e75d9 Adding Proxy support 2016-12-15 15:28:59 +01:00
Sébastien Han f0687995e1 common: add the name of the ceph conf file in the play
Just for clarity and because we can we now show the name of the
ceph configuration file that is generated.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-15 15:26:01 +01:00
Sébastien Han 03f7de68a9 Merge pull request #1167 from guits/ceph-fsid
common: do not regenerate initial mon keyring if cluster exists
2016-12-15 15:16:40 +01:00
Shengjing Zhu a1b00e96db enable prepare osd with partition devices in raw_multi_journal
Address #895

Signed-off-by: Shengjing Zhu <zsj950618@gmail.com>
2016-12-15 22:03:38 +08:00
Sébastien Han 81baa6bb73 osd: docker change required variables for check
when running a containerized deployment, some variable are not
applicable thus should not be checked.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-15 14:50:29 +01:00
Guillaume Abrioux 471be5e629 common: do not regenerate initial mon keyring if cluster exists
This commit solves the situation where you lost your fetch directory and
you are running ansible against an existing cluster. Since no fetch
directory is present the file containing the initial mon keyring
doesn't exist so we are generating a new one.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2016-12-15 14:39:56 +01:00
Shengjing Zhu c50b5867a6 fix ceph-rest-api not using correct configure file
Signed-off-by: Shengjing Zhu <zsj950618@gmail.com>
2016-12-15 20:23:36 +08:00
Sébastien Han 245a556c5e Merge pull request #1186 from ceph/apt-pining
common: set apt pinning with Red Hat Ceph Storage
2016-12-15 12:28:45 +01:00
Sébastien Han f6736e5420 common: show which file we are waiting for
We can now see the name of the file we are waiting for, depending on the
cluster name this will change.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-14 21:08:18 +01:00
Sébastien Han 3776c23b9a common: remove uncessary conditions and spell red hat entirely
We do not need to run another condition for 'ceph_rhcs' since the
include we came from already has it, so we are already inside this
condition.
We also spell red hat entirely instead of rh and we remove capital
letters.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-14 19:03:04 +01:00
Sébastien Han 13a72fd24b common: set apt pinning with Red Hat Ceph Storage
When `ceph_stable_rh_storage` is True, every cluster node should have a
`/etc/apt/preferences.d/rhcs.pref` file with the following contents:

```
Explanation: Prefer Red Hat packages
Package: *
Pin: release o=/Red Hat/
Pin-Priority: 999
```

ceph-deploy already did this when used with ice-setup, and we need to do
the same thing with the ceph-ansible stack.

Closes: #1182 and https://bugzilla.redhat.com/show_bug.cgi?id=1404515

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-14 19:01:54 +01:00
Alfredo Deza 640442bee6 ceph-agent: do not check for subscription manager for agent install
Signed-off-by: Alfredo Deza <adeza@redhat.com>

Resolves: rhbz#1403576
2016-12-14 08:13:26 -05:00
Sébastien Han 2f6d603738 Merge pull request #1183 from zhsj/fix-custom-repo
fix ceph_custom not usable
2016-12-14 13:16:58 +01:00
Shengjing Zhu 9051f8d5b4 fix ceph_custom not usable
Only when ceph_origin == "upstream", install_on_redhat.yml will include
redhat_ceph_repository.yml, same as debian.

In redhat_ceph_repository.yml, ceph_custom_repo will be added.

But in check_mandatory_vars.yml, ceph_origin=="upstream" can't be combined
with ceph_custom
2016-12-14 16:55:41 +08:00
tynorth-cisco 3c6e156d08 Change attribute call to .get call in ceph keys
If previous check was not run, .stdout_lines is not a valid key on the dictionary.
To get around this, use .get("stdout_lines") instead.

Also add in a default empty list
2016-12-13 16:48:14 -08:00
Casey Bodley 0ebb728d7d ceph-common: remove libcephfs1 from debian_ceph_packages
in hammer, ceph-common depended on libcephfs (indirectly, via
python-cephfs). this is no longer the case in jewel or later, so it can
be removed from debian_ceph_packages

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2016-12-09 16:49:20 -05:00
Sébastien Han 189f4fee47 common: do not run tasks in main.yml, use include
For readibility and clarity we do not run any tasks directly in the
main.yml file. This file should only contain include, which helps us
later to apply conditionnals if we want to.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-09 16:00:46 +01:00
Sébastien Han 1de8176bf4 common: move mandatory variables to their respective roles
Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-09 14:45:05 +01:00
Sébastien Han 75cb749570 docker: consolidate ceph-ansible and ceph-docker varible
This commit re-uses some of the existing ceph-ansible variables for a
containirzed deployment. There is no reasons why we should add new
variables for the containerized deployment.

Signed-off-by: Sébastien Han <seb@redhat.com>
2016-12-09 14:39:05 +01:00
Andrew Schoen 2c98b1d70f Merge pull request #1169 from ceph/shaman-support
Shaman support for `ceph_dev`
2016-12-09 07:09:38 -06:00