Commit Graph

1661 Commits (f1232b33fd7a8da53aa2e1ad2b11ee16109633b3)

Author SHA1 Message Date
Guillaume Abrioux 8fb68297a2 common: remove unusuable conditions
`ceph_release` isn't available at this step of the playbook because it
is set later based on the installed binaries.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1486062

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-05 14:59:37 +02:00
Sébastien Han c803dedec8 Merge pull request #1993 from jprovaznik/log
Fix bind mount for /var/lib/nfs/ganesha directory
2017-10-05 14:43:26 +02:00
Michel Rode b462b68e65 Fixing path to osd_fragment.yml 2017-10-05 14:42:10 +02:00
Jan Provaznik b8916ecbc1 Include exports dir in ceph-nfs config file
Exports dir is used when dynamic exports creation is enabled.
2017-10-05 14:37:15 +02:00
Sébastien Han b545080d71 Merge pull request #1988 from ceph/fix_keyrings
docker: fix keyrings copied on all nodes
2017-10-05 14:30:09 +02:00
Sébastien Han bbf6bebe32 jewel: remove rbd check
The value of doing this is fairly low compare to the added value.
So we remove these tasks, if rbd pool on Jewel doesn't have the right PG
value you can always increase it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-05 14:21:37 +02:00
Jan Provaznik 62ea6f6e7f Fix bind mount for /var/lib/nfs/ganesha directory 2017-10-05 13:44:43 +02:00
Jan Provaznik 43e57abfd8 Evaluate cephfs pool variables
Otherwise pools with names 'cephfs_data' and 'cephfs_metadata'
are created.
2017-10-05 10:00:20 +02:00
Guillaume Abrioux 70e2787fe2 docker: fix keyrings copied on all nodes
All keyring are getting copied to all nodes.
This commit fixes a leftover from a previous code refactor.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1498583

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-05 09:23:22 +02:00
Guillaume Abrioux 8fac8f54a6 iscsi-gw: Create a rbd pool if it doesn't exist
iscsi-gw needs a 'rbd' pool to configure iscsi target.
Note: I could have used the facts already set in `ceph-mon` but I voluntarily
didn't do it to not create a dependancy between these two roles.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-04 15:40:10 +02:00
Guillaume Abrioux 2c4258a0fd Refact code for set_osd_pool_default_*
This commit refacts the code regarding all `set_osd_pool_default_*`
related tasks by avoiding usage of useless `set_fact` to determine
whether a key is present in `ceph_conf_overrides`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-04 15:40:10 +02:00
Al Lau 6aca67bc9c Only perform actions on the rbd pool after it has been created
The rbd pool is the default pool that gets created during ceph cluster
initializaiton.  If we act on the rbd related operations too early, the
rbd pool does not exist yet.  Move the call to perform rbd operations
to a later stage after other pools have been created.

The rbd_pool.yml playbook has all the operations related to the rbd pool.

Replace the always_run (deprecated) directive with check_mode.

Most of the ceph related tasks only need to run once.  The run_once directive
executes the task on the first host.

The ceph sub-command to delete a pool is delete (not rm).

The changes submitted here were tested with this ceph version.
ceph version 0.94.9-9.el7cp (b83334e01379f267fb2f9ce729d74a0a8fa1e92c)

This upload includes these changes:
  - Use the fail module (instead of assert).
  - From luminous release, the rbd pool is no longer created by default.
    Delete the code to create the rbd pool for luminous release
  - Conform the .yml files to use the suggested syntax.

The commands are executed on the mcp nodes and I think shell ansible module
is the right one to use.  The command module is used to execute commands on
remote nodes.  I can make the change to use command module if that is
prefrerred.
2017-10-04 15:40:10 +02:00
Sébastien Han cac7d034bf defaults: fix check socket non-container handler
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-04 15:33:52 +02:00
Sébastien Han c751c2dc6b nfs: add run once to user creation
The create user call is idempotent but it's also blocking for some
reasons.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-04 15:01:13 +02:00
Guillaume Abrioux 784cc73da0 set docker_exec_cmd fact early in each role
This is to ensure `docker_exec_cmd` fact is set with the correct value
in case of daemons collocation

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-04 11:31:09 +02:00
Sébastien Han 5968cf09b1 ci: add collocation scenario
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-04 11:19:12 +02:00
Sébastien Han f37e014a65 Merge pull request #1974 from ceph/mgr-upgrade-luminous
upgrade: a support for mgrs
2017-10-03 19:57:31 +02:00
Sébastien Han 0ce76113bf Merge pull request #1956 from ceph/osd-container-id
Osd container
2017-10-03 18:52:24 +02:00
Sébastien Han 99466e79a1 upgrade: a support for mgrs
Also we now play ceph-config to have everything being generated for new
daemons bootstrap during upgrade.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1497959
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-03 16:57:31 +02:00
Sébastien Han 27808a64a4 iscsi: fix when condition
generate_crt|bool|default(false) won't apply the default value, this
generate_crt|default(false)|bool will

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-03 16:48:17 +02:00
Sébastien Han 3bd341f6c0 osd: container use id instead of dev name
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1494127
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-03 14:44:00 +02:00
Sébastien Han ba42894516 osd: do not copy admin key on collocated scenario
ceph-disk used to have a bug requiring the admin key to store the
encrypted key in the mon kv store. This was reported in:
http://tracker.ceph.com/issues/17849

Fixed and backported here: https://github.com/ceph/ceph/pull/11996

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-10-03 14:44:00 +02:00
Guillaume Abrioux 081f226106 defaults: change running order in main.yml
The task which sets `ceph_current_fsid` in `facts.yml` in case of containerized
deployment, will definitely fail because `docker_exec_cmd` is not set
yet.
This commits simply makes `facts.yml` played after `check_socket.yml` so
`docker_exec_cmd` is set properly.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-02 18:42:43 +02:00
Sébastien Han 30ce781c79 Merge pull request #1968 from ceph/bz-1488999
refact MDS role
2017-10-02 14:42:08 +02:00
Guillaume Abrioux 62770cd7de refact MDS role
This commits refacts the role ceph-mds

The goal here is to create cephfs in `ceph-mon` for both containerized
and non-containerized cases so we don't need the admin keyring on mds
nodes anymore.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1488999

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-10-02 09:12:31 +02:00
Sébastien Han 46a01df434 osd: add cluster name support
I forgot to add cluster name support so some partition were never
mounted correctly.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-29 20:30:54 +02:00
Sébastien Han 0da6d8e356 Merge pull request #1967 from ceph/use_systemd_module
Use systemd module instead of service.
2017-09-29 16:35:10 +02:00
Guillaume Abrioux 466f6f35b7 Use systemd module instead of service.
Using systemd module allows us to do in one task what we did in three
tasks:

- enable unit file,
- issue a `daemon-reload`,
- start the service

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-29 14:54:00 +02:00
Sébastien Han e121bc58e9 defaults: add missing handlers for rbd mirorr and mgr
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-29 02:38:24 +02:00
Sébastien Han 048b55be4a defaults: only run socket checks on their specific roles
Running the socket check on all the hosts will override the default
value of docker_exec_cmd, leaving it with the last value (currently
rbd-mirror), as a result the subsequent docker_exec_cmd usage for the
:x

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-29 02:38:24 +02:00
Sébastien Han 341c9e077b nfs: fix container setup and re-arrange files
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-29 02:38:24 +02:00
Sébastien Han fc29ccd0ad rbd-mirror: force sercice enable ceph-rbd-mirror.target
There is a bug in the rbd mirror unit file, the upstream fix is here:
https://github.com/ceph/ceph/pull/17969.
This should be reverted once the patch is merged and backport is done.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-29 02:38:24 +02:00
Sébastien Han deb5d3ba1f Merge pull request #1962 from ceph/fix_mgr_sestatus
[skip ci] mgr: add condition to run selinux tasks only on rhel os family
2017-09-29 02:37:03 +02:00
Guillaume Abrioux 913ad53709 docker: add condition to run selinux tasks only on rhel os family
This fixes the error :

```
The conditional check 'sestatus.stdout != 'Disabled'' failed.
```

that occurs when running on non rhel based system since the
`sestatus` fact is registered only on rhel based distribution.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-29 02:35:07 +02:00
Sébastien Han 77fc8ba87f Merge pull request #1931 from ceph/re-enable-iscsi
iscsi: re-enable the scenario
2017-09-28 19:44:52 +02:00
Sébastien Han 67c78da056 iscsi: re-enable the scenario
CentOS 7.4 vagrant box is now available so re-enabling this scenario.
For more info:
https://seven.centos.org/2017/09/updated-centos-vagrant-images-available-v1708-01/

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-28 18:46:28 +02:00
Sébastien Han 0010979412 Merge pull request #1641 from fullerdj/wip-djf-key-timeout
mon/ceph_keys: Add timeout flag to ceph-create-keys
2017-09-28 09:40:50 +02:00
Guillaume Abrioux d20dc54202 docker-common: fix wrong syntax
there is no need to backslash the quotes here.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-28 00:30:08 +02:00
Douglas Fuller 9bcbf748a3 mon/ceph_keys: Add timeout flag to ceph-create-keys
Specify the timeout flag to ceph-create-keys, which causes it to time out
if a monitor quorum isn't achieved. This overrides the default timeout
of 10 minutes, causing ceph-ansible to fail faster in the event of cluster
network issues.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-09-27 18:05:59 -04:00
Zack Cerza 70b321f34c ceph-common: Fix logic for ceph_repository_type
It's failing if a *valid* choice is specified.

Signed-off-by: Zack Cerza <zack@redhat.com>
2017-09-25 15:28:27 -06:00
Sébastien Han e4ac736071 Merge pull request #1943 from ceph/mgr-site
handler: enhance socket detection
2017-09-25 18:43:32 +02:00
Sébastien Han 4266bb5d3f Merge pull request #1933 from ceph/osd-container-reboot
[skip ci] osd: fix container reboot
2017-09-25 18:36:25 +02:00
Sébastien Han 8b6456dc8a handler: enhance socket detection
We have seen issues with leftover socker. So now, if a socket is found
we also check if it's accessed by a process. If so, we can run the
handler, if not we remove it and continue the playbook.

Signed-off-by: Sébastien Han <seb@redhat.com>
Co-Authored-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-25 13:44:51 +02:00
Sébastien Han 45797ab968 osd: fix container reboot
It's sad but we can not rely on the prepare container anymore since the
log are flushed after reboot. So inpecting the container does not return
anything.
Now, instead we use a ephemeral container to look up for the
journal/block.db/block.wal (depending if filestore or bluestore) and
build the activate command accordingly.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-25 13:34:47 +02:00
Guillaume Abrioux be757122f1 config: fix path to set `interface` in ceph.conf
need to use `hostvars[host]['XXX']` to retrieve the monitor
interface and/or radosgw interface.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1493920

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-23 14:18:28 +02:00
Sébastien Han f3851df0c7 Merge pull request #1940 from ceph/rgw-interface
config: fix rgw interface when using different interfaces
2017-09-22 18:52:51 +02:00
Sébastien Han 8f71c08e7b handler: display ceph status properly
Fix bash error, doing ceph "$CEPH_CLI" -s gives us

ceph '--name client.bootstrap-osd --keyring
/var/lib/ceph/bootstrap-osd/test.keyring --cluster test' -s

which results in a wrongly formatted command. Removing the double quotes
expands the array properly.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-22 17:45:35 +02:00
Sébastien Han 2e0c2928e9 nfs: fix docker_exec_cmd_nfs default value
the default is not an array, default is empty.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-22 16:22:36 +02:00
Sébastien Han 4a55085914 config: fix rgw interface when using different interfaces
Conf file generation failing on rgw nodes when nodes have different
interface names.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1493552
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-22 15:41:11 +02:00
Sébastien Han 64824baa83 nfs: fix undefined variable
This is what happens when you don't run all the jobs from the CI...

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-22 15:37:43 +02:00
Sébastien Han 839bc11df0 Merge pull request #1923 from ceph/nfs-container
[skip ci] tests: add nfs container test
2017-09-22 12:22:10 +02:00
Sébastien Han aa5c36f19c nfs: several fixes
- move the file fetch/push to the existing task
- rename the include
- generate the ganesha template from ansible
- re-arrange role structure
- re-use tasks for non-container and container
- configure keys for non-container and container
- fix rgw container key collection;

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-22 00:37:32 +02:00
Guillaume Abrioux 599429dd31 common: fix debian install
in addition to #1926 this commit fixes the error when trying to include
`install_debian_rhcs_packages.yml`

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1493231

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-21 13:26:29 +02:00
Guillaume Abrioux b8c3fa9727 nfs: change ownership on /var/log/ganesha
to fix selinux capability issue that prevent nfs to start.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-21 09:56:37 +02:00
Guillaume Abrioux 1886a69b8b docker-common: refact `stat_ceph_files.yml`
there is no need to build the `ceph_config_keys` fact in several steps
for rbd-mirror keyring.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-21 09:56:37 +02:00
Guillaume Abrioux 62cd0bae54 rbd: fix missing keyring on nodes
the rbd key was not pushed on rbd nodes because its keyring path was not
added in `ceph_config_keys`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-21 09:56:37 +02:00
Guillaume Abrioux 295c1b0610 docker-common: fix ceph_health check
`docker ps` will always return `0`
(see: https://github.com/docker/cli/issues/538).

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-21 09:56:37 +02:00
Guillaume Abrioux 6c9f3a08a7 rgw: refact start_docker_rgw.yml
remove usage of `shell` module in favor of `systemd` module.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-21 09:56:37 +02:00
Guillaume Abrioux 90c4066ce5 mgr: add missing admin key for mgr container
Followup on #1761.
Add missing admin key for mgr node in containerized deployment.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-21 09:56:37 +02:00
Sébastien Han adf5017924 config: remove max open file
This is only used by the old sysvinit scripts

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-20 23:06:01 +02:00
Sébastien Han a4baed1025 config: no not generate osd section if bluestore
This section is not needed when running a bluestore osd.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-20 18:14:48 +02:00
Sébastien Han cb05172605 docker: we don't need to copy the ceph.conf on all the nodes
We generate the ceph.conf on all the nodes through the
ceph-docker-common so there is no need to push it to the Ansible file.

Also this is breaking the ceph.conf template generation since we only
generate sections based on the host the ansible task is running on.

For example, what's typically happening, we bootstrap the monitor, we
get a ceph.conf generated for a mon only, we go on an osd, we generate
the ceph.conf with osd section (done by ceph-docker-common) but this
gets overwritten by the copy_config task of the ceph-osd role.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-20 16:33:29 +02:00
Sébastien Han 7aab133617 Merge pull request #1920 from jprovaznik/ganesha
Make ceph-nfs service enablement/start optional
2017-09-20 14:48:36 +02:00
Sébastien Han a89363b0ae Merge pull request #1926 from ceph/rhcs-debina
common: fix rhcs debian install
2017-09-19 19:50:40 +02:00
Sébastien Han 75e77f5948 common: fix rhcs debian install
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1493231
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-19 19:49:44 +02:00
Ali Maredia 3ba1a68cf5 nfs: ganesha.conf template fixes
- Change capitalization of config options to be
in line with what config.txt in the nfs-ganesha
tree says

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-09-19 12:45:24 -04:00
Sébastien Han dd7f21bd92 common: fix rhcs installation and rgw package for nfs
RHCS install wasn't working at all prior to this commit as the name of
the include was pointing to a non-existing file.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1492056
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-19 12:12:22 +02:00
Sébastien Han 87e2dae9d8 Merge pull request #1919 from ceph/iscsi-check
common: fix rhel check
2017-09-19 12:10:10 +02:00
Sébastien Han ace97e8720 Merge pull request #1904 from ceph/name-include-fact
name includes and set_fact for clarity
2017-09-19 12:09:25 +02:00
Jan Provaznik 8c510ab9f9 Make ceph-nfs service enablement/start optional
When ceph-nfs service is managed by pacemaker, it's useful to
not enable and start ceph-nfs service through systemd but let
pacemaker to start the service in a next step.
2017-09-19 11:59:54 +02:00
Sébastien Han dbe64f66f7 common: fix rhel check
Looks like Ansible is now using "RedHat" instead of "Red Hat Enterprise
Linux"

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-19 11:55:26 +02:00
Sébastien Han 773010ee49 Merge pull request #1911 from fghaas/1910
Introduce ceph_nfs_ceph_user
2017-09-19 10:03:46 +02:00
Florian Haas ada2f147f5 Introduce ceph_nfs_ceph_user
In analogy to ceph_nfs_rgw_user, we should be able to define a user
with which the nfs-ganesha Ceph FSAL connects to the cluster.

Introduce a ceph_nfs_ceph_user variable, setting its default to
"admin" (which preserves the prior behavior of always connecting as
client.admin).

Fixes #1910.
2017-09-19 09:07:28 +02:00
Sébastien Han d100b4e596 name includes and set_fact for clarity
When Ansible is not run with verbose options it's difficult to see which
include and/or set_fact does what. So adding a name for each clarifies.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-18 23:39:46 +02:00
Sébastien Han 66d41f342d Merge pull request #1889 from ceph/client-containers
client: ability to create keys and pool with no ceph binaries
2017-09-18 17:27:32 +02:00
Sébastien Han 2749368a2d Merge pull request #1915 from ceph/state-leftover
docker-common: re-introduce state for leftover files
2017-09-18 15:46:07 +02:00
Sébastien Han aa5d94fc87 docker-common: re-introduce state for leftover files
The variable "statleftover" was removed by commit
a60c74f61e
and never added back to the new playbook,
yet it is still being referenced.

Adding it back

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1492224
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-18 15:01:32 +02:00
Sébastien Han 85d73e3be2 client: ability to create keys and pool with no cpeh binaries
On a container env, machines don't have any ceph binaries so we need to
use a container to run the commands.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-18 14:41:52 +02:00
Sébastien Han 68a1390dc9 Merge pull request #1898 from ceph/restart-mon
defaults: restart docker daemon higher delay
2017-09-15 06:23:51 -06:00
Sébastien Han ed3003cf41 defaults: restart docker daemon higher delay
Use default delay since the mon (in particular) can take more time to
restart.
Solves error with:

STDERR:

Error response from daemon: No such container: ceph-mon-mon0

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-14 13:38:11 -06:00
Sébastien Han fb02b1d9d3 mon: create the mgr key for release >= luminous
This fixes RHCS builds. We know which Ceph version we are running on.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-14 11:06:44 -06:00
Sébastien Han 6f0b1fe803 rgw: remove old variables
Since the only support civetweb these variables are obsolete.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-14 09:42:50 -06:00
Sébastien Han 660893e70e osd: add meaningful message for journal_size
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-13 23:49:15 -06:00
Sébastien Han ef8d37dd0d Merge pull request #1800 from ceph/wip-osd-start-fix
ceph-osd: Fix osd start sequence
2017-09-13 17:20:10 -06:00
Sébastien Han 2f51f0de28 Merge pull request #1880 from ceph/wip-rgw-nfs
nfs: configure RGW FSAL to start up correctly
2017-09-13 14:20:14 -06:00
Sébastien Han f67b47d056 Merge pull request #1882 from ceph/multi-journal
osd: drop support for device partition
2017-09-13 11:43:48 -06:00
Sébastien Han ac62437609 Merge pull request #1883 from ceph/quick_refact
osd: refact include of `activate_osds.yml`
2017-09-12 22:11:31 -06:00
Sébastien Han c3866fc4bd Merge pull request #1747 from ceph/add-iscsi
resync ceph-iscsi-gw with old upstream
2017-09-13 02:06:50 +02:00
Sébastien Han aa364264cd resync ceph-iscsi-gw with old upstream
Taken from https://github.com/pcuzner/ceph-iscsi-ansible/tree/tcmu-fixes

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1454945 and
https://bugzilla.redhat.com/show_bug.cgi?id=1484083
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-12 18:06:10 -06:00
Sébastien Han 2a1b8a1997 Merge pull request #1884 from ceph/mon-container-ip
mon: add support for monitor_address block for containers
2017-09-13 01:46:18 +02:00
Sébastien Han fdf924401f osd: drop support for device partition
We have been struggling with this, it's still broken and breaking other
things too now.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1490283
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-12 17:42:07 -06:00
Guillaume Abrioux 49ad8528e5 osd: refact include of `activate_osds.yml`
remove duplicate code.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-12 16:53:11 -06:00
Sébastien Han 02ba65dbbe mon: add support for monitor_address block for containers
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-12 16:28:08 -06:00
Sébastien Han 6b8ed0440e Merge pull request #1761 from ceph/split_copy_keys
docker: split the task 'copy ceph configs&keys'
2017-09-13 00:21:50 +02:00
Ali Maredia 52efe92a87 nfs: configure RGW FSAL to start up correctly
- Add RGW keyring to nfs node
- Add RGW section to ganesha.conf
- Add RGW section to ceph.conf onf nfs node

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-09-12 16:27:16 -04:00
Guillaume Abrioux 20946f7220 ceph-osd: remove deprecated comment in sample file
Since #1724 has been merged, this comment is deprecated

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-12 16:48:10 +02:00
Guillaume Abrioux 0f506f4f0a Docker: split the task 'copy ceph configs&keys'
All keys are copied to all nodes.
This commit split that task in each roles so keys are copied to their
respective nodes.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1488999

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-11 21:14:13 +02:00
Sébastien Han 2ea7f287fa docker: simplify variable declaration
Less configuration for the user, the container inherit from the global
variables. No more container specific variables.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-09 01:22:06 +02:00
Sébastien Han 4767eaaab3 Merge pull request #1878 from ceph/add-rbd-mirror
Add rbd mirror
2017-09-09 01:21:12 +02:00
Sébastien Han 7054615551 ci: deploy rbd mirror
Deploy rbd mirorr in cluster scenario

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-09 01:17:10 +02:00
Sébastien Han 477f86e305 switch to container: fix ceph nfs
The service is nfs-ganesha where ceph-nfs@{{ ansible_hostname }} will be
the name of the container.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-08 22:43:50 +02:00
Sébastien Han d46d453b83 Merge pull request #1780 from ceph/wip-rgw-nfs
Wip RGW NFS
2017-09-08 19:26:02 +02:00
Guillaume Abrioux b59e9cc732 Merge pull request #1871 from ceph/handler-collocate
defaults: do not restart unconfigured (yet) daemons
2017-09-08 18:15:02 +02:00
Sébastien Han a05c58ba37 Merge pull request #1874 from ceph/rbd-mirror-mem
ceph-rbd-mirror; docker fix typo
2017-09-08 17:50:55 +02:00
Sébastien Han 7a93d88025 ceph-rbd-mirror; docker fix typo
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-08 17:47:48 +02:00
Ali Maredia f8171e8b4a nfs: rename host to have ceph- prefix
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-09-08 11:38:05 -04:00
Ali Maredia f3e2235b3a nfs-ganesha: add config overrides section
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-09-08 11:37:58 -04:00
Sébastien Han d53f55e807 Merge pull request #1870 from Logan2211/omit-default-release
Omit the apt default_release if it is not needed
2017-09-08 16:55:03 +02:00
Guillaume Abrioux 44fd928e23 mds: rename mds_socket fact
Rename this fact to keep consistency with handlers in `ceph-defaults`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-08 15:57:58 +02:00
Ali Maredia 55724c6e93 nfs-ganesha: add dev, stable, and rhcs nfs-ganesha's for ceph-nfs role
Signed-off-by: Ali Maredia <amaredia@redhat.com>
2017-09-08 09:13:20 -04:00
Sébastien Han 12f6e53090 defaults: do not restart unconfigured (yet) daemons
In a collocated scenario, where you might put a rgw, a mds and a mon on
the same node you don't want the handler blindly restart all the daemons
on the node. Indeed some of them might not be configured yet.
Implementing a more precise socket detection, for each daemon type.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1488813
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-08 12:02:37 +02:00
Logan V d8cb62c981 Omit the apt default_release if it is not needed
The apt module will fail to downgrade packages properly when defualt
release is unnecessarily defined.

Closes #1869
2017-09-07 11:50:57 -05:00
Sébastien Han 3753e6cfa7 ceph-osd: fix autodetection activation
Prior to this patch this activation sequence for autodetection was
always skipped because we were asking to activate on device without
partitions, which doesn't make sense.

We also fix the way we lookup for a device, since the data partition is
always numbered 1, we take the min element of the dict.

Closes: https://github.com/ceph/ceph-ansible/issues/1782
Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-09-07 17:47:37 +02:00
Sébastien Han 27b3f9a7d4 Merge pull request #1850 from fultonj/issue/1848
Add option to create client keyring file but not import it
2017-09-07 13:51:11 +02:00
Sébastien Han cf88c136f5 Merge pull request #1859 from ceph/container-limit
container: introduce resource limitation for containers
2017-09-07 12:51:34 +02:00
Sébastien Han d2032c92af Merge pull request #1862 from ceph/fail-ansible
fail if ansible version < 2.3
2017-09-07 08:44:01 +02:00
Sébastien Han fc3300ea4f fail if ansible version < 2.3
We only test and support 2.3.x at the moment.

Closes: https://github.com/ceph/ceph-ansible/issues/1858
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-07 07:53:17 +02:00
John Fulton a57f61efd9 Add option to create client keyring file but not import it
Add new boolean parameter for client config create_key_file_only
with a default of false. When create_key_file_only is true, the
client tasks to connect to the external ceph cluster to verify
the key `ceph auth import` the key are skipped.

Fixes: #1848
2017-09-06 13:56:06 +00:00
Sébastien Han 2fa151b9e8 container: introduce resource limitation for containers
This can be controlled via 2 options:

* ceph_$DAEMON_docker_memory_limit
* ceph_$DAEMON_docker_cpu_limit

All daemons default to 1GB for memory and 1 CPU by default.
Recommendations from:
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/2/html/red_hat_ceph_storage_hardware_guide/minimum_recommendations

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-06 14:52:21 +02:00
Sébastien Han b7db600caa switch-from-non-containerized-to-containerized: mask unit files
We must mask the image so we are sure that even if the system reboots
then the OSDs won't start.

Also remove Ceph udev rules if found on the system prior to deploy
containers. If we don't do this we are exposed to conflicts between udev
rules and sytemd unit files.

Also add the CI will now test the migration from a non-containerized cluster to a
containerized cluster.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-05 15:20:31 +02:00
Sébastien Han b6c1a0c68f Merge pull request #1853 from ceph/fix-prepare
ceph-osd: do not re-prepare if already prepared
2017-09-05 13:59:40 +02:00
Sébastien Han 5ed1a91aeb Merge pull request #1819 from ceph/no-container-log
ceph-docker-common: do not log inside the container
2017-09-05 11:47:11 +02:00
Sébastien Han 1dd976d28e ceph-osd: do not re-prepare if alreadyy prepared
I forgot to re-add the partition check while refactoring the osd

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-05 09:51:57 +02:00
Sébastien Han 23a0c26c4f client: do not copy admin key by default
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-02 00:54:17 +02:00
Sébastien Han 58f664fd17 ceph-rgw: fix systemd unit layout
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-09-01 19:02:48 +02:00
Sébastien Han 967e875fd0 Merge pull request #1827 from andymcc/rgw_systemd_fix
Fix RGW systemd directory
2017-09-01 18:12:23 +02:00
Alfredo Deza 98d107cebb common do not filter by distro when dev is set for figuring out ceph_release
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-31 16:18:08 -04:00
Sébastien Han 673938ec96 Merge pull request #1839 from ceph/colonwq-update-docker-rgw-exec
Update ceph_rgw_docker_extra_env to add bind ip
2017-08-31 19:47:16 +02:00
Sébastien Han ea9b6395cb Merge pull request #1838 from ceph/rgw-units
Rgw units
2017-08-31 19:38:23 +02:00
Andrew Schoen 29df79e54e Merge pull request #1841 from ceph/lvm-partitions
lvm-osds: test with a partition and an lv as journals
2017-08-31 12:09:19 -05:00
Sébastien Han 3dd47a45cb ceph-defaults: fix handlers for mds and rgw
The way we handle the restart for both mds and rgw is not ideal, it will
try to restart the daemon on the host that don't run the daemon,
resulting in a service file being created (see bug description).

Now we restart each daemon precisely and in a serialized fashion.

Note: the current implementation does NOT support multiple mds or rgw on
the same node.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1469781
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-31 19:02:21 +02:00
Sébastien Han 7ee1f88ee5 ceph-common: remove useless changed task
There is no need to show a "changed" at the end of the play for a
"command" module task.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-31 18:27:43 +02:00
Keith Schincke eaccc12797 Update ceph_rgw_docker_extra_env to add bind ip
This patch adds passing the RGW_CIVETWEB_IP to the docker
container. This IP defaults to the value of radosgw_civetweb_bind_ip.
radosgw_civetweb_bind_ip default to ipv4.default

Without this value, the RGW containter will bind to 0.0.0.0
2017-08-31 15:50:34 +02:00
Sébastien Han e581539e20 ceph-rgw: do not run a privileged rgw container
There is no need for a privileged rgw container

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-31 15:50:15 +02:00
Sébastien Han 7ccd10a15e rgw: cleanup old code and remove systemd condition
Remove the old check prior systemd.
We only support systemd so there is no need to run a condition on
systemd. The playbook will fail if systemd is not present.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-31 08:29:30 +02:00
Andrew Schoen fcba9d17f0 ceph-osd: add support for --journal vg/lv for lvm osds
This also updates the tests

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-30 15:55:16 -05:00
Alfredo Deza da90edce3e common dev repos should not need to specify a 'release'
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-30 13:37:24 -04:00
Alfredo Deza 6565c38238 common: ceph_repository should not be rhcs or dev
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-30 13:33:04 -04:00
Alfredo Deza 8fd2bf7e2c common: use the value of ceph_repository in the error message
Signed-off-by: Alfredo Deza <adeza@redhat.com>
2017-08-30 13:33:04 -04:00
Sébastien Han 13aac5027a Merge pull request #1741 from ceph/refactor-installation
common: refactor installation method
2017-08-30 17:42:29 +02:00
Sébastien Han b05271f464 Merge pull request #1724 from ceph/container-multi-journal
osd: allow multi dedicated journals for containers
2017-08-30 17:41:42 +02:00
Sébastien Han a60c74f61e ceph-docker-common: re-organize stat ceph file
Use a single file to run the checks instead of duplicating code.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-30 14:44:34 +02:00
Sébastien Han e0a264c7e9 osd: allow multi dedicated journals for containers
Fix: https://bugzilla.redhat.com/show_bug.cgi?id=1475820
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-30 12:34:06 +02:00
Sébastien Han ae2fd45994 common: refactor installation method
The installation process is now described as follow:

* you still have to choose a 'ceph_origin' installation method. The
origin can be a 'repository' (add a new repository), distro (it will use
the packages provided by the native repo source of your distribution),
local (only available on redhat system, it installs locally built
packages). This option is not well tested, so use it carefully

* if ceph_origin == 'repository' you will have to decide what kind of
repository you want to enable:
  - community: corresponds to the stable upstream/community version
  - enterprise: corresponds to the stable enterprise/downstream version
    (basically you are a red hat customer)
  - dev: it will install ceph from packages built out of the github
    development branches

Signed-off-by: Sébastien Han <seb@redhat.com>
Co-Authored-by: Guillaume Abrioux <gabrioux@redhat.com>
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-30 10:52:01 +02:00
Andy McCrae a9d91c3d69 Fix RGW systemd directory
The ceph RGW systemd services are actually named "ceph-radosgw" and not
"ceph-rgw", this patch fixes that for the systemd overrides file.
2017-08-29 17:24:52 +01:00
Sébastien Han 5743916092 common: add mimic release facts
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-29 17:21:37 +02:00
Sébastien Han fa9f2313d5 Merge pull request #1822 from ceph/rhcs-container-release
ceph-docker-common: detect ceph version
2017-08-29 12:16:20 +02:00
Sébastien Han d0515cb704 Merge pull request #1825 from ceph/fix-item
ceph-docker-common: fix empty array
2017-08-29 12:15:46 +02:00
Sébastien Han b3e5206289 Merge pull request #1814 from ceph/handler-defaults
handler: default to empty array if task skipped
2017-08-29 11:09:35 +02:00
Sébastien Han cfddd2903c ceph-docker-common: fix empty array
The list can not be evaluated properly if it containers '[]', which is
the case when using the filter "default([])". To fix this, we have to
properly merge the lists.

This is fixing the issue: "list object has no element 1"

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-29 10:25:46 +02:00
Sébastien Han 764e697186 ceph-docker-common: detect ceph version
By detecting the ceph version running in the container we can easily
apply conditions like:
ceph_release_num.{{ ceph_release }} >= ceph_release_num.luminous

We do that already, in ceph-docker-common/tasks/fetch_configs.yml.

This fixes the error:

TASK [ceph-docker-common : register rbd bootstrap key]
******************************************************

fatal: [magna005]: FAILED! => {"failed": true, "msg": "The conditional
check 'ceph_release_num.{{ ceph_release }} >= ceph_release_num.luminous'
failed. The error was: error while evaluating conditional
(ceph_release_num.{{ ceph_release }} >= ceph_release_num.luminous):
'dict object' has no attribute 'dummy'\n\nThe error appears to have been
in
'/home/ubuntu/ceph-ansible/roles/ceph-docker-common/tasks/fetch_configs.yml':
line 2, column 3, but may\nbe elsewhere in the file depending on the
exact syntax problem.\n\nThe offending line appears to be:\n\n---\n-
name: register rbd bootstrap key\n  ^ here\n"}

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1486062
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-28 23:28:47 +02:00
Sébastien Han aa69c2c007 ceph-docker-common: do not log inside the container
Logging inside the container is not useful since it writes to the
overlayfs partition, resulting in potential performance degradation on
the container.

If you need to check the logs, just look at journald.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-28 12:04:49 +02:00
Sébastien Han 29753da05c handler: default to empty array if task skipped
with_items is evaluated before the when condition so if the task that
registers the 'results' is skipped the task will fail with:

{"failed": true, "msg": "'dict object' has no attribute 'results'"}

Defaulting to an empty array fixes the issue.

Reverts: abdd66619e
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1482061
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-25 18:39:00 +02:00
Sébastien Han 972eb45d31 ceph-docker-common: apply 0600 to key permissions
Keys should only be readable and writable by their respective owners and that's all.

Closes: https://github.com/ceph/ceph-ansible/issues/1760
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-25 18:14:28 +02:00
Boris Ranto 5f1b8fcd75 ceph-osd: Fix osd start sequence
The script can fail to get the osd id because the osds are activated by
udev and it can take a while for them to activate. This commit fixes
that by trying to get all the osds per node in a loop.

This commit also makes the osd services enabled so that they are
available after reboot.

Signed-off-by: Boris Ranto <branto@redhat.com>
2017-08-25 13:40:04 +02:00
Sébastien Han 1f4082f200 update meta for ansible galaxy
Closes: https://github.com/ceph/ceph-ansible/issues/1637
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-25 00:05:44 +02:00
Sébastien Han aee8267be4 Merge pull request #1808 from ceph/role-path
ceph-mon: detect ANSIBLE_ROLES_PATH if present
2017-08-24 23:49:41 +02:00
Andrew Schoen 910bb036c6 ceph-config: when using local_action set become: false
There should be no need to use sudo when writing or using these files.
It creates an issue when the user running ansible-playbook does not
have sudo privs.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-24 10:07:03 -05:00
Sébastien Han 76ac9b077b ceph-mon: detect ANSIBLE_ROLES_PATH if present
Some deployments can't copy infrastructure playbooks outside of the
infrastructure-playbooks directory. Thus they use ANSIBLE_ROLES_PATH to
overcome this. However some roles have 'playbook_dir' hardcoded, which
results in wrong path since the execution comes from
infrastructure-playbooks. Basically the role triggered by a playbook
from infrastructure-playbooks believes that the roles are in
infrastructure-playbooks/roles. This commit fixes that.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-24 16:19:39 +02:00
Andrew Schoen d0a3034857 ceph-config: write ceph_conf_overrides_temp to fetch_directory
because /tmp is not always writable, but we can assume that the
fetch_directory will be

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-24 11:33:03 +02:00
Sébastien Han 80dc5eead7 ceph-config: add missing meta and files for the galaxy
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-24 11:33:03 +02:00
Guillaume Abrioux 539197a2fc Introduce new role ceph-config.
This will give us more flexibility and the possibility to deploy a client node
for an external ceph-cluster.

related BZ:
https://bugzilla.redhat.com/show_bug.cgi?id=1469426

Fixes: #1670

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-24 11:33:03 +02:00
Sébastien Han 6d894e556c ceph-mon: remove hardcoded ipv4 in containers
Before this commit we were forcing ipv4 which might not be available.
Now setting ip_version to ipv4 or ipv6 will give you the right support.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1484189
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-24 11:33:02 +02:00
Andrew Schoen 758c31b1cd ceph-osd: ceph-volume requires --data to be in vg/lv format
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-23 13:43:31 -05:00
Alfredo Deza e651469a2a Merge pull request #1797 from ceph/purge-lvm
adds purge support for the lvm_osds osd scenario
2017-08-23 14:28:29 -04:00
Sébastien Han f2499ff5ac Merge pull request #1788 from ceph/improve-switch
switch-from-non-containerized-to-containerized: simplify
2017-08-23 19:47:26 +02:00
Sébastien Han 4f0ecb7f30 switch-from-non-containerized-to-containerized: simplify
This commit eases the use of the
infrastructure-playbooks/switch-from-non-containerized-to-containerized-ceph-daemons.yml
playbook. We basically run it with a couple of pre-tasks and then we let
the playbook run the docker roles.

It obviously expect to have proper variables configured in order to
work.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-23 18:39:45 +02:00
Andrew Schoen 594d5e017a ceph-osd: restructure lvm_volumes variable for more flexiblity
The lvm_volumes variable is now a list of dictionaries that represent
each OSD you'd like to deploy using ceph-volume. Each dictionary must
have the following keys: data, journal and data_vg. Each dictionary also
can optionaly provide a journal_vg key.

The 'data' key represents the lv name used for the OSD and the 'data_vg'
key is the vg name that the given lv resides on. The 'journal' key is
either an lv, device or partition. The 'journal_vg' key is optional and
must be the vg name for the journal lv if given. This key is mainly used
for purging of the journal lv if purge-cluster.yml is run.

For example:

  lvm_volumes:
    - data: data_lv1
      journal: journal_lv1
      data_vg: vg1
      journal_vg: vg2
    - data: data_lv2
      journal: /dev/sdc
      data_vg: vg1

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-23 10:14:14 -05:00
Sébastien Han d9b3d4a981 Merge pull request #1731 from SirishaGuduru/rgw-civetwebIP-conf
Common: changed civetweb line in rgw section(conf)
2017-08-23 15:33:08 +02:00
Sébastien Han e0c43ccc53 Merge pull request #1784 from ceph/fix-restart-osd-container
ceph-defaults: fix handler for osd container
2017-08-23 12:40:01 +02:00
SirishaGuduru 1359869497 Common: changed civetweb line in rgw section(conf)
Resolves issue: Multiple RGW Ceph.conf Issue #1258

In multi-RGW setup, in ceph.conf the RGW sections
contain identical bind IP in civetweb line. So this
modification fixes that issue and puts the right IP
for each RGW.

Signed-off-by: SirishaGuduru SGuduru@walmartlabs.com

Modified ceph-defaults and ran generate_group_vars_sample.sh

group_vars/osds.yml.sample and group_vars/rhcs.yml.sample are
not part of the changes. But they got modified when
generate_group_vars_sample.sh is ran to generate group_vars/
all.yml.sample.

Uncommented added variables in ceph-defaults

Updated tests by adding value for radosgw_interface

Added radosgw_interface to centos cluster tests

Modified ceph-rgw role,rebased and ran generate_group_vars_sample.sh

In ceph-rgw role removed check_mandatory_vars.yml.
Rebased on master.
Ran generate_group_vars_sample.sh and then the below files got
modified.
2017-08-23 15:03:37 +05:30
Jason Dillaman b70d54ac80 rbd-mirror should use per-host user id keyring
The rbd-mirror daemon will be HA under luminous and new daemon health
features require a way to uniquely identify rbd-mirror instances.

Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-08-22 18:55:29 -04:00
Jason Dillaman 70c2b934ca distribute rbd bootstrap key if available
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2017-08-22 18:55:29 -04:00
Sébastien Han 07821d9bb1 Merge pull request #1786 from ceph/re-arrange-skipped
mon, osd: fix skipped condition
2017-08-22 19:44:48 +02:00
Sébastien Han a359fc35b4 mon, osd: fix skipped condition
To be properly evaluated the "skipped" conditions must always have the
first place on the list of condition, otherwise the other conditions are
evaluated before and make the task fail.

Closes: https://github.com/ceph/ceph-ansible/issues/1733
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-22 18:34:51 +02:00
Yoann Moulin a7e4562297 fix radosgw-admin call with another cluster name than "ceph" 2017-08-22 16:57:12 +02:00
Sébastien Han 38d575ce55 Merge pull request #1654 from andymcc/master
Allow ceph-mon systemd overrides to be specified
2017-08-22 15:32:39 +02:00
Sébastien Han abdd66619e ceph-defaults: fix handler for osd container
Problem: task "check for a ceph socket in containerized deployment" will
be skipped if we are not an OSD.

with_items are still evaluated before when conditions so if the task was
skipped the dict will be empty and then fail.
Adding a "not skipped" condition skips the execution of the task.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1482061
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-22 11:56:05 +02:00
Sébastien Han 19ae8b42e6 resync group_vars files
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-22 11:31:49 +02:00
Sébastien Han b8af5fc5a0 Merge pull request #1695 from fullerdj/wip-djf-docker-multimds
ceph-mds: Enable multimds under docker
2017-08-21 10:23:21 +02:00
Guillaume Abrioux e0e9bb33b1 common: install ceph-common on all nodes
This commits force ceph-common to be installed early in deployment on
nodes.

For instance, ceph-rbdmirror doesn't have the CLI installed while it is
needed for some tasks which uses it to set some facts.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-17 14:44:38 +02:00
Andy McCrae 4671b9e74e Allow ceph service systemd overrides to be specified
ceph services can fail to start under certain circumstances (for
example, when running in a container) because the default systemd
service configuration causes namespace issues.

To work around this we can override the system service settings by
placing an overrides file in the ceph-<service>@.service.d directory.
This can be generic so as to allow any potential changes required to
the ceph-<service> service files.

The overrides file is only setup when the
"ceph_<service>_systemd_overrides" config_template override variable is
specified.

The available service systemd override files are as follows:
ceph_mds_systemd_overrides
ceph_mgr_systemd_overrides
ceph_mon_systemd_overrides
ceph_osd_systemd_overrides
ceph_rbd_mirror_systemd_overrides
ceph_rgw_systemd_overrides
2017-08-16 17:57:06 +01:00
John Fulton c04559181e Set the permissions mode on all of the OpenStack keys
The original fix to issue #1755 only set the permissions on
the monitors to which the key was copied, but not the original
monitor where the key was created. Thus, we use a separate task
to set the permission of the key.
2017-08-10 13:50:08 -04:00
John Fulton 7d42941090 Allow user to specify the mode of the openstack keys
The openstack_keys structure now supports a key called mode
whose value is a string that one could pass to chmod to set
the mode of the key file. The ansible file module applies the
mode to all openstack keys with this property.

Fixes: #1755
2017-08-10 15:34:39 +00:00
Andrew Schoen 30f9b0e075 ceph-common: render ceph_conf_overrides into fetch_directory
Writing into /tmp is not always allowed, but we can assume the
fetch_directory is writable.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 12:19:12 -05:00
Douglas Fuller aab3318802 ceph-mds: Enable multimds under docker
This is under the MDS role instead of the mon role because that role
does not create the filesystem under docker.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-08-04 10:46:11 -04:00
Andrew Schoen be78bc1a90 ceph-defaults: fix containerized osd restarts
This needs to check `containerized_deployment` because
socket_osd_container is undefined otherwise.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:38:38 -05:00
Andrew Schoen d1c7ec81c1 ceph-common: move release-rhs.yml after ceph_version is set
These tasks needs to be run after we set ceph_version or
they fail because it's undefined.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:38:37 -05:00
Andrew Schoen 1d5f876729 ceph-osd: devices is not required when osd_scenario == lvm
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:38:37 -05:00
Andrew Schoen e597628be9 lvm: update scenario for new osd_scenario variable
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:38:36 -05:00
Andrew Schoen 3b5a06bb3c lvm-osds: reorder mandatory vars checks
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:13:10 -05:00
Andrew Schoen 96c92a154e lvm-osds: check for osd_objectstore == 'filestore'
ceph-volume currently only has support for filestore, not bluestore

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:13:10 -05:00
Andrew Schoen 61d63f8468 lvm-osds: make task name and files consistent
Removes capitilization and newlines to keep these files consistent in
style with the existing tasks.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:13:10 -05:00
Andrew Schoen 63b7e3d36c lvm_osds: ensure osd daemons are started
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:13:09 -05:00
Andrew Schoen b93794bed4 adds a new 'lvm_osds' osd scenario
This scenario will create OSDs using ceph-volume and is only available
in ceph releases greater than Luminous.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-08-04 06:13:09 -05:00
Sébastien Han e6a5c3b815 Merge pull request #1734 from ceph/debian-repo
common: automate setting up online repositories for ceph deployments …
2017-08-03 22:45:31 +02:00
Sébastien Han 7559a2deff common: automate setting up online repositories for ceph deployments on debian nodes
This commits automates the process of setting up online repositories for
Red Hat Ceph Storage on Debian nodes. The manual steps are currently
described here:
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/2/html/installation_guide_for_ubuntu/prerequisites#online_repositories

If you are an RHCS customer and run a Debian based system you can now
access package through the Red Hat CDN.
For this set: ceph_rhcs and ceph_rhcs_cdn_install to true. Then set your
customer credentials in ceph_rhcs_cdn_debian_repo. Replace
customername:customerpasswd with your details.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1434175
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-03 17:15:07 +02:00
Andrew Schoen d2f4d3666f Merge pull request #1725 from ceph/simplify-osd-scenario
osd: simply osd scenario declaration
2017-08-03 09:31:57 -05:00
Sébastien Han af1e4d16d2 common: override and autodetect ceph_stable_release
For ceph_dev and rhcs installation we need to detect the release since
we do not declare it explicitly. Keeping the default ceph_stable_release
could lead to several things going wrong and some have already been
reported.

Fixes: https://github.com/ceph/ceph-ansible/issues/1712 and
https://bugzilla.redhat.com/show_bug.cgi?id=1476210
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-03 14:13:31 +02:00
Sébastien Han 30991b1c0a osd: simplify scenarios
There is only two main scenarios now:

* collocated: everything remains on the same device:
  - data, db, wal for bluestore
  - data and journal for filestore
* non-collocated: dedicated device for some of the component

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-08-03 10:20:39 +02:00
Guillaume Abrioux 608bad901d docker-common: Fix bug when updating config
in containerized deployment, if you try to update your `ceph.conf` file
it won't be actually updated on your nodes because it is overwritten by
the copy of the file which is present in your fetch directory.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-02 17:12:20 +02:00
Guillaume Abrioux 7a333d05ce Add handlers for containerized deployment
Until now, there is no handlers for containerized deployments.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-02 17:12:20 +02:00
Guillaume Abrioux fc6b6e9859 Move basics facts to `ceph-defaults`
Move `fsid`,`monitor_name`,`docker_exec_cmd` and `ceph_release` set_fact
to `ceph-defaults` role.
It will allow to reuse these facts without having to play `ceph-common`
or `ceph-docker-common`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-02 17:12:20 +02:00
Guillaume Abrioux 5adbf0fdaa Move role dependencies in site.yml/site-docker.yml
This will give us more flexibility and avoid a lot of useless when
skipping all tasks from a non-desired role.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-02 17:12:14 +02:00
Guillaume Abrioux 7322526838 Add new role `ceph-defaults`
Add a new role `ceph-defaults`.
This role aims to handle all defaults vars for `ceph-docker-common` and
`ceph-common` and set basic facts (eg. `fsid`)

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-02 14:46:57 +02:00
Guillaume Abrioux 1d003aa887 merge docker-common and common defaults vars
Merge `ceph-docker-common` and `ceph-common` defaults vars in
`ceph-defaults` role.
Remove redundant variables declaration in `ceph-mon` and `ceph-osd` roles.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-08-02 14:46:51 +02:00
Sébastien Han 511750f31e Merge pull request #1018 from ceph/profile-examples
profiles: introducing cluster profiles
2017-07-28 15:38:06 +02:00
Sébastien Han 050807471a Merge pull request #1723 from ceph/mount-skip
osd: fail check mount partition if not skipped
2017-07-28 14:39:33 +02:00
Sébastien Han 078e5f8e64 Merge pull request #1713 from ceph/ceph-stable
common: make ceph_stable_release mandatory
2017-07-27 17:23:20 +02:00
Sébastien Han 26f4752bc4 common: only add a daemon section if we run on the host
We don't want to have heterogeous ceph.conf anymore and believe that we
should have the right section for the running daemon.
If we don't do this and use profiles, e.g: rgw, we will get a new rgw
section on some of the nodes.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-27 16:34:17 +02:00
Sébastien Han af431a0802 Merge pull request #1596 from czunker/restart_with_crush_location
Restart OSDs during initial setup when crush location is used
2017-07-27 14:22:45 +02:00
Sébastien Han 63cbcc8260 osd: fail check mount partition if not skipped
We forgot to handle the case where "check if any of the raw partitions
are mounted" task gets skipped.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-27 11:39:19 +02:00
Sébastien Han da4cce2c74 common: make ceph_stable_release mandatory
It is mandatory now to set the Ceph version you want to install, e.g:
ceph_stable_release: luminous

To find the release names, you can look at the release not doc:
http://docs.ceph.com/docs/master/release-notes/

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-27 09:49:21 +02:00
Sébastien Han 8ac7d2e4c9 osd: do not enable osd@id unit file
ceph-disk is responsable for enabling the unit file if needed. Actually
since https://github.com/ceph/ceph/pull/12241 it seems that it's not
even needed. On an event of a restart, udev rules will be trigger and
they will ceph-disk activate the device too so the 'enabled' is not
needed.

Closes: https://github.com/ceph/ceph-ansible/issues/1142
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-26 17:17:57 +02:00
Christian Zunker 381600a092 Restart OSDs during initial setup when crush location is used
OSDs get started by ceph-disk before the ceph.conf file is written
with a crush location. That results in a crush map without configured
crush location.

To prevent this, we have to restart the OSDs during the initial setup
after the crush location was added to the ceph.conf file.
2017-07-25 12:06:12 +02:00
Guillaume Abrioux cfd7ae87e4 Merge pull request #1707 from ceph/admin-ker-perms
mon: add mgr cap to admin key
2017-07-24 17:06:59 +02:00
Sébastien Han 2566db3e7c mon: add mgr cap to admin key
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-24 16:00:06 +02:00
Sébastien Han afa29889c1 ceph-docker-common: update etcd kv port
New registered port is 2379

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-24 15:40:44 +02:00
Sébastien Han 33c1f0cb03 osd: refactor osd scenarios
We have multiple issues with ceph-disk's cli with bluestore and Ceph
releases. This is mainly due to cli changes with Luminous. Luminous
introduced a --bluestore and --filestore options which respectively does
not exist on releases older than Luminous. The default store being
bluestore on Luminous, simply checking for the store is not enough so we
have to build a specific command line for ceph-disk depending on the
Ceph version we are running and the desired osd_store.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-24 13:48:08 +02:00
John Fulton 73633f05a0 Allow user to define ACLs for OpenStack keys
The keys and openstack_keys structure now supports an optional
key called acls whose value is a list of strings one could pass
to setfacl. The ansible ACL module applies the ACLs to all
openstack keys with this property.

Fixes: #1688
2017-07-20 13:20:16 +00:00
Guillaume Abrioux 07fb79ce13 Mon: Copy openstack keyring files on all mons
Copies all created openstack keys on all mons.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-19 16:28:18 +00:00
Guillaume Abrioux 233696d1b1 Common: Add profiles
Remove `rgw enable static website` and `rgw enable usage log` from
ceph.conf and make it usable with ceph_config_overrides as profiles.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-19 11:50:03 +02:00
Guillaume Abrioux 894df4b8c8 Common: Avoid duplicate section in ceph.conf
See:
 - https://github.com/ceph/ceph-ansible/pull/1018#issuecomment-253491094
 - https://github.com/ceph/ceph-ansible/pull/1018#issuecomment-260276588

The workaround here is rendering `ceph_conf_overrides` before passing it
to `config_template` to be sure we won't have a section added twice in
ceph.conf

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-19 11:50:03 +02:00
Sébastien Han fcba2c801a profiles: introducing cluster profiles
This commit introduces a new directory called "profiles" which
contains some set of variables for a particular use case. These profiles
provide guidance for certain scenarios such as:

* configuring rgw with keystone v3

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-19 11:50:03 +02:00
Alfredo Deza 92fc34eb9f Merge pull request #1693 from batrick/debian-obsolete-key
remove obsolete debian devel repo key
2017-07-18 15:08:33 -04:00
Patrick Donnelly 8800a23c38
remove obsolete debian devel repo key
According to Alfredo, this was used for gitbuilders. Right now shaman/chacra
dev repos are unsigned.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
2017-07-18 08:47:47 -07:00
Guillaume Abrioux 151d59db4e Mon: Create rbd pool on luminous
There is no more an rbd pool that comes with luminous.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-18 01:01:53 +02:00
Guillaume Abrioux 14d2d08340 Docker-common: Make `fsid` available for all roles.
Move condition at task level and not at include level to make `fsid`
variable available for all roles.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-17 21:11:01 +02:00
Guillaume Abrioux d11975800c Common: change path for fetch_directory
Some tasks fetch file to `{{ fetch_directory }}/docker_mon_files` and
then try to copy from `{{ fetch_directory }}/{{ fsid }}`. That causes
the playbook to fail.

Fixes: #1683

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-17 21:11:01 +02:00
Guillaume Abrioux d0311c6aa3 Client: keep consistency between `openstack_key` and `keys`
To keep consistency between `{{ openstack_keys }}` and `{{ keys }}`
respectively in `ceph-mon` and `ceph-client` roles.
This commit also add the possibility to set mds caps.

Fixes: #1680

Co-Authored-by: John Fulton <johfulto@redhat.com>
Co-Authored-by: Giulio Fidente <gfidente@redhat.com>
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-17 21:11:01 +02:00
yanyx 7e56b5c531 ceph-osd: when ceph relase >= luminous add --filestore config 2017-07-14 09:53:59 +08:00
Andrew Schoen 4e87c1f0f5 ceph-common: make sure monitor_interface or monitor_address is defined
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-07-12 16:34:41 +02:00
Andrew Schoen a9a3d24a3d ceph-common: fixes bug related to monitor_interface set in inventory
The ceph.conf template needs to look for the value of monitor_interface
in hostvars[host] because there might be different values set per host.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2017-07-12 16:34:41 +02:00
Guillaume Abrioux 30a0fa31e3 Docker: Fix bug "waiting for /dev/XXX to show up"
Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-12 15:02:39 +02:00
Guillaume Abrioux 0a38bfaadc Osd: Fix bug 'uniq' command not found
Due to a breaking space introduced by d2320e412e the command here is
broken.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-12 15:02:39 +02:00
Guillaume Abrioux 94c3756167 Tests: Add bluestore scenarios
Since we started testing against Luminous, we need to add more scenarios
testing.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-12 15:02:32 +02:00
Guillaume Abrioux a517ab5583 Osd: Force filestore and bluestore usage
In Luminous, ceph-disk defaults to bluestore so all our scenarios are
using bluestore, we need to force testing both.

Signed-off-by: Sébastien Han <seb@redhat.com>
Co-Authored-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-12 11:30:30 +02:00
Sébastien Han f678b11d3d Merge pull request #1663 from ceph/add_docker_custom_admin_secret
Doc: containerized deploy with custom admin secret
2017-07-07 16:46:03 +02:00
Guillaume Abrioux da755cb317 Doc: containerized deploy with custom admin secret
In addition to ceph/ceph-docker@69d9aa6, this explains how to deploy a
containerized cluster with a custom admin secret.
Basically, just need to pass the `admin_secret` defined in your
`group_vars/all.yml` to the `ceph_mon_docker_extra_env` variable.

Eg:

`ceph_mon_docker_extra_env: -e CLUSTER={{ cluster }} -e FSID={{ fsid }}
-e MON_NAME={{ monitor_name }} -e ADMIN_SECRET={{ admin_secret }}`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-07 16:29:56 +02:00
Sébastien Han 0f1f4388da Merge pull request #1644 from fullerdj/wip-djf-check-devices
osd: validate devices variable input
2017-07-07 15:48:39 +02:00
Sébastien Han 5245473b3c Merge pull request #1662 from fullerdj/wip-djf-install-rsync
common: ensure rsync is installed for local install
2017-07-07 15:46:22 +02:00
Douglas Fuller e5d06a449f osd: validate devices variable input
Fail with a sane message if the devices or raw_journal_devices variables
are strings instead of lists during manual device assignment.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-07-07 13:37:29 +00:00
Douglas Fuller 79ba50cd9f common: ensure rsync is installed for local install
rsync is required by the ansible synchronize package. Ensure
it is installed when local installation is selected.

Signed-off-by: Douglas Fuller <dfuller@redhat.com>
2017-07-06 17:29:43 +00:00
Sébastien Han 035846217e Merge pull request #1627 from ceph/ceph-osd-prepare-script
osd: docker, refactor ceph-osd-run.sh.j2
2017-07-06 16:08:59 +02:00
Sébastien Han d2320e412e osd: docker, refactor ceph-osd-run.sh.j2
Easier to read and enhance.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-06 15:49:14 +02:00
Guillaume Abrioux 7e1df484db Mon: Allow to deploy with custom admin secret
Add a new parameter `admin_secret` that allow to deploy a ceph cluster
with a custom admin secret.

Fix: #1630

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2017-07-05 14:57:49 +02:00
Sébastien Han e7ae65b6f9 Merge pull request #1632 from ceph/bluestore-redundant-options
osd: remove redundant options to enable bluestore
2017-07-04 19:33:54 +02:00
Sébastien Han 7d657ac643 osd: ability to set db and wal to bluestore
This commits refactors how we deploy bluestore. We have existing
scenarios that we don't want to change too much. This commits eases the
user experience by now changing the way you use scenarios. Bluestore is
just a different interface to store objects but the scenarios more or
less remain the same.

If you set osd_objectstore == 'bluestore' along with
journal_collocation: true, you will get an OSD running bluestore with DB
and WAL partitions on the same device.

If you set osd_objectstore == 'bluestore' along with
raw_multi_journal: true, you will get an OSD running bluestore with a
dedicated drive for the rocksdb DB, then the remaining
drives (used with 'devices') will have WAL and DATA collocated.

If you set osd_objectstore == 'bluestore' along with
raw_multi_journal: true and declare bluestore_wal_devices you will get
an OSD running bluestore with a dedicated drive for rocksdb db, a
dedicated drive partition for rocksdb WAL and a dedicated drive for
DATA.

Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-04 19:07:16 +02:00
Sébastien Han adf752f241 Merge pull request #1597 from czunker/openstack_pools_with_rule
Create OpenStack pools with crush rule
2017-07-04 18:34:00 +02:00
Sébastien Han fc0e54c59e osd: remove redundant options to enable bluestore
There is no need for 2 variables to enable bluestore, prior to this
patch one had to do the following to activate bluestore:

osd_objectstore: bluestore
bluestore: true

Now you just need to set `osd_objectstore: bluestore`.

Fixes: https://github.com/ceph/ceph-ansible/issues/1475
Signed-off-by: Sébastien Han <seb@redhat.com>
2017-07-04 18:22:03 +02:00