Commit Graph

5842 Commits (12c700c1b02ea2574b0211bd016df64a05b47a13)
 

Author SHA1 Message Date
Johannes Kastl 4711a7d626 fix openSUSE OBS repo creation
roles/ceph-common/tasks/installs/suse_obs_repository.yml:
ansible's zypper_repository module does not know a parameter 'uri', this is
called 'repo' instead

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-29 10:23:07 -04:00
Nick Erdmann 7953ee1b81 ceph-infra: open ceph iscsi/prometheus port
Signed-off-by: Nick Erdmann <n@nirf.de>
2019-08-28 16:09:55 -04:00
Dimitri Savineau 825045f6b4 tests: use a single grafana node on podman
We don't use multiple grafana nodes for the moment on the others
scenarios and I don't think this is supposed to be working.
We can often see failure on grafana on that scenario.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-28 11:42:48 -04:00
Guillaume Abrioux e0ba011f50 tests: change container image tag for switch_to_containers
test switch_to_containers job against the latest ceph@master
ceph-container image tag available.
In order to be sure the ceph release deployed in the first step (non
containerized deployment) isn't newer than the tag used for the
containerized migration (which would mean we try to downgrade the
version).

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-28 10:48:21 -04:00
Johannes Kastl bd507fa147 set discovered_python_interpreter if ansible_python_interpreter is defined
If the user has set the `ansible_python_interpreter`, ansible will not try to
discover python, so `discovered_python_interpreter` will not be set.

Solution: Set `discovered_python_interpreter` to `ansible_python_interpreter`
if `ansible_python_interpreter` is defined

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-27 20:54:59 +02:00
Dimitri Savineau 2b0616ecca ceph-mon: Bind mount the ca-trust directory
On containerized deployment, the mon container sometimes needs to
access to the radosgw endpoint (via the radosgw-admin command). When
using TLS on the radosgw with self-signed certificates then we need to
access to the CA certification from the mon container.
The CA certificate needs to be added on the host and then the directory
will be bind mount on the container.

Resolves: #4358

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-27 20:53:45 +02:00
Dimitri Savineau 49aa05b96c ceph-client: Use profile rbd in keyring caps
Like the OpenStack keyrings, we can use the profile rbd for the clients
keyring (both mon and osd).

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-27 20:52:23 +02:00
Dimitri Savineau 717af83475 Revert "osd: add 'osd blacklist' cap for osp keyrings"
This reverts commit 2d955757ee.

The "osd blacklist" isn't an osd caps but should be used with mon caps.
Also the correct caps for this is: 'allow command "osd blacklist"'.
The current change is breaking the openstack and clients keyrings.
By using the profile rbd (which is already used) we already rely on the
ability to blacklist dead client.

Resolves: #4385

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-27 20:52:23 +02:00
Guillaume Abrioux 5986b26a01 global: add newline at end of file
This commit re-add a newline at end of files when it's missing.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-23 15:56:47 +02:00
Artur Fijalkowski 011270ca69 global: make directories mode parameterizable
This commit makes it possible to parametrize the ceph directories modes.
So it changes hardocded mode for ceph related directories from 0755 to
customizable with `ceph_directories_mode` variable.

Closes: #2920

Signed-off-by: Artur Fijalkowski <artur.fijalkowski@ing.com>
Co-authored-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-23 09:38:17 +02:00
guihecheng a0590cae9d rgw/multisite: assign 'rgw_zone' to the exact section in ceph.conf
since the following commit:
  commit 1ac94c048f
  rgw: add support for multiple rgw instances on a single host

we have multi-instance rgw support on a single host and
the config section name of the rgw changed from
[client.rgw.$(hostname)] -> [client.rgw.$(hostname).rgwX]
when X is the sequence number: 0,1,2,...
So we should assign 'rgw_zone' item to the exact rgw instance
config section in ceph.conf

Signed-off-by: guihecheng <guihecheng@cmiot.chinamobile.com>
2019-08-23 08:14:10 +02:00
Guillaume Abrioux 327d564106 lint: fix error [301], add `changed_when: false` when needed
This commit fixes the error [301]:

`[301] Commands should not change things if nothing needs doing`

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-23 00:23:47 +02:00
Guillaume Abrioux 102edaeb61 lint: fix error [306], add pipefail on shell command using pipe
This commit fixes the error [306]:

`[306] Shells that use pipes should set the pipefail option`

using `/bin/bash` as executable because Debian/Ubuntu systems use `dash`
by default which doesn't have the `-o pipefail`. (See:
https://github.com/ansible/ansible-lint/issues/497#issue-424623501)

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-23 00:23:47 +02:00
Johannes Kastl f6b6f7d5b0 plugins/actions/validate.py: allow ceph_repository 'obs' on openSUSE
Allow the use of 'obs' as a valid value for ceph_repository, and validate that
- OS is openSUSE
- ceph_obs_repo is defined

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-22 20:23:15 +02:00
Johannes Kastl efd38ecc88 ceph-validate: Refactor check for installation check on SUSE/openSUSE
Move the validation from roles/ceph-common/tasks/installs/install_on_suse.yml
to roles/ceph-validate/ and fix the syntax.

There are two valid combinations of `ceph_origin` and `ceph_repository` on
SUSE/openSUSE:
- ceph_origin == 'distro'
- ceph_origin == 'repository' and ceph_repository == 'obs'

The current when condition would fail even in the valid second combination,
as ceph_origin != distro would be true then

Fixes: #4362

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-22 20:22:13 +02:00
Johannes Kastl e1b9312084 facts: fix a typo
This commit fixes a typo in roles/ceph-facts/tasks/facts.yml

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-22 18:08:28 +02:00
Kevin Coakley e11cbbbcb1 ceph-config: Set changed_when to false on fact gathering statements
The "run 'ceph-volume lvm batch --report' to see how many osds are to be
created" and "run 'ceph-volume lvm list' to see how many osds have already been
created" statements only register the lvm_batch_report and lvm_list variables.
Running those ceph-volume commands should never produce a change on the system.
Adding changed_when: false prevents irrelevant change messages from Ansible.

Signed-off-by: Kevin Coakley <kcoakley@sdsc.edu>
2019-08-22 17:27:58 +02:00
Johannes Kastl 8e3511ddc7 fix SUSE/openSUSE naming
As SUSE 15.x and openSUSE Leap 15.x share the same base, make clear
that both are targeted by the respective tasks

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-22 17:20:21 +02:00
Johannes Kastl cdbe958e55 roles/ceph-validate/tasks/check_system.yml: fail on unsupported SUSE versions
Fail if SUSE distributions other than 15.x are found, similar to what we have
for openSUSE

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-22 17:17:21 +02:00
Dimitri Savineau 9a4ac46d19 ceph-osd: Add ulimit nofile on container start
On containerized deployment, the OSD entrypoint runs some ceph-volume
commands (lvm/simple scan and/or activate) which perform badly without
the ulimit option.
This option was added for all previous ceph-volume commands but not on
the ceph-osd container startup.
Also updating hard limit value to 4096 to reflect default baremetal
value.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-22 16:59:08 +02:00
Kevin Jones 47bf47c9d8 Set proper ownership command performance improvement
By changing the set ownership command from using the file module in combination with a with_items loop to a raw chown command, we can achieve a 98% performance increase here.

On a ceph cluster with a significant amount of directories and files in /var/lib/ceph, the file module has to run checks on ownership of all those directories and files to determine whether a change is needed.

In this case, we just want to explicitly set the ownership of all these directories and files to the ceph_uid

Added context note to all set proper ownership tasks

Signed-off-by: Kevin Jones <kevinjones@redhat.com>
2019-08-22 10:26:47 +02:00
Johannes Kastl 11aa5dbb58 ceph-nfs: fail on openSUSE Leap using distro packages
roles/ceph-validate/tasks/check_nfs.yml: fail on openSUSE Leap
using `ceph_origin = distro`, as the ganesha packages are not available from
the distribution repositories

Fixes: #4342

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-21 09:58:54 +02:00
Johannes Kastl c721cb99cb install ceph-mds packages on SUSE/openSUSE
install packages on SUSE/openSUSE distributions, using the
same logic as on RedHat-based distributions

Fixes #4340

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-21 09:57:56 +02:00
Guillaume Abrioux 9329bbb3af handler: do not validate the server certificate against the CA
Otherwise rgw handler ends up with an error when using https.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-20 13:52:15 +02:00
Johannes Kastl 504017d562 remove duplicate task installing suse dependencies
roles/ceph-common/tasks/installs/install_on_suse.yml: remove the task that
installs the dependencies, as this is done later in install_suse_packages.yml

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-20 12:59:25 +02:00
Guillaume Abrioux 243edfbc96 validate: do not validate devices or lvm_volumes in osd_auto_discovery case
we shouldn't validate these two variables when `osd_auto_discovery` is
set.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1644623

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-20 09:12:20 +02:00
Guillaume Abrioux 70cf2a5846 osd: remove useless condition
just like `ceph_osd_pool_default_size`, a pool size might change after an
initial deployment. Having this condition prevents from customizing the
pool in that case.
This is not needed so let's remove it.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-19 16:17:22 +02:00
Guillaume Abrioux a08f5f2ea2 mergify: disable automatic merging on master
automatic merging by mergify is failing for a while now.
Until we can figure out what's wrong, let's disable it on master for now
so we don't merge "failing" PRs although they passed all scenarios.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-19 12:32:56 +02:00
Guillaume Abrioux c49b8d04c7 doc: update backport section
Only maintainers can set labels on PRs, so let's clarify that point in
the doc which says something confusing at the moment.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-14 16:42:25 +02:00
Guillaume Abrioux 5e33d224d3 tests: tests switch_to_containers against octopus
since we have container images for ceph@master, we shouldn't use
nautilus anymore.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-14 16:42:02 +02:00
Guillaume Abrioux 4df92152c0 common: replace shell module
there is no need to use `shell` in these tasks. Let's use `command`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-14 16:42:02 +02:00
Guillaume Abrioux 5573f17e76 shrink-mon: refact 'verify the monitor is out of the cluster' task
use `from_json` filter instead of a `| python` so we can get rid of the
`shell` module usage here.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-14 16:42:02 +02:00
Guillaume Abrioux 687087fd43 osd: refact 'wait for all osd to be up' task
let's use `until` instead of doing test in bash using python oneliner
also, use `command` instead of `shell`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-14 16:42:02 +02:00
Guillaume Abrioux 13815ad3ca common: use discovered_interpreter_python fact
in order to use the right binary name when using python cli in command
or shell module.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-14 16:42:02 +02:00
Guillaume Abrioux 05686509f3 tests: update test_mgr_is_up()
the data structure has changed in octopus:

```
    "mgrmap": {
        "available": true,
        "modules": [
            "dashboard",
            "prometheus"
        ],
        "num_standbys": 0,
        "services": {
            "prometheus": "http://mgr0:9283/"
        }
    },
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-14 16:42:02 +02:00
Guillaume Abrioux a5e359ee80 osd: update the check for 'all osd to be up'
the data structure has changed in octopus.
eg: the path to `num_osds` is now `["osdmap"]["num_osds"]`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-14 16:42:02 +02:00
Guillaume Abrioux d3fa3c2d72 refact python installation
This commit refacts the python installation when no available.

In order to avoid generating errors, we check for each package manager
to detect which system we are running on.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-14 16:42:02 +02:00
Igor 2fdf7316a4 tests: fix wrong paths for lv-create in tox.ini
solution: change paths inside tox.ini file
Fixes: #4311
Signed-off-by: Bogomolov Igor <igor95n@gmail.com>
2019-08-08 13:55:41 +02:00
Dimitri Savineau 31bd5e08a6 Revert "tests: disable nfs-ganesha deployment"
This reverts commit 83940e624b.

Because nfs-ganesha@master (2.9-dev) build has been fixed by [1] then
we can test nfs-ganesha in the CI for master/octopus.

[1] https://github.com/ceph/ceph-build/pull/1346

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-07 10:40:43 +02:00
Guillaume Abrioux 5b9b841108 mgr: refact 'wait for all mgr to be up' task
There's no need to use `shell` module here.
Instead of using `| python -c`, let's use `from_json` filter.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-07 10:33:54 +02:00
Dimitri Savineau 4c6ec1dccb mgr/dashboard: Fix grafana/prometheus url config
When configuring grafana/prometheus embed in the mgr/dashboard, we need
to use the address of the grafana-server node and not the current
hostname because mgr/dashboard and grafana/prometheus could be present
on different hosts.
We should instead rely on the grafana_server_addr variable and remove
the dashboard_url.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-06 09:34:20 +02:00
Dimitri Savineau 16939eff9e dashboard: run dashboard role on mgr/mon nodes
We don't need to execute the ceph-dashboard role on the nodes present
in the grafana-server group. This one is dedicated to the grafana and
prometheus stack.
The ceph-dashboard needs to executed where the ceph-mgr is running. It
is either on the dedicated mgr nodes or if mgr and mon are collocated
implicitly on the mon nodes.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-06 09:34:20 +02:00
Dimitri Savineau f545b5be0d ceph-dashboard: Add run_once on delegate tasks
Because we need to execute commands from a monitor node (the first one
in the mons list) we are using delegate_to option.
If there's multiple nodes running the ceph-dashboard role then the
delegated task will be executed multiple times.
Also remove a mgr config-key option not present for nautilus+ releases.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-06 09:34:20 +02:00
Johannes Kastl 5ee3d96fb4 only support openSUSE Leap 15.x, fail on 42.x
openSUSE switched from 'openSUSE 13.x' to 'openSUSE Leap 42.x' and then to
'openSUSE Leap 15.x' to align with SLES15 development.
The previous logic did not correctly allow the current release, as 15.x matched
the 'less than 42.3' condition.

For now only support openSUSE Leap 15.x, and extend support once 16.x is
released (or whatever the exact version will be)

Signed-off-by: Johannes Kastl <kastl@b1-systems.de>
2019-08-05 09:46:31 -04:00
Dimitri Savineau 771f25b1f8 ceph-infra: Apply firewall rules with container
We don't have a reason to not apply firewall rules on the host when
using a containerized deployment.
The TripleO environments already manage the ceph firewall rules outside
ceph-ansible and set the configure_firewall variable to false.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1733251

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-01 15:16:49 +02:00
Dimitri Savineau 34036c667c ceph-grafana: Set grafana uid/gid on files
We don't need to create a grafana system user (in fact we even don't
set the righ uid to this user) because we're using a container setup.
Instead we just need to be sure to set the owner/group to 472 (grafana
user/group from the container) like we do for ceph/167.
We don't need to set the user/group recursively on /etc/grafana
directory in a dedicated task.
Also on Ubuntu system, the ceph-grafana-dashboards isn't present so on
non containerized deployment we won't have the
/etc/grafana/dashboards/ceph-dashboard directory present (coming with
the package) so we need to be sure it exists.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-08-01 10:10:56 +02:00
Guillaume Abrioux dc7eb535b6 dashboard: do not deploy on Debian based OS/non-containerized
in non-containerized deployment, we can't deploy dashboard on Debian
based distribution since the package `ceph-grafana-dashboards` isn't
available.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-08-01 10:08:41 +02:00
Theo Ouzhinski 7c4e8f0f08 docs: Correct weird wording
for the Ceph master branch.

Signed-off-by: Theo Ouzhinski touzhinski@gmail.com
2019-08-01 10:08:05 +02:00
Dimitri Savineau 867583d5dd tests/shrink_rgw: Disable dashboard
The shrink_rgw scenario has been merge just after the PR about enable
ceph dashboard by default.
So right now the shrink_rgw scenrio doesn't have nodes in the grafana
group and fails.
We just need to set dashboard_enabled to false.

Signed-off-by: Dimitri Savineau <dsavinea@redhat.com>
2019-07-31 14:53:05 -04:00
Guillaume Abrioux 0f620b2584 tests: add more memory in podman job
Typical error :

```
fatal: [mon1 -> mon0]: FAILED! => changed=true
  cmd:
  - podman
  - exec
  - ceph-mon-mon0
  - ceph
  - config
  - set
  - mgr
  - mgr/dashboard/ssl
  - 'false'
  delta: '0:00:00.644870'
  end: '2019-07-30 10:17:32.715639'
  msg: non-zero return code
  rc: 1
  start: '2019-07-30 10:17:32.070769'
  stderr: |-
    Traceback (most recent call last):
      File "/usr/bin/ceph", line 140, in <module>
        import rados
    ImportError: libceph-common.so.0: cannot map zero-fill pages: Cannot allocate memory
    Error: exit status 1
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>
```

Let's add more memory to get around this issue.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-07-30 13:52:44 +02:00