Commit Graph

372 Commits (b4f14aba8e314beaa9c5a84d617e4973b7b599c5)

Author SHA1 Message Date
Rishabh Dave 62abe7068a use os.path.join() correctly
os.path.join adds the separator (i.e. '/') between the provided path
components only if needed. Providing a single path component doesn't
lead to any checks.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-03-14 22:35:12 +00:00
Guillaume Abrioux 3d3eee8f38 tests: add symlink for ubuntu hosts inventory
otherwise a bunch of jobs will fail like following:

```
 [WARNING]: Unable to parse /home/jenkins-build/build/workspace/ceph-ansible-nightly-luminous-ubuntu-container-stable-3.2-bluestore_lvm_osds/tests/functional/bs-lvm-osds/container/hosts-ubuntu as an inventory source
 [WARNING]: No inventory was parsed, only implicit localhost is available
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-03-04 18:05:26 +01:00
Guillaume Abrioux 207fae38d4 tests: add lvm bluestore dmcrypt support
Add coverage for container / non container lvm bluestore dmcrypt OSDs

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-28 12:01:18 +00:00
Guillaume Abrioux 4ab02d2cd1 tests: set ceph_origin and ceph_repository for non_container-collocation
those variables are mandatory.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-27 15:58:35 +00:00
Guillaume Abrioux 7fd92348bb tests: add mgr node for all_daemons scenario
add a monitor node to cover in the CI the case where mgrs and monitors
are not collocated

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-26 13:19:06 +00:00
Guillaume Abrioux fa13289c65 tests: fix network interfaces names in conftest.py
Set network interfaces names according to the OS distribution in
conftest.py

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-22 16:24:18 +01:00
Guillaume Abrioux 2ed203da61 Revert "tests: add ubuntu bionic support"
This reverts commit 33c09af250.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-22 16:24:18 +01:00
Guillaume Abrioux 33c09af250 tests: add ubuntu bionic support
This commit brings all modifications needed to test against ubuntu
bionic.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-22 09:20:16 +01:00
Guillaume Abrioux f80e43a0d8 tests: share fixture instance across the whole module.
there's no need to rerun this part of the code for each function test.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-19 16:40:08 +01:00
Guillaume Abrioux 187b2bc9d9 tests: avoid 'Cannot allocate memory' error in testinfra
```
------------------------------ Captured log setup ------------------------------
display.py                 174 INFO     Wednesday 13 February 2019  15:54:15 +0000 (0:00:07.787)       0:02:11.607 ****
ansible.py                  61 INFO     RUN Ansible('setup', None, {'check': True, 'become': False}): {'_ansible_no_log': False,
 '_ansible_parsed': False,
 '_ansible_verbose_override': True,
 'changed': False,
 'module_stderr': u'Connection to 192.168.121.87 closed.\r\n',
 'module_stdout': u'bash: /bin/sh: Cannot allocate memory\r\n',
 'msg': u'MODULE FAILURE\nSee stdout/stderr for the exact error',
 'rc': 126}
```

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-14 10:34:37 +00:00
Guillaume Abrioux ac4aded4aa tests: fix ubuntu-container-all_daemons
the public_network subnet used for this scenario was wrong.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-14 10:34:37 +00:00
Guillaume Abrioux ac7f4b3a01 tests: increase amount of memory for all vms
double the amount of memory from 512m to 1024m.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-05 18:14:28 +01:00
Guillaume Abrioux ff5509295a tests: remove useless test
`test_mon_host_line_has_correct_value()` will cover this test in
anycase. It doesn't worth to have a dedicated test for this.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-05 18:14:28 +01:00
Guillaume Abrioux efc051d17c tests: update test_mon_host_line_has_correct_value()
since msgr2 introduction, this test must be updated.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-05 18:14:28 +01:00
Guillaume Abrioux 0d72fe9b30 tests: add a rhel8 scenario testing
test upstream with rhel8 vagrant image

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-02-05 18:14:28 +01:00
Andrew Schoen e0dcd9f2c7 tests: fix Vagrantfile symlink for lvm-auto-discovery tests
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2019-02-01 12:28:12 +01:00
Andrew Schoen fc9502039d tests: adds the lvm_auto_discovery container testing scenario
This tests osd_auto_discovery: True, containerized_deployment: True
and the lvm osd scenario

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
2019-02-01 12:28:12 +01:00
Guillaume Abrioux c0ad91957c tests: add missing hosts file for ubuntu testing
we need it for dev-ubuntu-container-all_daemons job

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-31 08:19:17 +01:00
Guillaume Abrioux fd8856b80a tests: do not deploy iscsigw node on ubuntu
iSCSI gateways can only be deployed on Red Hat OS family.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-31 08:19:17 +01:00
Guillaume Abrioux 02b18b15c0 tests: run lvm_setup.yml only when osd_scenario is lvm
especially for ooo_collocation scenario which is still using ceph-disk
testing.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-31 08:19:17 +01:00
Ramana Raja dfff89ce67 Install nfs-ganesha stable v2.7
nfs-ganesha v2.5 and 2.6 have hit EOL. Install nfs-ganesha v2.7
stable that is currently being maintained.

Signed-off-by: Ramana Raja <rraja@redhat.com>
2019-01-30 14:57:26 +01:00
Guillaume Abrioux 25d8198c5d tests: do not play dev_setup on containerized deployment
using `!` mark in tox.ini doesn't work on comma separated list.
The idea here is to skip all containerized scenario in dev_setup.yml and
use the `!` for the update scenario.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-29 14:12:05 +00:00
Guillaume Abrioux 312867af56 tests: fix rgw testinfra failure
fix the wrong path used in various rgw testinfra tests.
set `1` as default value for `radosgw_num_instances`: if
`ansible_vars.get(radosgw_num_instances)` returns `None`, we can assume
there's only 1 instance since it's the default value in ceph-defaults.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-25 11:11:16 +00:00
Guillaume Abrioux 973f316595 tests: fix symlink to Vagrantfile
fix the symplink for Vagrantfile in rgw-multisite/container/secondary

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-24 11:46:32 +01:00
Guillaume Abrioux 5ccfa95b9a tests: update dev_setup for nfs_ganesha_stable value
since we now set this variable in inventory host, the regexp needs to be
updated, the assignment operator is `=`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-24 11:46:32 +01:00
Guillaume Abrioux d177711f6e tests: rename all node
let's name all node the same way to avoid confusion.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-24 11:46:32 +01:00
Guillaume Abrioux e6205e0070 tests: change ceph_osd_docker_run_script_path for add_osds
set `ceph_osd_docker_run_script_path` to /var/tmp for `add_osds`
scenario

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-24 11:46:32 +01:00
Guillaume Abrioux 7d705395fb tests: play lvm_setup.yml on all scenarios
We should play lvm_setup.yml on all scenario except `lvm_batch`.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-24 11:46:32 +01:00
Guillaume Abrioux 6293a98a0c tests: reorganize directories layout
This commit reorganizes the testing directory layout.
The idea is to have more consistency with the names of scenario and
their corresponding path, eg: non-container vs. container: each scenario
has a subdirectory for container deployment.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-24 11:46:32 +01:00
Guillaume Abrioux fd91096215 tox: refact environment naming
this commit refacts the way the environment are named by adding a factor
`{non_container,container}`. This will avoid a lot of duplicate
definition in tox.ini and bring kind of consistency.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-24 11:46:32 +01:00
Rishabh Dave 557d44451d tests: fix identation issue in test_rgw.py
The error was introduced by the commit
1ac94c048f.

Fixes: https://github.com/ceph/ceph-ansible/issues/3528
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2019-01-23 14:50:41 +01:00
guihecheng 1ac94c048f rgw: add support for multiple rgw instances on a single host
With this, we could have multiple rgw instances on a single host
with a single run, don't have to use rgw-standalone.yml which does not
seems able to bind ports separately.
If you want to have multiple rgw instances, just change 'radosgw_instances'
to the number you want, which defaults to 1.
Not compatible with Multi-Site yet.

Signed-off-by: guihecheng <guihecheng@cmiot.chinamobile.com>
2019-01-18 11:12:28 +01:00
Guillaume Abrioux b94290af43 refact the 'raw' installation of python
to avoid duplicating code in `site.yml.sample`, `site-docker.yml.sample`
and `setup.yml`, let's isolate this part of the code and simply include
it each time we need it.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2019-01-16 10:16:11 +01:00
Alfredo Deza 9e4ec1a776 testinfra/osds double the amount of ports OSDs listen to
Since msgr2 changes got merged, the OSDs in master (to be nautilus) will
double the amount of ports they listen to.

Signed-off-by: Alfredo Deza <adeza@redhat.com>
2019-01-08 13:40:34 +01:00
Guillaume Abrioux d7e77012ef retry on packages and repositories failures
add register/until on all packaging related tasks to avoid non valid CI
failure.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2018-12-19 14:48:27 +00:00
Sébastien Han 14cd286e3a test: disable nfs for containers
Based on https://github.com/ceph/ceph-container/pull/1269 and given
there are no stable packages and reliable repository, we disable nfs
ganesha temporarly.

Signed-off-by: Sébastien Han <seb@redhat.com>
2018-12-04 12:34:54 +01:00
Sébastien Han 1b6b275229 test: remove leftover [mgrs]
Since we now collocated mgrs and mons on the same machine we have to
remove the mgrs section, they are not needed anymore.

Signed-off-by: Sébastien Han <seb@redhat.com>
2018-12-04 12:34:54 +01:00
Sébastien Han 1c760904b0 site: collocated mon and mgr by default
This will speed up the deployment and also deploy mon and mgr collocated
just as recommended.
This won't prevent you of adding more and dedicaded machines for mgr if
needed.

Signed-off-by: Sébastien Han <seb@redhat.com>
2018-12-03 14:39:43 +01:00
Sébastien Han a502327e52 disable nfs scenario
The packages are broken, so let's remove it, until this solved.

Signed-off-by: Sébastien Han <seb@redhat.com>
2018-12-03 14:39:43 +01:00
Guillaume Abrioux 5d05a09b03 tests: update default pg num and pool size for podman scenario
bring the recent refact about `osd_pool_default_pg_num` and
`osd_pool_default_size` into podman scenario as well.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2018-11-28 11:22:04 +00:00
Sébastien Han 4e5d862bb7 testinfra: linting
Make flake8 happy on the testinfra files.

Signed-off-by: Sébastien Han <seb@redhat.com>
2018-11-27 16:47:40 +00:00
Sébastien Han dcc765d7c7 testinfra: add support for podman
Since we are now testing on docker and podman our functionnal tests must
reflect that. So now, if we detect the podman binary we will use it,
otherwise we default to docker.

Signed-off-by: Sébastien Han <seb@redhat.com>
2018-11-27 16:47:40 +00:00
Sébastien Han a96e910114 Add new container scenario
Test with podman instead of docker and also support for python 3 only.

Signed-off-by: Sébastien Han <seb@redhat.com>
2018-11-27 16:47:40 +00:00
Guillaume Abrioux f290e49df8 tests: do not fully override previous ceph_conf_overrides
We run an initial deployment with `osd_pool_default_size: 1` in
`ceph_conf_overrides`.
When re-running the playbook to test idempotency and handlers, we reset
`ceph_conf_overrides`, we must append a new value instead of just
overwritting it, otherwise, this can lead to error in the CI.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2018-11-26 18:22:20 +01:00
Guillaume Abrioux 5601af8de2 tests: change default pools size
default pool size in our test should be explicitly set to 1

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2018-11-21 18:23:07 +00:00
Guillaume Abrioux d4c0960f04 mon: move `osd_pool_default_pg_num` in `ceph-defaults`
`osd_pool_default_pg_num` parameter is set in `ceph-mon`.
When using ceph-ansible with `--limit` on a specifc group of nodes, it
will fail when trying to access this variables since it wouldn't be
defined.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1518696

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2018-11-21 15:42:50 +00:00
Guillaume Abrioux 3ac6619fb9 tests: set pool size to 1 in ceph-override.json
setting this setting to 1 makes the CI covering the related code in the
playbook without breaking the upgrade scenarios.

Those scenarios were broken because there is a check `TASK [waiting for
clean pgs...]` in rolling_update.yml, since the pool size for
`cephfs_metadata` and `cephfs_data` are updated to `2` in
`ceph-override.json` and there is not enough osd to honor this size,
some PGs are degraded and make the mentioned check failing.

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
2018-11-12 10:51:48 +01:00
Sébastien Han e552026418 rbd-mirror: use the new rbd-mirror key
Instead of using the old rbd key let's use the new rbr-mirror key to
bootstrap the rbd -mirror daemon.

Signed-off-by: Sébastien Han <seb@redhat.com>
2018-11-09 12:45:52 +01:00
Noah Watkins 50255b9640 Fixup shrink_osd[_container] scenario config
** configuration seems to be for filestore:

[ERROR]: [ceph-osd0] Validation failed for variable: lvm_volumes

** Removing `radosgw_interface: eth1` to resolve:

The task includes an option with an undefined variable. The error was:
'ansible.vars.hostvars.HostVarsVars object' has no attribute
u'ansible_eth1'

The error appears to have been in
'/home/nwatkins/src/ceph-ansible/roles/ceph-defaults/tasks/set_radosgw_address.yml':
line 21, column 5, but may be elsewhere in the file depending on the
exact syntax problem.

The offending line appears to be:

  - name: set_fact _radosgw_address to radosgw_interface - ipv4
    ^ here

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2018-11-08 17:45:37 +01:00
Rishabh Dave 8edbda96df use blocks directives to group tasks
Using block directives simplifies the playbooks and makes them more
readable.

Fixes: https://github.com/ceph/ceph-ansible/issues/2835
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2018-10-31 09:37:43 +01:00