By default, this roles will create a ceph config file and get the admin
key. You can optionnally add other users, keys and pools for your tests.
Closes: #769
Signed-off-by: Sébastien Han <seb@redhat.com>
Add support to allow ceph-ansible to install and
configure Ceph on Debian on the ppc64le architecture.
Canonical has ppc64le Debian packages in Ubuntu distros
and on Ubuntu Cloud Archive. Both of which can be installed
and configured using the 'distro' or 'uca' options in
ceph-ansible when this patch is used.
Signed-off-by: Samuel Matzek <smatzek@us.ibm.com>
Since ##461 we have been having the ability to override ceph default
options. Previously we had to add a new line in the template and then
another variable as well. Doing a PR for one option was such a pain. As
a result, we now have tons of options that we need to maintain across
all the ceph version, yet another painful thing to do.
This commit removes all the ceph options so they are handled by ceph
directly. If you want to add a new option, feel free to to use the
`ceph_conf_overrides` variable of your `group_vars/all`.
Risks, for those who have been managing their ceph using ceph-ansible
this is not a trivial change as it will trigger a change in your
`ceph.conf` and then restart all your ceph services. Moreover if you did
some specific tweaks as well, prior to run ansible you should update the
`ceph_conf_overrides` variable to reflect your previous changes.
To avoid service restart, you need to know a bit of ansible for this,
but generally the idea would be to run ansible on a dummy host to
generate the ceph.conf, then scp this file to all your ceph hosts and
you should be good.
Closes: #693
Signed-off-by: Sébastien Han <seb@redhat.com>
This is purely a refactor. Converts when 'and' conditionals into lists
rather than multiline strings. This does not work for nested
conditionals, but those can be formated with indents.
Moves one line when statements onto the same line as the when command
itself.
A small logic bug was found in ceph-osd/tasks/check_devices.yml which
which was also fixed.
Signed-off-by: Sam Yaple <sam@yaple.net>
This adds a helper fact that uses the ``init_system`` fact to determine if
we should be using systemd or not when controlling services.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
The ceph-osd role currently uses ansible_service_mgr, which is a fact
only available on ansible 2.x and greater. This commit sets a similar
fact called init_system which will store the contents of /proc/1/comm
(systemd, init, etc.) and then references it ceph-osd instead.
Closes#741
This adds support to allow the install of Ceph from the
Ubuntu Cloud Archive. The Ubuntu Cloud Archive provides newer
release of Ceph than the normal Ubuntu distro repository.
Signed-off-by: Samuel Matzek <smatzek@us.ibm.com>
Since developement versions of Ceph are after infernalis a package split
happened. So basically ceph-mon, ceph-osd, ceph-mds need to be
installed.
Signed-off-by: Sébastien Han <seb@redhat.com>
This will allow a user to conditionally install the ceph package on rpm
based systems. Installing this package is not required or wanted in
versions passed infernalis.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Introducing a new config option: `radosgw_civetweb_bind_ip` which points
to the `ansible_default_ipv4` by default. You can override this
variable. Use ansible facts to put a proper value.
Signed-off-by: Sébastien Han <seb@redhat.com>
This fixes the ceph.conf template so that it will look for an inventory
defined value for monitor_interface or for monitor_interface defined in
a group_vars file.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
This fixes a bug where monitor_interface might be set in your inventory
file and not by using group_vars or --extra-vars causing the template to
use the default address of 0.0.0.0 instead of the defined
monitor_interface.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Instead of creating the RBD client socket path three different places
in three different ways, this creates it once. Ceph on OpenStack users
have the option to customize the permissions of the RBD client
directories.
Fixes#687
we now have the ability to enable the `cluster` variable with a specific
value that will determine the name of the cluster.
Signed-off-by: Sébastien Han <seb@redhat.com>
ceph.conf.j2 template requires a new line between mon_containerized_deployment_with_kv and fsid variables
With this commit , i have added a new line for better readablity
ceph.conf file generation task in ceph-common role was getting failed
because it ansible cant find defination of varriable mon_containerized_deployment_with_kv
This fix declare mon_containerized_deployment_with_kv under ceph-common/defaults/main.yml which fixes this issue
Signed-off-by: ksingh7 <karan.singh731987@gmail.com>
Some versions (?) of libvirt provide a 'libvirt' group instead of
'libvirtd'. (Observed with libvirt-daemon-1.2.17-13.el7_2.2.x86_64.)
This makes the RBD client directory owner and group configurable to
allow for this.
This would allow users who don't know what interface to provide to
give an IP address to use for the monitor instead.
Note: the includes are needed in ceph.conf.j2 because without them
jinja2 can not properly evaluate the template and will complain about a
missing 'ansible_interface' variable. The includes allow the template to
be evaluated correctly and then the correct include will be used during
render time.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Where it was located before meant it might be skipped if you don't run
tasks with the package-install tag. This fixes the situation where you
want to configure an rhcs node, but do not want to do any package
installs.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
When installing RHCS there is an option to install from distro provided
packages, this commit modifies the check to allow that to happen.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
changing the name of the directory causes issues with git subtree which
will create new commits. Creating a symlink for vagrant to be happy.
Signed-off-by: Sébastien Han <seb@redhat.com>
in order to have a build on the galaxy we need to have a proper
dependency set for ceph-common. On the galaxy ceph-common does not
exist, only ceph.ceph-common is available.
Signed-off-by: Sébastien Han <seb@redhat.com>
this commit introduces the ability to use fqdn for mon/mds name while
generation the ceph.conf file from the template.
Simply turn mon_use_fqdn and or mds_use_fqdn to true to use FQDN.
Signed-off-by: Sébastien Han <seb@redhat.com>
This adds a script, generate_group_vars_sample.sh, that generates
group_vars/*.sample from roles/ceph-*/defaults/main.yml to avoid
discrepancies between the sets of files. It also converts the line
endings in the various main.yml from DOS to Unix, since generating the
samples was spreading the line ending plague around to more files.
0644 should never be a directory mode. 1777 makes it so that any user
can create a ceph client, not just root. (This is helpful if, for
instance, nova-compute is running as non-root.)
If using another method to generate a consistent fsid, then we can
skip creation of an (unused) cluster UUID file. If cephx is disabled
as well, we can skip creation of the fetch directory entirely.
The firewall checks can fail for any number of reasons -- e.g., the
ceph cluster hostnames are unresolvable from the ansible host, or the
ports are filtered by some intermediate hop, etc. Make two changes to
make those checks better:
* Set pipefail when running the checks, so if nmap itself fails the
command will be marked as 'failed'. Specifically, this fixes the
case where the hostnames cannot be resolved.
* Add a new variable, check_firewall, which can be used to disable
checks entirely. Specifically, this fixes the case where some
intermediate firewall filters the ports, so nmap returns "filtered".
Installs on RHEL with ceph_origin set to distro previously would fail
because no packages would get installed, but all of the checks passed
fine. This adds support for ceph_origin: distro, simply installing the
packages using yum/dnf and assuming that the sysadmin has provided a
repository containing them.
This also supports the use case where Satellite or a similar local
mirror is in use, and the admin does not or cannot use the additional
repositories the role would otherwise add.
In our use case we might only be configuring mons and not osds in the
same call, so we don't want to check variables needed for osds when they
are not needed to configure a mon.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Currently deploying a MON fails with "bad symbolic permission for mode"
errors due to the file/directory modes not being interpreted as octal
values. This commit updates roles/ceph-common/tasks/main.yml to set
the file/directory modes to strings so they can be interpreted
correctly.
Closes issue #525
At the moment, all the tasks using the file module are duplicated to have differents ownerships depending on the fact `is_ceph_infernalis`.
The goal of this commit is to have a new logic for this:
- First set facts depending on the `is_ceph_infernalis` fact
- Create the files or directories using the setted facts as ownerships.
We have a requirement to install the packages first without
configuration. These tags should allow us to target the tasks need to do
that.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
as reported in #510 some systems don't have uuidgen installed so we
better use a more global way to generate it. It sounds like python
should be available in case uuidgen is not.
Otherwise we will have to find another way :)
closes#510
Signed-off-by: Sébastien Han <seb@redhat.com>
Currently, all the ceph package installation resources use
"state=latest", which means subsequent runs of the ceph playbooks
could result in ceph being upgraded if there are package updates
available in the selected repo.
This commit adds a new variable to ceph-common called
'upgrade_ceph_packages' which defaults to False. This variable is used
in the package installation resources for ceph packages to determine if
the resource should use "state=present" or "state=latest". If the
variable gets set to True, "state=latest" will be used.
Additionally, we update rolling_update.yml to override
upgrade_ceph_packages to true to permit package upgrades in this
context specifically.
Closes issue #506
It seems that in ansible 2.0 even if a task is skipped by it's `when`
clause not evaluating to true the variables in the play are still
rendered. Because these were not defined in defaults/main.yml ansible
was failing in installs/install_on_redhat where those variables are
being used in a `with_items` stanza.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
This change allows for configurable Ceph Conf Directory permissions. This
is required for integrators of Ceph, like OpenStack Cinder, which needs to
read from /etc/ceph for operation.
Use command module instead of shell since we do not do anything fancy
here. Remove the duplicate register.
Signed-off-by: Sébastien Han <seb@redhat.com>
As raised in #466 it is important in order to avoid unnecessary
troubleshooting to check that ceph ports are allowed on the platform.
The check runs a nmap command from the host running Ansible
to all the ceph nodes with their respective ports.
Signed-off-by: Sébastien Han <seb@redhat.com>
Thanks to @cloudnull great patch at
https://github.com/ansible/ansible/pull/12555
we now have the ability to add more configuration options instead of
having to push a PR to add a new option to the template. So you can
dynamically add and remove flags.
To use it, edit `ceph_conf_overrides` in `group_vars/all` like so:
```
ceph_conf_overrides
global:
foo: 12345
bar: 6789
```
Signed-off-by: Sébastien Han <seb@redhat.com>
Because of some permission issue, likely due to the recent ceph user, if
80 is used for civetweb we get:
set_ports_option: cannot bind to 80: 13 (Permission denied)
Changing the port to 8080 until this gets solved.
Signed-off-by: Sébastien Han <seb@redhat.com>
It should be used to disable health warnings about number of PGs
being too low if some pools have very few objects bringing down
the average number of objects per pool. This happens when running RadosGW.
The default is 10 and since the warnings only occur with some use cases,
the default here is 10 as well. Set to 20 or more to silence the warnings.
Currently, the fetch directory is created in your working directory
(where ansible is run from). We prefer to not keep any state in this
directory and would prefer to have the fetch directory configurable so
we can store it outside of our code checkout.
This commit creates a new variable in each role called
`fetch_directory` (defaulting to the previous value of 'fetch/'), and
then updates each reference to 'fetch' to use the new variable instead.
Closes issue #383
When multiple monitor hosts attempt to create the fetch directory there
is the potential for the task to fail with:
"OSError: [Errno 17] File exists: 'fetch'"
This appear to be an issue with the file module trying to create the
same directory at the same time when the tasks has been delegated to a
single host.
This commit enables run_once on the affected task which should address
the issue.
This is a rare case but it happens. Since we're just calling
`monitor_interface` and not `hostvars[host]['monitor_interface'],
an error may occur when the current host's interface does not
exist on the other hosts. (eg. eth0 exists for node0, but it does
not exist on node1 and node2)
Fix for this is to use hostvars[host]['monitor_interface']
Fix back the rolling update playbook.
However every single time the playbook will run it will check for new
packages and install the latest ones. I don't think this is always the
desired behaviour. We need to find a way to conciliate both...
Signed-off-by: Sébastien Han <seb@redhat.com>
Fix the logic for the mandatory devices check so that it applies to
raw_multi_journal and journal_collocation scenarios separately.
This fails otherwise because whichever var is "first" in the or is most
likely undefined.
I'm currently getting a KeyError due to missing 'dependencies' on this
role when I attempt to install it with ansible-galaxy (ansible 1.9.2).
This commit simply defines an empty dependencies list so that
ansible-galaxy executes correctly.
Cool stuff :). We don't need to specify an initial monitor key anymore.
A key will automatically be generated.
The default key can always be overriden with the `monitor_secret`
variable.
Signed-off-by: leseb <seb@redhat.com>
We don't always have a dedicated cluster network so we can by default
re-use the public network value.
This is just laziness :).
Signed-off-by: leseb <seb@redhat.com>
While re-running the playbook we do not want to check for new packages.
We shouldn't perform upgrades, we leave this to the operators.
Signed-off-by: leseb <seb@redhat.com>
Feel so bad about this one...
Now it's fixed, the rgw section will be activated once the rgws hosts
are part of the inventory.
Signed-off-by: leseb <seb@redhat.com>
Even if the subcription command is indempotent it takes around 15/16sec
to get it done. Where with the simple yum check we lower down this to
3sec.
Signed-off-by: leseb <seb@redhat.com>
Since the command is indempotent we don't need to check if the repo is
enabled as it will likely take twice the time.
Signed-off-by: leseb <seb@redhat.com>
We want to force the user to only enable the options they need. Thus
they shouldn't have to enable one option and then disable another.
Signed-off-by: leseb <seb@redhat.com>
Now we don't need to activate the services through a variable. If the
role is activated in the inventory, actions will occur automatically.
Fixing the repo creation for red hat storage too.
Signed-off-by: leseb <seb@redhat.com>
The new product version has jsut came out. ICE doesn't exist anymore and
Red Hat Storage is the name of the new product.
Signed-off-by: leseb <seb@redhat.com>
Following the best practice, we don't create a key from the monitor but
we really on the initial keys created by the mons to bootstrap each
daemon.
Signed-off-by: Sébastien Han <seb@redhat.com>
This branch has been sitting on my local repo for a while. I guess I had
time to spend on a plane :).
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
* fix the Vagrantfile ruby check
* fix the variable positions
Bring more mandatory variables and try to separate Vagrant vars from the
playbook vars.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Now the Ceph REST API can be deployed.
Default implementation deploys it on the same nodes as the monitors
which should be fine.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Fix the usage of Upstart for Ubuntu machines instead of the init.d
script.
Note that because of the way upstart init script looks at the radosgw id
the command 'start radosgw id=' is broken, you should use 'start
radosgw-all' instead.
Keep backard compatibility with the radosgw init script as well by using
client prefixed by 'client.radosgw'.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
If we use the hostname, the radosgw will lookup for a wrong secret.
Using the same name for all the gateways.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Use hostname in socket and log.
Improve jinja template so when a var doesn't exist we don't indent the
next line.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
We isolated the key operations into a file and modified the fetch
function to collect all the new keys.
In the mean time fixed the pool creation since the command is not
indempotent.
Renamed the rgw key to work with the key collection.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Without this plugin if a Ceph version is present in a repo (let's say
epel) it will install the epel version and not the ICE version.
We install yum-plugin-priorities.noarch to honor the 'priority=1' flag.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
In storage world it's often recommended to disable transparent hugepages
as they will tend to lower performance.
Note that this change won't survive reboot. There are several ways to
disable this permanently such as:
* rc.local
* grub boot line
It's a bit tricky to do this in Ansible since it really depends on the
OS you're running on.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Depending on the OS you are runnning on you should be able to configure
these values.
Re-ordering file for clarity as well.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Big cluster will easily reach the default limit so we need to increase
it and make it configurable.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
MDS and RGW are not deployed often (RGW more), so we disable them from
the default deployment to only get MONs and OSDs.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
With the appropriate subscription details you will be able to use the
Inktank Ceph Enterprise version of Ceph running on RHEL7.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
It has becomes really anoying to manually generate an fsid prior to the
inital bootstrap. This commit introduces a method that auto-generates an
fsid. If for whatever reasons you want to force your own fsid you can
simply edit these 3 files and override the fsid variable:
- roles/ceph-common/vars/main.yml
- roles/ceph-mon/vars/main.yml
- roles/ceph-osd/vars/main.yml
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
While running big boxes with 72 disks it's easy to get out of PID for
all the threads needed by Ceph. Increasing the default value removes
this limitation.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
In ceph-common you load {{ ansible_managed }} at the top of the main
config file - this will trigger handlers on that file whenever an
Ansible run is made.
I'd suggest replacing it with a vanilla text comment 'managed by
Ansible' to warn
admins but avoid unnecessary cluster bounces.
fixes: #125
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
The ceph.conf.j2 template currently always uses the current host facts
to get the IP address of each host in the mon loop. This is not the
expected behavior. This patch uses the correct facts to get the IP.
Recovery and/or re-balancing decrease performance, adding more options
might help tweaking this behavior.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Since 192.168.0.0/24 is very commong and might overlap with some
existing networks on your laptop, using another subnet like '42' is less
bound to happen.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Proviously we used osd_crush_update_on_start: true, this was interpreted
by Ansible as a boolean and appeared as 'True' inside the Ceph configuration
file. However the Ceph's init script is looking for 'true'.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
This commits introduces the support of the developpement branches of
Ceph. You can now install Ceph from master.
The behavior is done through 2 new options:
* ceph_stable: true will use the stable branch
* ceph_dev: true will use the dev branch
For the dev packages don't forget to set the branch that you want to
use.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Prior to this patch, the first match was winning and the playbook wasn't
doing any difference both "restart ceph", adding a distro filtrer fixes
this.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
It has been reported a couple of months ago by Dan van der Ster from
CERN that updatedb was consumming 100% of CPU while parsing system's
directories. Indeed the process was parsing the OSD PG directories that
might contains billions of objects.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
After a change is made on the configuration file we must restart the
Ceph services. I also added a check that verifies if a socker exists
because during the first play there are no services running. We check if
a socket exists, if not we don't try to restart the services, if it
exists we can restart them.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
I added a 'ceph-' prefix to all the roles related to Ceph. Since we are
about to push the roles into the Ansible Galaxy that will be easier when
we want to use these roles into a larger environement with other roles.
Fixes: #94
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Andrew Schoen
Alfredo Deza
Sébastien Han
Samuel Matzek
Stephan Hohn
Sam Yaple
Matt Thompson
Chris St. Pierre
James Saint-Rossy
Ben England
ksingh7
runsisi
pprokop
Huamin Chen
François Charlier
Zack Cerza
Niklas Haas
Guillaume Abrioux
mcsage
Byron McCollum
Logan V
crcceph
Abhishek Varshney
Eric Cook
Bjoern Teipel
Eike Frost
Bill Sanders
Emilien Mantel
Sébastien Han
Michael Sambol
Marc Heckmann
Timo Beckers
dexter
git-harry
Andy McCrae
Michael Sambol
Andrew F. Ly
Bjørnar Ness
Sébastien Han
Yin Jifeng
marmot21
Boris Manojlovic
Matthew Rees
Gauvain Pocentek
jjoos
Ramón Cahenzli
Alexis Lahouze
eltom