The logic was broken here for repeated runs. We only want to run
'ceph-disk prepare' when the disk does not contain a ceph partition, is
not a partition, and raw_multi_journal is set. Previously it would
attempt to run 'ceph-disk prepare' when there was a ceph partition
because the second half of the 'or' was still true since it isn't a
partition.
Following the best practice, we don't create a key from the monitor but
we really on the initial keys created by the mons to bootstrap each
daemon.
Signed-off-by: Sébastien Han <seb@redhat.com>
This branch has been sitting on my local repo for a while. I guess I had
time to spend on a plane :).
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
* fix the Vagrantfile ruby check
* fix the variable positions
Bring more mandatory variables and try to separate Vagrant vars from the
playbook vars.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Once again and hopefully final commit to rework the support of both
upstart and sysvinit. As from now, Ubuntu systems will use upstart and
the others will use sysvinit.
A later commit might include the support of systemd as the unit files
come out. This will be for Hammer so probably soon.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Depending on the distro, init scripts will look for different files to
be available on the ceph data dir.
Fixing the upstart support here.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
If the distribution wasn't Ubuntu, the check wasn't performed so the
evaluation in the task later wasn't possible.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Now the Ceph REST API can be deployed.
Default implementation deploys it on the same nodes as the monitors
which should be fine.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Fix the usage of Upstart for Ubuntu machines instead of the init.d
script.
Note that because of the way upstart init script looks at the radosgw id
the command 'start radosgw id=' is broken, you should use 'start
radosgw-all' instead.
Keep backard compatibility with the radosgw init script as well by using
client prefixed by 'client.radosgw'.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
The ceph fs new command was introduced in Ceph 0.84. Prior to this
release, no manual steps are required to create a filesystem, and pools
named data and metadata exist by default.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
If we use the hostname, the radosgw will lookup for a wrong secret.
Using the same name for all the gateways.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Use hostname in socket and log.
Improve jinja template so when a var doesn't exist we don't indent the
next line.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
We isolated the key operations into a file and modified the fetch
function to collect all the new keys.
In the mean time fixed the pool creation since the command is not
indempotent.
Renamed the rgw key to work with the key collection.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Without this plugin if a Ceph version is present in a repo (let's say
epel) it will install the epel version and not the ICE version.
We install yum-plugin-priorities.noarch to honor the 'priority=1' flag.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
In storage world it's often recommended to disable transparent hugepages
as they will tend to lower performance.
Note that this change won't survive reboot. There are several ways to
disable this permanently such as:
* rc.local
* grub boot line
It's a bit tricky to do this in Ansible since it really depends on the
OS you're running on.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Default behavior is to fail if a variable is not declared however this
can be disable in your ansible.cfg so we force this variable as
mandatory.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Still WIP, @mwheckmann free to test
As requested by #162
Current known issue, since ceph.conf gets modified during every single
run (at the end during the merge) so this will restart ceph daemons.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Depending on the OS you are runnning on you should be able to configure
these values.
Re-ordering file for clarity as well.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Big cluster will easily reach the default limit so we need to increase
it and make it configurable.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
We remove all the partitions, label and re-create something clean prior
to prepare the design. This will help solving many issues with existing
disks or while scratching/deploy test environments often.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
MDS and RGW are not deployed often (RGW more), so we disable them from
the default deployment to only get MONs and OSDs.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
With the appropriate subscription details you will be able to use the
Inktank Ceph Enterprise version of Ceph running on RHEL7.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
It has becomes really anoying to manually generate an fsid prior to the
inital bootstrap. This commit introduces a method that auto-generates an
fsid. If for whatever reasons you want to force your own fsid you can
simply edit these 3 files and override the fsid variable:
- roles/ceph-common/vars/main.yml
- roles/ceph-mon/vars/main.yml
- roles/ceph-osd/vars/main.yml
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
- We don’t need ceph-extra for trusty
- Enable multiverse repo for access to libapache2-mod-fastcgi
- Update cache before attempting to install packages to register
multiverse repo and only refresh cache once an hour to avoid delays in
the playbook
- Add wildcard to disabling default site as on Ubuntu it is 000_default
by…default
While running big boxes with 72 disks it's easy to get out of PID for
all the threads needed by Ceph. Increasing the default value removes
this limitation.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
In ceph-common you load {{ ansible_managed }} at the top of the main
config file - this will trigger handlers on that file whenever an
Ansible run is made.
I'd suggest replacing it with a vanilla text comment 'managed by
Ansible' to warn
admins but avoid unnecessary cluster bounces.
fixes: #125
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
The ceph.conf.j2 template currently always uses the current host facts
to get the IP address of each host in the mon loop. This is not the
expected behavior. This patch uses the correct facts to get the IP.
Recovery and/or re-balancing decrease performance, adding more options
might help tweaking this behavior.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Since 192.168.0.0/24 is very commong and might overlap with some
existing networks on your laptop, using another subnet like '42' is less
bound to happen.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Proviously we used osd_crush_update_on_start: true, this was interpreted
by Ansible as a boolean and appeared as 'True' inside the Ceph configuration
file. However the Ceph's init script is looking for 'true'.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
This commits introduces the support of the developpement branches of
Ceph. You can now install Ceph from master.
The behavior is done through 2 new options:
* ceph_stable: true will use the stable branch
* ceph_dev: true will use the dev branch
For the dev packages don't forget to set the branch that you want to
use.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Prior to this patch, the first match was winning and the playbook wasn't
doing any difference both "restart ceph", adding a distro filtrer fixes
this.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
It has been reported a couple of months ago by Dan van der Ster from
CERN that updatedb was consumming 100% of CPU while parsing system's
directories. Indeed the process was parsing the OSD PG directories that
might contains billions of objects.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
After a change is made on the configuration file we must restart the
Ceph services. I also added a check that verifies if a socker exists
because during the first play there are no services running. We check if
a socket exists, if not we don't try to restart the services, if it
exists we can restart them.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
I added a 'ceph-' prefix to all the roles related to Ceph. Since we are
about to push the roles into the Ansible Galaxy that will be easier when
we want to use these roles into a larger environement with other roles.
Fixes: #94
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
This commit implements a fourth scenario where we can directely use a
directory instead of a block device for the OSDs. The purpose of this
scenario is more testing-oriented. Please note that we do not check
the filesystem underneath the directory so it is really up to you to
configure this properly. Declaring more than one directory on the
same filesystem will confuse Ceph.
Fixes: #14
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
This commit introduces a new config option 'osd crush chooseleaf type'.
With the help of this option and by setting it to '0' we tell Ceph to
store all the replicas on a single host. Basically we tell CRUSH to
iterate over disk and not over host.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
since we're now using fsid for the directory name, it should be safe to
just copy the keys from all mon hosts. Once they are copied, the rest of
the hosts will just skip copying. :)
The mon_initial_members is not used since we declare the mon section in
the ceph.conf file. Later, we could reduce the ceph.conf file by only
using the mon_host flag instead of all the mon sections.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
We introduced a key generation mechanism that aimed to ease deployment.
In the end, it brought more complexity to the playbook and doesn't
scale.
Reverting the auto generation commit and instructing users to generate
their own keys.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Currently everything lives in main.yml, the file has become difficult to
read at some point and can be a real mess since we keep adding new
scenarios.
I think we should separate the scenarios into dedicated files and just
do includes in the main.yml file.
Closes: #16
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
As mentionned in the issue 24 it's not really safe to store a default
fsid nor a monitor key. Thus the commit brings the auto-generation of
the initial monitor key. However it is quite complex to do the same for
the fsid, so I leave this to the person in charge of the deployment to
generate one and edit group_vars/all accordingly. The default fsid has
been removed as well.
Close: #24
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Even if MDS are not configured in site.yml the playbook has a
dependancy on the ceph.conf template.
This disables the mds section from the ceph.conf file.
Closes: #21
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Now the playbook is able to install Ceph on RedHat systems.
This has been tested on CentOS 6.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
This commits brings the support of multiple journals where each journal
points to a specific OSD and vice-versa. The commit also clarifies the
usage of multi scenarios for both journal and osd_data.
In the meantime, it fixes the collocation scenario.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
This brings the support of heterogeneous hardware. Not all the servers
are identical, some have more or less disks than the others. Prior this
commit, the 'parted' command was hanging, now the command simply exits 1
if the device doesn't exist, same for the 'egrep' piped command after.
Then we skip these errors and continue to run. So now, you can specify
multiple devices in group_vars/osds that don't exist on all the
servers.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
Since the fetch module takes care of the permissions it is not necessary
to set them with another module. The second command is useless.
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
leseb
Bryan Stillwell
Bjørnar Ness
Sébastien Han
Yin Jifeng
marmot21
Boris Manojlovic
Matthew Rees
Igor Shestakov
Cornelius Keller
Gauvain Pocentek
jjoos
Ramón Cahenzli
Piotr Bak
Alexis Lahouze
eltom
Kyle Christensen
Dexter John Genterone
Hunter Nield
Alessandro Corbelli
Jimmy Tang
Alfredo Deza