mirror of https://github.com/ceph/ceph-ansible.git
154 lines
5.0 KiB
Markdown
154 lines
5.0 KiB
Markdown
RGW Multisite
|
|
=============
|
|
|
|
This document contains directions for configuring the RGW Multisite support in ceph-ansible when the desired multisite configuration involves only one realm, one zone group and one zone in a cluster.
|
|
|
|
For information on configuring RGW Multisite with multiple realms, zone groups, or zones in a cluster, refer to [README-MULTISITE-MULTIREALM.md](README-MULTISITE-MULTIREALM.md).
|
|
|
|
In Ceph Multisite, a realm, master zone group, and a master zone is created on a Primary Ceph Cluster.
|
|
|
|
The realm on the primary cluster is pulled onto a secondary cluster where a new zone is created and joins the realm.
|
|
|
|
Once the realm is pulled on the secondary cluster and the new zone is created, data will now sync between the primary and secondary clusters.
|
|
|
|
## Requirements
|
|
|
|
Multisite replication can be configured either over multiple Ceph clusters or in a single Ceph cluster using Ceph version **Jewel or newer**.
|
|
|
|
* At least 2 Ceph clusters
|
|
* at least 1 RGW per cluster
|
|
* 1 RGW per host
|
|
* Jewel or newer
|
|
|
|
## Configuring the Master Zone in the Primary Ceph Cluster
|
|
|
|
This will setup the realm, master zonegroup and master zone and make them the defaults on the Primary Cluster.
|
|
|
|
``
|
|
1. Generate System Access and System Secret Keys
|
|
|
|
```
|
|
echo system_access_key: $(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 20 | head -n 1) > multi-site-keys.txt
|
|
echo system_secret_key: $(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 40 | head -n 1) >> multi-site-keys.txt
|
|
```
|
|
2. Edit `group_vars/all.yml` for the Primary Cluster
|
|
|
|
```
|
|
## Rados Gateway options
|
|
#
|
|
radosgw_num_instances: 1
|
|
|
|
#############
|
|
# MULTISITE #
|
|
#############
|
|
|
|
# Enable Multisite support
|
|
rgw_multisite: true
|
|
rgw_zone: jupiter
|
|
rgw_zonemaster: true
|
|
rgw_zonegroupmaster: true
|
|
rgw_zonesecondary: false
|
|
rgw_multisite_proto: "http"
|
|
rgw_zonegroup: solarsystem
|
|
rgw_zone_user: zone.user
|
|
rgw_zone_user_display_name: "Zone User"
|
|
rgw_realm: milkyway
|
|
system_access_key: 6kWkikvapSnHyE22P7nO
|
|
system_secret_key: MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt
|
|
```
|
|
|
|
**Note**: `radosgw_num_instances` must be set to 1. The playbooks do not support deploying RGW Multisite on hosts with more than 1 RGW.
|
|
|
|
**Note:** `rgw_zone` cannot be set to "default"
|
|
|
|
**Note:** `rgw_zonemaster` should have the value of `true` and `rgw_zonesecondary` should be `false`
|
|
|
|
**Note:** replace the `system_access_key` and `system_secret_key` values with the ones you generated
|
|
|
|
3. **(Optional)** Edit the rgws.yml in group_vars for rgw related pool creation
|
|
|
|
```
|
|
rgw_create_pools:
|
|
"{{ rgw_zone }}.rgw.buckets.data":
|
|
pg_num: 64
|
|
size: ""
|
|
type: ec
|
|
ec_profile: myecprofile
|
|
ec_k: 5
|
|
ec_m: 3
|
|
"{{ rgw_zone }}.rgw.buckets.index":
|
|
pg_num: 8
|
|
size: ""
|
|
type: replicated
|
|
```
|
|
|
|
**Note:** A pgcalc tool should be used to determine the optimal sizes for the rgw.buckets.data, rgw.buckets.index pools as well as any other pools declared in this dictionary.
|
|
|
|
4. Run the ceph-ansible playbook on your 1st cluster
|
|
|
|
## Pulling the Realm and Configuring a New Zone on a Secondary Ceph Cluster
|
|
|
|
This configuration will pull the realm from the primary cluster onto the secondary cluster and create a new zone on the cluster as well.
|
|
|
|
5. Edit `group_vars/all.yml` for the Secondary Cluster
|
|
|
|
```
|
|
## Rados Gateway options
|
|
#
|
|
radosgw_num_instances: 1
|
|
|
|
#############
|
|
# MULTISITE #
|
|
#############
|
|
|
|
rgw_multisite: true
|
|
rgw_zone: mars
|
|
rgw_zonemaster: false
|
|
rgw_zonesecondary: true
|
|
rgw_zonegroupmaster: false
|
|
rgw_multisite_proto: "http"
|
|
rgw_zonegroup: solarsystem
|
|
rgw_zone_user: zone.user
|
|
rgw_zone_user_display_name: "Zone User"
|
|
rgw_realm: milkyway
|
|
system_access_key: 6kWkikvapSnHyE22P7nO
|
|
system_secret_key: MGecsMrWtKZgngOHZdrd6d3JxGO5CPWgT2lcnpSt
|
|
rgw_pull_proto: http
|
|
rgw_pull_port: 8080
|
|
rgw_pullhost: cluster0-rgw0
|
|
```
|
|
|
|
**Note:** `rgw_zone` cannot be set to "default"
|
|
|
|
**Note:** `rgw_zonemaster` should have the value of `false` and `rgw_zonesecondary` should be `true`
|
|
|
|
**Note:** The endpoint made from `rgw_pull_proto` + `rgw_pull_host` + `rgw_pull_port` for each realm should be resolvable by the Primary Ceph clusters mons and rgws
|
|
|
|
**Note:** `rgw_zone_user`, `system_access_key`, and `system_secret_key` should match what you used in the Primary Cluster
|
|
|
|
6. **(Optional)** Edit the rgws.yml in group_vars for rgw related pool creation
|
|
|
|
```
|
|
rgw_create_pools:
|
|
"{{ rgw_zone }}.rgw.buckets.data":
|
|
pg_num: 64
|
|
size: ""
|
|
type: ec
|
|
ec_profile: myecprofile
|
|
ec_k: 5
|
|
ec_m: 3
|
|
"{{ rgw_zone }}.rgw.buckets.index":
|
|
pg_num: 8
|
|
size: ""
|
|
type: replicated
|
|
```
|
|
**Note:** The pg_num values should match the values for the rgw pools created on the primary cluster. Mismatching pg_num values on different sites can result in very poor performance.
|
|
|
|
**Note:** An online pgcalc tool (ex: https://ceph.io/pgcalc) should be used to determine the optimal sizes for the rgw.buckets.data, rgw.buckets.index pools as well as any other pools declared in this dictionary.
|
|
|
|
7. Run the ceph-ansible playbook on your 2nd cluster
|
|
|
|
## Conclusion
|
|
|
|
You should now have a master zone on cluster0 and a secondary zone on cluster1 in an Active-Active mode.
|