kubespray/docs/external_storage_provisioners/cephfs_provisioner.md

# CephFS Volume Provisioner for Kubernetes 1.5+

[![Docker Repository on Quay](https://quay.io/repository/external_storage/cephfs-provisioner/status "Docker Repository on Quay")](https://quay.io/repository/external_storage/cephfs-provisioner)

Using Ceph volume client

## Development

Compile the provisioner

``` console
make
```

Make the container image and push to the registry

``` console
make push
```

## Test instruction

- Start Kubernetes local cluster

See [Kubernetes](https://kubernetes.io/)

- Create a Ceph admin secret

``` bash
ceph auth get client.admin 2>&1 |grep "key = " |awk '{print  $3'} |xargs echo -n > /tmp/secret
kubectl create ns cephfs
kubectl create secret generic ceph-secret-admin --from-file=/tmp/secret --namespace=cephfs
```

- Start CephFS provisioner

The following example uses `cephfs-provisioner-1` as the identity for the instance and assumes kubeconfig is at `/root/.kube`. The identity should remain the same if the provisioner restarts. If there are multiple provisioners, each should have a different identity.

``` bash
docker run -ti -v /root/.kube:/kube -v /var/run/kubernetes:/var/run/kubernetes --privileged --net=host cephfs-provisioner /usr/local/bin/cephfs-provisioner -master=http://127.0.0.1:8080 -kubeconfig=/kube/config -id=cephfs-provisioner-1
```

Alternatively, deploy it in kubernetes, see [deployment](deploy/README.md).

- Create a CephFS Storage Class

Replace Ceph monitor's IP in [example class](example/class.yaml) with your own and create storage class:

``` bash
kubectl create -f example/class.yaml
```

- Create a claim

``` bash
kubectl create -f example/claim.yaml
```

- Create a Pod using the claim

``` bash
kubectl create -f example/test-pod.yaml
```

## Known limitations

- Kernel CephFS doesn't work with SELinux, setting SELinux label in Pod's securityContext will not work.
- Kernel CephFS doesn't support quota or capacity, capacity requested by PVC is not enforced or validated.
- Currently each Ceph user created by the provisioner has `allow r` MDS cap to permit CephFS mount.

## Acknowledgement

Inspired by CephFS Manila provisioner and conversation with John Spray
Docs improvements (#7660) * Docs: update sidebar * Docs: move registry documentation into docs/ * Docs: move rbd_provisioner documentation into docs/ * Docs: move cephfs_provisioner into docs/ * Docs: move local_volume_provisioner documentation into docs/ * Docs: move ambassador.md to docs/ingress_controller/ * Docs: move metallb.md to docs/ingress_controller/ * Docs: move ingress_nginx documentation into docs/ * Docs: move alb_ingress_controller documentation into docs/ * Docs: merge ambassador documentation into docs/ingress_controller/ * Docs: move cert_manager documentation into docs/ * Docs: move bootstrap-os documentation into docs/ * Docs: update file locations in sidebar 2021-06-01 22:30:27 +08:00			`# CephFS Volume Provisioner for Kubernetes 1.5+`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
			`[![Docker Repository on Quay](https://quay.io/repository/external_storage/cephfs-provisioner/status "Docker Repository on Quay")](https://quay.io/repository/external_storage/cephfs-provisioner)`

			`Using Ceph volume client`

Docs improvements (#7660) * Docs: update sidebar * Docs: move registry documentation into docs/ * Docs: move rbd_provisioner documentation into docs/ * Docs: move cephfs_provisioner into docs/ * Docs: move local_volume_provisioner documentation into docs/ * Docs: move ambassador.md to docs/ingress_controller/ * Docs: move metallb.md to docs/ingress_controller/ * Docs: move ingress_nginx documentation into docs/ * Docs: move alb_ingress_controller documentation into docs/ * Docs: merge ambassador documentation into docs/ingress_controller/ * Docs: move cert_manager documentation into docs/ * Docs: move bootstrap-os documentation into docs/ * Docs: update file locations in sidebar 2021-06-01 22:30:27 +08:00			`## Development`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
			`Compile the provisioner`

			``` console
			`make`
			```

			`Make the container image and push to the registry`

			``` console
			`make push`
			```

Docs improvements (#7660) * Docs: update sidebar * Docs: move registry documentation into docs/ * Docs: move rbd_provisioner documentation into docs/ * Docs: move cephfs_provisioner into docs/ * Docs: move local_volume_provisioner documentation into docs/ * Docs: move ambassador.md to docs/ingress_controller/ * Docs: move metallb.md to docs/ingress_controller/ * Docs: move ingress_nginx documentation into docs/ * Docs: move alb_ingress_controller documentation into docs/ * Docs: merge ambassador documentation into docs/ingress_controller/ * Docs: move cert_manager documentation into docs/ * Docs: move bootstrap-os documentation into docs/ * Docs: update file locations in sidebar 2021-06-01 22:30:27 +08:00			`## Test instruction`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
Fix markdownlint failures under ./roles/ (#7089) This fixes markdownlint failures under roles/ 2020-12-30 21:07:49 +08:00			`- Start Kubernetes local cluster`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
Fix markdownlint failures under ./roles/ (#7089) This fixes markdownlint failures under roles/ 2020-12-30 21:07:49 +08:00			`See [Kubernetes](https://kubernetes.io/)`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
Fix markdownlint failures under ./roles/ (#7089) This fixes markdownlint failures under roles/ 2020-12-30 21:07:49 +08:00			`- Create a Ceph admin secret`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
			``` bash
			`ceph auth get client.admin 2>&1 \|grep "key = " \|awk '{print $3'} \|xargs echo -n > /tmp/secret`
			`kubectl create ns cephfs`
			`kubectl create secret generic ceph-secret-admin --from-file=/tmp/secret --namespace=cephfs`
			```

Fix markdownlint failures under ./roles/ (#7089) This fixes markdownlint failures under roles/ 2020-12-30 21:07:49 +08:00			`- Start CephFS provisioner`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
			The following example uses `cephfs-provisioner-1` as the identity for the instance and assumes kubeconfig is at `/root/.kube`. The identity should remain the same if the provisioner restarts. If there are multiple provisioners, each should have a different identity.

			``` bash
remove the redundant space (#3420) 2018-09-30 12:03:27 +08:00			`docker run -ti -v /root/.kube:/kube -v /var/run/kubernetes:/var/run/kubernetes --privileged --net=host cephfs-provisioner /usr/local/bin/cephfs-provisioner -master=http://127.0.0.1:8080 -kubeconfig=/kube/config -id=cephfs-provisioner-1`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00			```

			`Alternatively, deploy it in kubernetes, see [deployment](deploy/README.md).`

Fix markdownlint failures under ./roles/ (#7089) This fixes markdownlint failures under roles/ 2020-12-30 21:07:49 +08:00			`- Create a CephFS Storage Class`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
Fix markdownlint failures under ./roles/ (#7089) This fixes markdownlint failures under roles/ 2020-12-30 21:07:49 +08:00			`Replace Ceph monitor's IP in [example class](example/class.yaml) with your own and create storage class:`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
			``` bash
			`kubectl create -f example/class.yaml`
			```

Fix markdownlint failures under ./roles/ (#7089) This fixes markdownlint failures under roles/ 2020-12-30 21:07:49 +08:00			`- Create a claim`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
			``` bash
			`kubectl create -f example/claim.yaml`
			```

Fix markdownlint failures under ./roles/ (#7089) This fixes markdownlint failures under roles/ 2020-12-30 21:07:49 +08:00			`- Create a Pod using the claim`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
			``` bash
			`kubectl create -f example/test-pod.yaml`
			```

Docs improvements (#7660) * Docs: update sidebar * Docs: move registry documentation into docs/ * Docs: move rbd_provisioner documentation into docs/ * Docs: move cephfs_provisioner into docs/ * Docs: move local_volume_provisioner documentation into docs/ * Docs: move ambassador.md to docs/ingress_controller/ * Docs: move metallb.md to docs/ingress_controller/ * Docs: move ingress_nginx documentation into docs/ * Docs: move alb_ingress_controller documentation into docs/ * Docs: merge ambassador documentation into docs/ingress_controller/ * Docs: move cert_manager documentation into docs/ * Docs: move bootstrap-os documentation into docs/ * Docs: update file locations in sidebar 2021-06-01 22:30:27 +08:00			`## Known limitations`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
Fix markdownlint failures under ./roles/ (#7089) This fixes markdownlint failures under roles/ 2020-12-30 21:07:49 +08:00			`- Kernel CephFS doesn't work with SELinux, setting SELinux label in Pod's securityContext will not work.`
			`- Kernel CephFS doesn't support quota or capacity, capacity requested by PVC is not enforced or validated.`
			- Currently each Ceph user created by the provisioner has `allow r` MDS cap to permit CephFS mount.
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
Docs improvements (#7660) * Docs: update sidebar * Docs: move registry documentation into docs/ * Docs: move rbd_provisioner documentation into docs/ * Docs: move cephfs_provisioner into docs/ * Docs: move local_volume_provisioner documentation into docs/ * Docs: move ambassador.md to docs/ingress_controller/ * Docs: move metallb.md to docs/ingress_controller/ * Docs: move ingress_nginx documentation into docs/ * Docs: move alb_ingress_controller documentation into docs/ * Docs: merge ambassador documentation into docs/ingress_controller/ * Docs: move cert_manager documentation into docs/ * Docs: move bootstrap-os documentation into docs/ * Docs: update file locations in sidebar 2021-06-01 22:30:27 +08:00			`## Acknowledgement`
CephFS Provisioner Addon Fixup 2018-02-13 09:55:59 +08:00
			`Inspired by CephFS Manila provisioner and conversation with John Spray`