2017-08-21 18:44:34 +08:00
<!DOCTYPE HTML>
< html lang = "zh-cn" >
< head >
< meta charset = "UTF-8" >
< meta content = "text/html; charset=utf-8" http-equiv = "Content-Type" >
< title > 4.4.1.1 使用GlusterFS做持久化存储 · Kubernetes Handbook< / title >
< meta http-equiv = "X-UA-Compatible" content = "IE=edge" / >
< meta name = "description" content = "" >
< meta name = "generator" content = "GitBook 3.2.2" >
< meta name = "author" content = "Jimmy Song" >
< link rel = "stylesheet" href = "../gitbook/style.css" >
< link rel = "stylesheet" href = "../gitbook/gitbook-plugin-splitter/splitter.css" >
< link rel = "stylesheet" href = "../gitbook/gitbook-plugin-page-toc-button/plugin.css" >
< link rel = "stylesheet" href = "../gitbook/gitbook-plugin-image-captions/image-captions.css" >
< link rel = "stylesheet" href = "../gitbook/gitbook-plugin-page-footer-ex/style/plugin.css" >
< link rel = "stylesheet" href = "../gitbook/gitbook-plugin-search-plus/search.css" >
< link rel = "stylesheet" href = "../gitbook/gitbook-plugin-highlight/website.css" >
< link rel = "stylesheet" href = "../gitbook/gitbook-plugin-fontsettings/website.css" >
< meta name = "HandheldFriendly" content = "true" / >
< meta name = "viewport" content = "width=device-width, initial-scale=1, user-scalable=no" >
< meta name = "apple-mobile-web-app-capable" content = "yes" >
< meta name = "apple-mobile-web-app-status-bar-style" content = "black" >
< link rel = "apple-touch-icon-precomposed" sizes = "152x152" href = "../gitbook/images/apple-touch-icon-precomposed-152.png" >
< link rel = "shortcut icon" href = "../gitbook/images/favicon.ico" type = "image/x-icon" >
< link rel = "next" href = "storage-for-containers-using-glusterfs-with-openshift.html" / >
< link rel = "prev" href = "glusterfs.html" / >
< / head >
< body >
< div class = "book" >
< div class = "book-summary" >
< div id = "book-search-input" role = "search" >
< input type = "text" placeholder = "輸入並搜尋" / >
< / div >
< nav role = "navigation" >
< ul class = "summary" >
< li class = "chapter " data-level = "1.1" data-path = "../" >
< a href = "../" >
1. 前言
< / a >
< / li >
< li class = "chapter " data-level = "1.2" data-path = "../concepts/" >
< a href = "../concepts/" >
2. 概念原理
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.2.1" data-path = "../concepts/concepts.html" >
< a href = "../concepts/concepts.html" >
2.1 设计理念
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2" data-path = "../concepts/objects.html" >
< a href = "../concepts/objects.html" >
2.2 主要概念
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.2.2.1" data-path = "../concepts/pod-overview.html" >
< a href = "../concepts/pod-overview.html" >
2.2.1 Pod
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.2.2.1.1" data-path = "../concepts/pod.html" >
< a href = "../concepts/pod.html" >
2.2.1.1 Pod解析
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.2.2.2" data-path = "../concepts/node.html" >
< a href = "../concepts/node.html" >
2.2.2 Node
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.3" data-path = "../concepts/namespace.html" >
< a href = "../concepts/namespace.html" >
2.2.3 Namespace
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.4" data-path = "../concepts/service.html" >
< a href = "../concepts/service.html" >
2.2.4 Service
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.5" data-path = "../concepts/volume.html" >
< a href = "../concepts/volume.html" >
2.2.5 Volume和Persistent Volume
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.6" data-path = "../concepts/deployment.html" >
< a href = "../concepts/deployment.html" >
2.2.6 Deployment
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.7" data-path = "../concepts/secret.html" >
< a href = "../concepts/secret.html" >
2.2.7 Secret
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.8" data-path = "../concepts/statefulset.html" >
< a href = "../concepts/statefulset.html" >
2.2.8 StatefulSet
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.9" data-path = "../concepts/daemonset.html" >
< a href = "../concepts/daemonset.html" >
2.2.9 DaemonSet
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.10" data-path = "../concepts/serviceaccount.html" >
< a href = "../concepts/serviceaccount.html" >
2.2.10 ServiceAccount
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.11" data-path = "../concepts/replicaset.html" >
< a href = "../concepts/replicaset.html" >
2.2.11 ReplicationController和ReplicaSet
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.12" data-path = "../concepts/job.html" >
< a href = "../concepts/job.html" >
2.2.12 Job
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.13" data-path = "../concepts/cronjob.html" >
< a href = "../concepts/cronjob.html" >
2.2.13 CronJob
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.14" data-path = "../concepts/ingress.html" >
< a href = "../concepts/ingress.html" >
2.2.14 Ingress
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.15" data-path = "../concepts/configmap.html" >
< a href = "../concepts/configmap.html" >
2.2.15 ConfigMap
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.16" data-path = "../concepts/horizontal-pod-autoscaling.html" >
< a href = "../concepts/horizontal-pod-autoscaling.html" >
2.2.16 Horizontal Pod Autoscaling
< / a >
< / li >
< li class = "chapter " data-level = "1.2.2.17" data-path = "../concepts/label.html" >
< a href = "../concepts/label.html" >
2.2.17 Label
< / a >
< / li >
< / ul >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.3" data-path = "../guide/" >
< a href = "../guide/" >
3. 用户指南
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.3.1" data-path = "../guide/resource-configuration.html" >
< a href = "../guide/resource-configuration.html" >
3.1 资源配置
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.3.1.1" data-path = "../guide/configure-liveness-readiness-probes.html" >
< a href = "../guide/configure-liveness-readiness-probes.html" >
3.1.1 配置Pod的liveness和readiness探针
< / a >
< / li >
< li class = "chapter " data-level = "1.3.1.2" data-path = "../guide/configure-pod-service-account.html" >
< a href = "../guide/configure-pod-service-account.html" >
3.1.2 配置Pod的Service Account
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.3.2" data-path = "../guide/command-usage.html" >
< a href = "../guide/command-usage.html" >
3.2 命令使用
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.3.2.1" data-path = "../guide/using-kubectl.html" >
< a href = "../guide/using-kubectl.html" >
3.2.1 使用kubectl
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.3.3" data-path = "../guide/cluster-management.html" >
< a href = "../guide/cluster-management.html" >
3.3 集群管理
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.3.3.1" data-path = "../guide/managing-tls-in-a-cluster.html" >
< a href = "../guide/managing-tls-in-a-cluster.html" >
3.3.1 管理集群中的TLS
< / a >
< / li >
< li class = "chapter " data-level = "1.3.3.2" data-path = "../guide/kubelet-authentication-authorization.html" >
< a href = "../guide/kubelet-authentication-authorization.html" >
3.3.2 kubelet的认证授权
< / a >
< / li >
< li class = "chapter " data-level = "1.3.3.3" data-path = "../guide/tls-bootstrapping.html" >
< a href = "../guide/tls-bootstrapping.html" >
3.3.3 TLS bootstrap
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.3.4" data-path = "../guide/access-kubernetes-cluster.html" >
< a href = "../guide/access-kubernetes-cluster.html" >
3.4 访问 Kubernetes 集群
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.3.4.1" data-path = "../guide/access-cluster.html" >
< a href = "../guide/access-cluster.html" >
3.4.1 访问集群
< / a >
< / li >
< li class = "chapter " data-level = "1.3.4.2" data-path = "../guide/authenticate-across-clusters-kubeconfig.html" >
< a href = "../guide/authenticate-across-clusters-kubeconfig.html" >
3.4.2 使用 kubeconfig 文件配置跨集群认证
< / a >
< / li >
< li class = "chapter " data-level = "1.3.4.3" data-path = "../guide/connecting-to-applications-port-forward.html" >
< a href = "../guide/connecting-to-applications-port-forward.html" >
3.4.3 通过端口转发访问集群中的应用程序
< / a >
< / li >
< li class = "chapter " data-level = "1.3.4.4" data-path = "../guide/service-access-application-cluster.html" >
< a href = "../guide/service-access-application-cluster.html" >
3.4.4 使用 service 访问群集中的应用程序
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.3.5" data-path = "../guide/application-development-deployment-flow.html" >
< a href = "../guide/application-development-deployment-flow.html" >
3.5 在kubernetes中开发部署应用
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.3.5.1" data-path = "../guide/deploy-applications-in-kubernetes.html" >
< a href = "../guide/deploy-applications-in-kubernetes.html" >
3.5.1 适用于kubernetes的应用开发部署流程
< / a >
< / li >
< li class = "chapter " data-level = "1.3.5.2" data-path = "../guide/migrating-hadoop-yarn-to-kubernetes.html" >
< a href = "../guide/migrating-hadoop-yarn-to-kubernetes.html" >
3.5.2 迁移传统应用到kubernetes中——以Hadoop YARN为例
< / a >
< / li >
< / ul >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.4" data-path = "./" >
< a href = "./" >
4. 最佳实践
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.4.1" data-path = "install-kbernetes1.6-on-centos.html" >
< a href = "install-kbernetes1.6-on-centos.html" >
4.1 在CentOS上部署kubernetes1.6集群
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.4.1.1" data-path = "create-tls-and-secret-key.html" >
< a href = "create-tls-and-secret-key.html" >
4.1.1 创建TLS证书和秘钥
< / a >
< / li >
< li class = "chapter " data-level = "1.4.1.2" data-path = "create-kubeconfig.html" >
< a href = "create-kubeconfig.html" >
4.1.2 创建kubeconfig文件
< / a >
< / li >
< li class = "chapter " data-level = "1.4.1.3" data-path = "etcd-cluster-installation.html" >
< a href = "etcd-cluster-installation.html" >
4.1.3 创建高可用etcd集群
< / a >
< / li >
< li class = "chapter " data-level = "1.4.1.4" data-path = "kubectl-installation.html" >
< a href = "kubectl-installation.html" >
4.1.4 安装kubectl命令行工具
< / a >
< / li >
< li class = "chapter " data-level = "1.4.1.5" data-path = "master-installation.html" >
< a href = "master-installation.html" >
4.1.5 部署master节点
< / a >
< / li >
< li class = "chapter " data-level = "1.4.1.6" data-path = "node-installation.html" >
< a href = "node-installation.html" >
4.1.6 部署node节点
< / a >
< / li >
< li class = "chapter " data-level = "1.4.1.7" data-path = "kubedns-addon-installation.html" >
< a href = "kubedns-addon-installation.html" >
4.1.7 安装kubedns插件
< / a >
< / li >
< li class = "chapter " data-level = "1.4.1.8" data-path = "dashboard-addon-installation.html" >
< a href = "dashboard-addon-installation.html" >
4.1.8 安装dashboard插件
< / a >
< / li >
< li class = "chapter " data-level = "1.4.1.9" data-path = "heapster-addon-installation.html" >
< a href = "heapster-addon-installation.html" >
4.1.9 安装heapster插件
< / a >
< / li >
< li class = "chapter " data-level = "1.4.1.10" data-path = "efk-addon-installation.html" >
< a href = "efk-addon-installation.html" >
4.1.10 安装EFK插件
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.4.2" data-path = "service-discovery-and-loadbalancing.html" >
< a href = "service-discovery-and-loadbalancing.html" >
4.2 服务发现与负载均衡
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.4.2.1" data-path = "traefik-ingress-installation.html" >
< a href = "traefik-ingress-installation.html" >
4.2.1 安装Traefik ingress
< / a >
< / li >
< li class = "chapter " data-level = "1.4.2.2" data-path = "distributed-load-test.html" >
< a href = "distributed-load-test.html" >
4.2.2 分布式负载测试
< / a >
< / li >
< li class = "chapter " data-level = "1.4.2.3" data-path = "network-and-cluster-perfermance-test.html" >
< a href = "network-and-cluster-perfermance-test.html" >
4.2.3 网络和集群性能测试
< / a >
< / li >
< li class = "chapter " data-level = "1.4.2.4" data-path = "edge-node-configuration.html" >
< a href = "edge-node-configuration.html" >
4.2.4 边缘节点配置
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.4.3" data-path = "operation.html" >
< a href = "operation.html" >
4.3 运维管理
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.4.3.1" data-path = "service-rolling-update.html" >
< a href = "service-rolling-update.html" >
4.3.1 服务滚动升级
< / a >
< / li >
< li class = "chapter " data-level = "1.4.3.2" data-path = "app-log-collection.html" >
< a href = "app-log-collection.html" >
4.3.2 应用日志收集
< / a >
< / li >
< li class = "chapter " data-level = "1.4.3.3" data-path = "configuration-best-practice.html" >
< a href = "configuration-best-practice.html" >
4.3.3 配置最佳实践
< / a >
< / li >
< li class = "chapter " data-level = "1.4.3.4" data-path = "monitor.html" >
< a href = "monitor.html" >
4.3.4 集群及应用监控
< / a >
< / li >
< li class = "chapter " data-level = "1.4.3.5" data-path = "jenkins-ci-cd.html" >
< a href = "jenkins-ci-cd.html" >
4.3.5 使用Jenkins进行持续构建与发布
< / a >
< / li >
< li class = "chapter " data-level = "1.4.3.6" data-path = "data-persistence-problem.html" >
< a href = "data-persistence-problem.html" >
4.3.6 数据持久化问题
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.4.4" data-path = "storage.html" >
< a href = "storage.html" >
4.4 存储管理
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.4.4.1" data-path = "glusterfs.html" >
< a href = "glusterfs.html" >
4.4.1 GlusterFS
< / a >
< ul class = "articles" >
< li class = "chapter active" data-level = "1.4.4.1.1" data-path = "using-glusterfs-for-persistent-storage.html" >
< a href = "using-glusterfs-for-persistent-storage.html" >
4.4.1.1 使用GlusterFS做持久化存储
< / a >
< / li >
< li class = "chapter " data-level = "1.4.4.1.2" data-path = "storage-for-containers-using-glusterfs-with-openshift.html" >
< a href = "storage-for-containers-using-glusterfs-with-openshift.html" >
4.4.1.2 在OpenShift中使用GlusterFS做持久化存储
< / a >
< / li >
< / ul >
< / li >
< / ul >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.5" data-path = "../usecases/" >
< a href = "../usecases/" >
5. 领域应用
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.5.1" data-path = "../usecases/microservices.html" >
< a href = "../usecases/microservices.html" >
5.1 微服务架构
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.5.1.1" data-path = "../usecases/istio.html" >
< a href = "../usecases/istio.html" >
5.1.1 Istio
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.5.1.1.1" data-path = "../usecases/istio-installation.html" >
< a href = "../usecases/istio-installation.html" >
5.1.1.1 安装istio
< / a >
< / li >
< li class = "chapter " data-level = "1.5.1.1.2" data-path = "../usecases/configuring-request-routing.html" >
< a href = "../usecases/configuring-request-routing.html" >
5.1.1.2 配置请求的路由规则
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.5.1.2" data-path = "../usecases/linkerd.html" >
< a href = "../usecases/linkerd.html" >
5.1.2 Linkerd
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.5.1.2.1" data-path = "../usecases/linkerd-user-guide.html" >
< a href = "../usecases/linkerd-user-guide.html" >
5.1.2.1 Linkerd 使用指南
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.5.1.3" data-path = "../usecases/service-discovery-in-microservices.html" >
< a href = "../usecases/service-discovery-in-microservices.html" >
5.1.3 微服务中的服务发现
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.5.2" data-path = "../usecases/big-data.html" >
< a href = "../usecases/big-data.html" >
5.2 大数据
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.5.2.1" data-path = "../usecases/spark-on-kubernetes.html" >
< a href = "../usecases/spark-on-kubernetes.html" >
5.2.1 Spark on Kubernetes
< / a >
< / li >
< / ul >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.6" data-path = "../develop/" >
< a href = "../develop/" >
6. 开发指南
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.6.1" data-path = "../develop/developing-environment.html" >
< a href = "../develop/developing-environment.html" >
6.1 开发环境搭建
< / a >
< / li >
< li class = "chapter " data-level = "1.6.2" data-path = "../develop/testing.html" >
< a href = "../develop/testing.html" >
6.2 单元测试和集成测试
< / a >
< / li >
< li class = "chapter " data-level = "1.6.3" data-path = "../develop/client-go-sample.html" >
< a href = "../develop/client-go-sample.html" >
6.3 client-go示例
< / a >
< / li >
< li class = "chapter " data-level = "1.6.4" data-path = "../develop/contribute.html" >
< a href = "../develop/contribute.html" >
6.4 社区贡献
< / a >
< / li >
< / ul >
< / li >
< li class = "chapter " data-level = "1.7" data-path = "../appendix/" >
< a href = "../appendix/" >
7. 附录
< / a >
< ul class = "articles" >
< li class = "chapter " data-level = "1.7.1" data-path = "../appendix/docker-best-practice.html" >
< a href = "../appendix/docker-best-practice.html" >
7.1 Docker最佳实践
< / a >
< / li >
< li class = "chapter " data-level = "1.7.2" data-path = "../appendix/issues.html" >
< a href = "../appendix/issues.html" >
7.2 问题记录
< / a >
< / li >
< li class = "chapter " data-level = "1.7.3" data-path = "../appendix/tricks.html" >
< a href = "../appendix/tricks.html" >
7.3 使用技巧
< / a >
< / li >
< / ul >
< / li >
< li class = "divider" > < / li >
< li >
< a href = "https://www.gitbook.com" target = "blank" class = "gitbook-link" >
本書使用 GitBook 釋出
< / a >
< / li >
< / ul >
< / nav >
< / div >
< div class = "book-body" >
< div class = "body-inner" >
< div class = "book-header" role = "navigation" >
<!-- Title -->
< h1 >
< i class = "fa fa-circle-o-notch fa-spin" > < / i >
< a href = ".." > 4.4.1.1 使用GlusterFS做持久化存储< / a >
< / h1 >
< / div >
< div class = "page-wrapper" tabindex = "-1" role = "main" >
< div class = "page-inner" >
< div class = "search-plus" id = "book-search-results" >
< div class = "search-noresults" >
< section class = "normal markdown-section" >
< h1 id = "使用glusterfs做持久化存储" > 使 用 glusterfs做 持 久 化 存 储 < / h1 >
< p > 我 们 复 用 kubernetes的 三 台 主 机 做 glusterfs存 储 。 < / p >
< p > 以 下 步 骤 参 考 自 ： < a href = "https://www.xf80.com/2017/04/21/kubernetes-glusterfs/" target = "_blank" > https://www.xf80.com/2017/04/21/kubernetes-glusterfs/< / a > < / p >
< h2 id = "安装glusterfs" > 安 装 glusterfs< / h2 >
< p > 我 们 直 接 在 物 理 机 上 使 用 yum安 装 ， 如 果 你 选 择 在 kubernetes上 安 装 ， 请 参 考 ： < a href = "https://github.com/gluster/gluster-kubernetes/blob/master/docs/setup-guide.md" target = "_blank" > https://github.com/gluster/gluster-kubernetes/blob/master/docs/setup-guide.md< / a > < / p >
< pre > < code class = "lang-bash" > < span class = "hljs-comment" > # 先 安 装 gluster 源 < / span >
$ yum install centos-release-gluster -y
< span class = "hljs-comment" > # 安 装 glusterfs 组 件 < / span >
$ yum install -y glusterfs glusterfs-server glusterfs-fuse glusterfs-rdma glusterfs-geo-replication glusterfs-devel
< span class = "hljs-comment" > ## 创 建 glusterfs 目 录 < / span >
$ mkdir /opt/glusterd
< span class = "hljs-comment" > ## 修 改 glusterd 目 录 < / span >
$ sed -i < span class = "hljs-string" > ' s/var\/lib/opt/g' < / span > /etc/glusterfs/glusterd.vol
< span class = "hljs-comment" > # 启 动 glusterfs< / span >
$ systemctl start glusterd.service
< span class = "hljs-comment" > # 设 置 开 机 启 动 < / span >
$ systemctl < span class = "hljs-built_in" > enable< / span > glusterd.service
< span class = "hljs-comment" > #查 看 状 态 < / span >
$ systemctl status glusterd.service
< / code > < / pre >
< h2 id = "配置-glusterfs" > 配 置 glusterfs< / h2 >
< pre > < code class = "lang-Bash" > < span class = "hljs-comment" > # 配 置 hosts< / span >
$ vi /etc/hosts
172.20.0.113 sz-pg-oam-docker-test-001.tendcloud.com
172.20.0.114 sz-pg-oam-docker-test-002.tendcloud.com
172.20.0.115 sz-pg-oam-docker-test-003.tendcloud.com
< / code > < / pre >
< pre > < code class = "lang-bash" > < span class = "hljs-comment" > # 开 放 端 口 < / span >
$ iptables -I INPUT -p tcp --dport 24007 -j ACCEPT
< span class = "hljs-comment" > # 创 建 存 储 目 录 < / span >
$ mkdir /opt/gfs_data
< / code > < / pre >
< pre > < code class = "lang-bash" > < span class = "hljs-comment" > # 添 加 节 点 到 集 群 < / span >
< span class = "hljs-comment" > # 执 行 操 作 的 本 机 不 需 要 probe 本 机 < / span >
[root@sz-pg-oam-docker-test-001 ~]< span class = "hljs-comment" > #< / span >
gluster peer probe sz-pg-oam-docker-test-002.tendcloud.com
gluster peer probe sz-pg-oam-docker-test-003.tendcloud.com
< span class = "hljs-comment" > # 查 看 集 群 状 态 < / span >
$ gluster peer status
Number of Peers: 2
Hostname: sz-pg-oam-docker-test-002.tendcloud.com
Uuid: f25546cc-2011-457d-ba24-342554b51317
State: Peer < span class = "hljs-keyword" > in< / span > Cluster (Connected)
Hostname: sz-pg-oam-docker-test-003.tendcloud.com
Uuid: 42b6cad1-aa01-46d0-bbba< span class = "hljs-_" > -f< / span > 7ec6821d66d
State: Peer < span class = "hljs-keyword" > in< / span > Cluster (Connected)
< / code > < / pre >
< h2 id = "配置-volume" > 配 置 volume< / h2 >
< p > GlusterFS中 的 volume的 模 式 有 很 多 中 ， 包 括 以 下 几 种 ： < / p >
< ul >
< li > < strong > 分 布 卷 （ 默 认 模 式 ） < / strong > ： 即 DHT, 也 叫 分 布 卷 : 将 文 件 已 hash算 法 随 机 分 布 到 一 台 服 务 器 节 点 中 存 储 。 < / li >
< li > < strong > 复 制 模 式 < / strong > ： 即 AFR, 创 建 volume 时 带 replica x 数 量 : 将 文 件 复 制 到 replica x 个 节 点 中 。 < / li >
< li > < strong > 条 带 模 式 < / strong > ： 即 Striped, 创 建 volume 时 带 stripe x 数 量 ： 将 文 件 切 割 成 数 据 块 ， 分 别 存 储 到 stripe x 个 节 点 中 ( 类 似 raid 0 )。 < / li >
< li > < strong > 分 布 式 条 带 模 式 < / strong > ： 最 少 需 要 4台 服 务 器 才 能 创 建 。 创 建 volume 时 stripe 2 server = 4 个 节 点 ： 是 DHT 与 Striped 的 组 合 型 。 < / li >
< li > < strong > 分 布 式 复 制 模 式 < / strong > ： 最 少 需 要 4台 服 务 器 才 能 创 建 。 创 建 volume 时 replica 2 server = 4 个 节 点 ： 是 DHT 与 AFR 的 组 合 型 。 < / li >
< li > < strong > 条 带 复 制 卷 模 式 < / strong > ： 最 少 需 要 4台 服 务 器 才 能 创 建 。 创 建 volume 时 stripe 2 replica 2 server = 4 个 节 点 ： 是 Striped 与 AFR 的 组 合 型 。 < / li >
< li > < strong > 三 种 模 式 混 合 < / strong > ： 至 少 需 要 8台 服 务 器 才 能 创 建 。 stripe 2 replica 2 , 每 4个 节 点 组 成 一 个 组 。 < / li >
< / ul >
< p > 这 几 种 模 式 的 示 例 图 参 考 ： < a href = "http://www.cnblogs.com/jicki/p/5801712.html" target = "_blank" > CentOS7安 装 GlusterFS< / a > 。 < / p >
< p > 因 为 我 们 只 有 三 台 主 机 ， 在 此 我 们 使 用 默 认 的 < strong > 分 布 卷 模 式 < / strong > 。 < strong > 请 勿 在 生 产 环 境 上 使 用 该 模 式 ， 容 易 导 致 数 据 丢 失 。 < / strong > < / p >
< pre > < code class = "lang-bash" > < span class = "hljs-comment" > # 创 建 分 布 卷 < / span >
$ gluster volume create k8s-volume transport tcp sz-pg-oam-docker-test-001.tendcloud.com:/opt/gfs_data sz-pg-oam-docker-test-002.tendcloud.com:/opt/gfs_data sz-pg-oam-docker-test-003.tendcloud.com:/opt/gfs_data force
< span class = "hljs-comment" > # 查 看 volume状 态 < / span >
$ gluster volume info
Volume Name: k8s-volume
Type: Distribute
Volume ID: 9a3b0710-4565-4eb7-abae-1d5c8ed625ac
Status: Created
Snapshot Count: 0
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: sz-pg-oam-docker-test-001.tendcloud.com:/opt/gfs_data
Brick2: sz-pg-oam-docker-test-002.tendcloud.com:/opt/gfs_data
Brick3: sz-pg-oam-docker-test-003.tendcloud.com:/opt/gfs_data
Options Reconfigured:
transport.address-family: inet
nfs.disable: on
< span class = "hljs-comment" > # 启 动 分 布 卷 < / span >
$ gluster volume start k8s-volume
< / code > < / pre >
< h2 id = "glusterfs调优" > Glusterfs调 优 < / h2 >
< pre > < code class = "lang-bash" > < span class = "hljs-comment" > # 开 启 指 定 volume 的 配 额 < / span >
$ gluster volume quota k8s-volume < span class = "hljs-built_in" > enable< / span >
< span class = "hljs-comment" > # 限 制 指 定 volume 的 配 额 < / span >
$ gluster volume quota k8s-volume < span class = "hljs-built_in" > limit< / span > -usage / 1TB
< span class = "hljs-comment" > # 设 置 cache 大 小 , 默 认 32MB< / span >
$ gluster volume < span class = "hljs-built_in" > set< / span > k8s-volume performance.cache-size 4GB
< span class = "hljs-comment" > # 设 置 io 线 程 , 太 大 会 导 致 进 程 崩 溃 < / span >
$ gluster volume < span class = "hljs-built_in" > set< / span > k8s-volume performance.io-thread-count 16
< span class = "hljs-comment" > # 设 置 网 络 检 测 时 间 , 默 认 42s< / span >
$ gluster volume < span class = "hljs-built_in" > set< / span > k8s-volume network.ping-timeout 10
< span class = "hljs-comment" > # 设 置 写 缓 冲 区 的 大 小 , 默 认 1M< / span >
$ gluster volume < span class = "hljs-built_in" > set< / span > k8s-volume performance.write-behind-window-size 1024MB
< / code > < / pre >
< h2 id = "kubernetes中配置glusterfs" > Kubernetes中 配 置 glusterfs< / h2 >
< p > 官 方 的 文 档 见 ： < a href = "https://github.com/kubernetes/kubernetes/tree/master/examples/volumes/glusterfs" target = "_blank" > https://github.com/kubernetes/kubernetes/tree/master/examples/volumes/glusterfs< / a > < / p >
< p > 以 下 用 到 的 所 有 yaml和 json配 置 文 件 可 以 在 < a href = "manifests/glusterfs" > glusterfs< / a > 中 找 到 。 注 意 替 换 其 中 私 有 镜 像 地 址 为 你 自 己 的 镜 像 地 址 。 < / p >
< h2 id = "kubernetes安装客户端" > kubernetes安 装 客 户 端 < / h2 >
< pre > < code class = "lang-bash" > < span class = "hljs-comment" > # 在 所 有 k8s node 中 安 装 glusterfs 客 户 端 < / span >
$ yum install -y glusterfs glusterfs-fuse
< span class = "hljs-comment" > # 配 置 hosts< / span >
$ vi /etc/hosts
172.20.0.113 sz-pg-oam-docker-test-001.tendcloud.com
172.20.0.114 sz-pg-oam-docker-test-002.tendcloud.com
172.20.0.115 sz-pg-oam-docker-test-003.tendcloud.com
< / code > < / pre >
< p > 因 为 我 们 glusterfs跟 kubernetes集 群 复 用 主 机 ， 因 为 此 这 一 步 可 以 省 去 。 < / p >
< h2 id = "配置-endpoints" > 配 置 endpoints< / h2 >
< pre > < code class = "lang-bash" > $ curl -O https://raw.githubusercontent.com/kubernetes/kubernetes/master/examples/volumes/glusterfs/glusterfs-endpoints.json
< span class = "hljs-comment" > # 修 改 endpoints.json ， 配 置 glusters 集 群 节 点 ip< / span >
< span class = "hljs-comment" > # 每 一 个 addresses 为 一 个 ip 组 < / span >
{
< span class = "hljs-string" > " addresses" < / span > : [
{
< span class = "hljs-string" > " ip" < / span > : < span class = "hljs-string" > " 172.22.0.113" < / span >
}
],
< span class = "hljs-string" > " ports" < / span > : [
{
< span class = "hljs-string" > " port" < / span > : 1990
}
]
},
< span class = "hljs-comment" > # 导 入 glusterfs-endpoints.json< / span >
$ kubectl apply < span class = "hljs-_" > -f< / span > glusterfs-endpoints.json
< span class = "hljs-comment" > # 查 看 endpoints 信 息 < / span >
$ kubectl get ep
< / code > < / pre >
< h2 id = "配置-service" > 配 置 service< / h2 >
< pre > < code class = "lang-bash" > $ curl -O https://raw.githubusercontent.com/kubernetes/kubernetes/master/examples/volumes/glusterfs/glusterfs-service.json
< span class = "hljs-comment" > # service.json 里 面 查 找 的 是 enpointes 的 名 称 与 端 口 ， 端 口 默 认 配 置 为 1， 我 改 成 了 1990< / span >
< span class = "hljs-comment" > # 导 入 glusterfs-service.json< / span >
$ kubectl apply < span class = "hljs-_" > -f< / span > glusterfs-service.json
< span class = "hljs-comment" > # 查 看 service 信 息 < / span >
$ kubectl get svc
< / code > < / pre >
< h2 id = "创建测试-pod" > 创 建 测 试 pod< / h2 >
< pre > < code class = "lang-bash" > $ curl -O https://raw.githubusercontent.com/kubernetes/kubernetes/master/examples/volumes/glusterfs/glusterfs-pod.json
< span class = "hljs-comment" > # 编 辑 glusterfs-pod.json< / span >
< span class = "hljs-comment" > # 修 改 volumes 下 的 path 为 上 面 创 建 的 volume 名 称 < / span >
< span class = "hljs-string" > " path" < / span > : < span class = "hljs-string" > " k8s-volume" < / span >
< span class = "hljs-comment" > # 导 入 glusterfs-pod.json< / span >
$ kubectl apply < span class = "hljs-_" > -f< / span > glusterfs-pod.json
< span class = "hljs-comment" > # 查 看 pods 状 态 < / span >
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
glusterfs 1/1 Running 0 1m
< span class = "hljs-comment" > # 查 看 pods 所 在 node< / span >
$ kubectl describe pods/glusterfs
< span class = "hljs-comment" > # 登 陆 node 物 理 机 ， 使 用 df 可 查 看 挂 载 目 录 < / span >
$ df -h
172.20.0.113:k8s-volume 1073741824 0 1073741824 0% 172.20.0.113:k8s-volume 1.0T 0 1.0T 0% /var/lib/kubelet/pods/3de9< span class = "hljs-built_in" > fc< / span > 69-30b7-11e7-bfbd-8af1e3a7c5bd/volumes/kubernetes.io~glusterfs/glusterfsvol
< / code > < / pre >
< h2 id = "配置persistentvolume" > 配 置 PersistentVolume< / h2 >
< p > PersistentVolume（ PV） 和 PersistentVolumeClaim（ PVC） 是 kubernetes提 供 的 两 种 API资 源 ， 用 于 抽 象 存 储 细 节 。 管 理 员 关 注 于 如 何 通 过 pv提 供 存 储 功 能 而 无 需 关 注 用 户 如 何 使 用 ， 同 样 的 用 户 只 需 要 挂 载 PVC到 容 器 中 而 不 需 要 关 注 存 储 卷 采 用 何 种 技 术 实 现 。 < / p >
< p > PVC和 PV的 关 系 跟 pod和 node关 系 类 似 ， 前 者 消 耗 后 者 的 资 源 。 PVC可 以 向 PV申 请 指 定 大 小 的 存 储 资 源 并 设 置 访 问 模 式 。 < / p >
< p > < strong > PV属 性 < / strong > < / p >
< ul >
< li > storage容 量 < / li >
< li > 读 写 属 性 ： 分 别 为 ReadWriteOnce： 单 个 节 点 读 写 ； ReadOnlyMany： 多 节 点 只 读 ； ReadWriteMany： 多 节 点 读 写 < / li >
< / ul >
< pre > < code class = "lang-bash" > $ cat glusterfs-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: gluster-dev-volume
spec:
capacity:
storage: 8Gi
accessModes:
- ReadWriteMany
glusterfs:
endpoints: < span class = "hljs-string" > " glusterfs-cluster" < / span >
path: < span class = "hljs-string" > " k8s-volume" < / span >
< span class = "hljs-built_in" > read< / span > Only: < span class = "hljs-literal" > false< / span >
< span class = "hljs-comment" > # 导 入 PV< / span >
$ kubectl apply < span class = "hljs-_" > -f< / span > glusterfs-pv.yaml
< span class = "hljs-comment" > # 查 看 pv< / span >
$ kubectl get pv
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM STORAGECLASS REASON AGE
gluster-dev-volume 8Gi RWX Retain Available 3s
< / code > < / pre >
< p > PVC属 性 < / p >
< ul >
< li > 访 问 属 性 与 PV相 同 < / li >
< li > 容 量 ： 向 PV申 请 的 容 量 < = PV总 容 量 < / li >
< / ul >
< h2 id = "配置pvc" > 配 置 PVC< / h2 >
< pre > < code class = "lang-Bash" > $ cat glusterfs-pvc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: glusterfs-nginx
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 8Gi
< span class = "hljs-comment" > # 导 入 pvc< / span >
$ kubectl apply < span class = "hljs-_" > -f< / span > glusterfs-pvc.yaml
< span class = "hljs-comment" > # 查 看 pvc< / span >
$ kubectl get pv
NAME STATUS VOLUME CAPACITY ACCESSMODES STORAGECLASS AGE
glusterfs-nginx Bound gluster-dev-volume 8Gi RWX 4s
< / code > < / pre >
< h2 id = "创建-nginx-deployment-挂载-volume" > 创 建 nginx deployment 挂 载 volume< / h2 >
< pre > < code class = "lang-Bash" > $ vi nginx-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-dm
spec:
replicas: 2
template:
metadata:
labels:
name: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
volumeMounts:
- name: gluster-dev-volume
mountPath: < span class = "hljs-string" > " /usr/share/nginx/html" < / span >
volumes:
- name: gluster-dev-volume
persistentVolumeClaim:
claimName: glusterfs-nginx
< span class = "hljs-comment" > # 导 入 deployment< / span >
$ kubectl apply < span class = "hljs-_" > -f< / span > nginx-deployment.yaml
< span class = "hljs-comment" > # 查 看 deployment< / span >
$ kubectl get pods |grep nginx-dm
nginx-dm-3698525684-g0mvt 1/1 Running 0 6s
nginx-dm-3698525684-hbzq1 1/1 Running 0 6s
< span class = "hljs-comment" > # 查 看 挂 载 < / span >
$ kubectl < span class = "hljs-built_in" > exec< / span > -it nginx-dm-3698525684-g0mvt -- df -h|grep k8s-volume
172.20.0.113:k8s-volume 1.0T 0 1.0T 0% /usr/share/nginx/html
< span class = "hljs-comment" > # 创 建 文 件 测 试 < / span >
$ kubectl < span class = "hljs-built_in" > exec< / span > -it nginx-dm-3698525684-g0mvt -- touch /usr/share/nginx/html/index.html
$ kubectl < span class = "hljs-built_in" > exec< / span > -it nginx-dm-3698525684-g0mvt -- ls < span class = "hljs-_" > -lt< / span > /usr/share/nginx/html/index.html
-rw-r--r-- 1 root root 0 May 4 11:36 /usr/share/nginx/html/index.html
< span class = "hljs-comment" > # 验 证 glusterfs< / span >
< span class = "hljs-comment" > # 因 为 我 们 使 用 分 布 卷 ， 所 以 可 以 看 到 某 个 节 点 中 有 文 件 < / span >
[root@sz-pg-oam-docker-test-001 ~] ls /opt/gfs_data/
[root@sz-pg-oam-docker-test-002 ~] ls /opt/gfs_data/
index.html
[root@sz-pg-oam-docker-test-003 ~] ls /opt/gfs_data/
< / code > < / pre >
< h2 id = "参考" > 参 考 < / h2 >
< p > < a href = "https://www.xf80.com/2017/04/21/kubernetes-glusterfs/?utm_source=tuicool&utm_medium=referral#kubernetes-glusterfs" target = "_blank" > 在 kubernetes中 安 装 glusterfs< / a > < / p >
< p > < a href = "http://www.cnblogs.com/jicki/p/5801712.html" target = "_blank" > CentOS 7 安 装 GlusterFS< / a > < / p >
< p > < a href = "https://github.com/kubernetes/kubernetes/tree/master/examples/volumes/glusterfs" target = "_blank" > GlusterFS with kubernetes< / a > < / p >
< footer class = "page-footer-ex" > < span class = "page-footer-ex-copyright" > for GitBook< / span >                       < span class = "page-footer-ex-footer-update" > update
2017-08-21 18:23:35
< / span > < / footer >
< / section >
< / div >
< div class = "search-results" >
< div class = "has-results" >
< h1 class = "search-results-title" > < span class = 'search-results-count' > < / span > results matching "< span class = 'search-query' > < / span > "< / h1 >
< ul class = "search-results-list" > < / ul >
< / div >
< div class = "no-results" >
< h1 class = "search-results-title" > No results matching "< span class = 'search-query' > < / span > "< / h1 >
< / div >
< / div >
< / div >
< / div >
< / div >
< / div >
< a href = "glusterfs.html" class = "navigation navigation-prev " aria-label = "Previous page: 4.4.1 GlusterFS" >
< i class = "fa fa-angle-left" > < / i >
< / a >
< a href = "storage-for-containers-using-glusterfs-with-openshift.html" class = "navigation navigation-next " aria-label = "Next page: 4.4.1.2 在OpenShift中使用GlusterFS做持久化存储" >
< i class = "fa fa-angle-right" > < / i >
< / a >
< / div >
< script >
var gitbook = gitbook || [];
gitbook.push(function() {
2017-08-22 11:43:22 +08:00
gitbook.page.hasChanged({"page":{"title":"4.4.1.1 使用GlusterFS做持久化存储","level":"1.4.4.1.1","depth":4,"next":{"title":"4.4.1.2 在OpenShift中使用GlusterFS做持久化存储","level":"1.4.4.1.2","depth":4,"path":"practice/storage-for-containers-using-glusterfs-with-openshift.md","ref":"practice/storage-for-containers-using-glusterfs-with-openshift.md","articles":[]},"previous":{"title":"4.4.1 GlusterFS","level":"1.4.4.1","depth":3,"path":"practice/glusterfs.md","ref":"practice/glusterfs.md","articles":[{"title":"4.4.1.1 使用GlusterFS做持久化存储","level":"1.4.4.1.1","depth":4,"path":"practice/using-glusterfs-for-persistent-storage.md","ref":"practice/using-glusterfs-for-persistent-storage.md","articles":[]},{"title":"4.4.1.2 在OpenShift中使用GlusterFS做持久化存储","level":"1.4.4.1.2","depth":4,"path":"practice/storage-for-containers-using-glusterfs-with-openshift.md","ref":"practice/storage-for-containers-using-glusterfs-with-openshift.md","articles":[]}]},"dir":"ltr"},"config":{"plugins":["github","codesnippet","splitter","page-toc-button","image-captions","page-footer-ex","editlink","-lunr","-search","search-plus"],"styles":{"website":"styles/website.css","pdf":"styles/pdf.css","epub":"styles/epub.css","mobi":"styles/mobi.css","ebook":"styles/ebook.css","print":"styles/print.css"},"pluginsConfig":{"github":{"url":"https://github.com/rootsongjc/kubernetes-handbook"},"editlink":{"label":"编辑本页","multilingual":false,"base":"https://github.com/rootsongjc/kubernetes-handbook/blob/master/"},"page-footer-ex":{"copyright":"for GitBook","update_format":"YYYY-MM-DD HH:mm:ss","update_label":"update"},"splitter":{},"codesnippet":{},"fontsettings":{"theme":"white","family":"sans","size":2},"highlight":{},"page-toc-button":{},"sharing":{"facebook":true,"twitter":true,"google":false,"weibo":false,"instapaper":false,"vk":false,"all":["facebook","google","twitter","weibo","instapaper"]},"theme-default":{"styles":{"website":"styles/website.css","pdf":"styles/pdf.css","epub":"styles/epub.css","mobi":"styles/mobi.css","ebook":"styles/ebook.css","print":"styles/print.css"},"showLevel":false},"search-plus":{},"image-captions":{"variable_name":"_pictures"}},"page-footer-ex":{"copyright":"Jimmy Song","update_label":"最后更新：","update_format":"YYYY-MM-DD HH:mm:ss"},"theme":"default","author":"Jimmy Song","pdf":{"pageNumbers":true,"fontSize":12,"fontFamily":"Arial","paperSize":"a4","chapterMark":"pagebreak","pageBreaksBefore":"/","margin":{"right":62,"left":62,"top":56,"bottom":56}},"structure":{"langs":"LANGS.md","readme":"README.md","glossary":"GLOSSARY.md","summary":"SUMMARY.md"},"variables":{"_pictures":[{"backlink":"index.html#fig1.1.1","level":"1.1","list_caption":"Figure: wercker status","alt":"wercker status","nro":1,"url":"https://app.wercker.com/status/b8b69e593784e17ddcfd1286adfd8f3c/s/master","index":1,"caption_template":"Figure: _CAPTION_","label":"wercker status","attributes":{},"title":"wercker status","skip":false,"key":"1.1.1"},{"backlink":"concepts/index.html#fig1.2.1","level":"1.2","list_caption":"Figure: Borg架构","alt":"Borg架构","nro":2,"url":"../images/borg.png","index":1,"caption_template":"Figure: _CAPTION_","label":"Borg架构","attributes":{},"skip":false,"key":"1.2.1"},{"backlink":"concepts/index.html#fig1.2.2","level":"1.2","list_caption":"Figure: Kubernetes架构","alt":"Kubernetes架构","nro":3,"url":"../images/architecture.png","index":2,"caption_template":"Figure: _CAPTION_","label":"Kubernetes架构","attributes":{},"skip":false,"key":"1.2.2"},{"backlink":"concepts/index.html#fig1.2.3","level":"1.2","list_caption":"Figure: kubernetes整体架构示意图","alt":"kubernetes整体架构示意图","nro":4,"url":"../images/kubernetes-whole-arch.png","index":3,"caption_template":"Figure: _CAPTION_","label":"kubernetes整体架构示意图","attributes":{},"skip":false,"key":"1.2.3"},{"backlink":"concepts/index.html#fig1.2.4","level":"1.2","list_caption":"Figure: Kubernetes master架构示意图","alt":"Kubernetes master架构示意图","nro":5,"url":".
2017-08-21 18:44:34 +08:00
});
< / script >
< / div >
< script src = "../gitbook/gitbook.js" > < / script >
< script src = "../gitbook/theme.js" > < / script >
< script src = "../gitbook/gitbook-plugin-github/plugin.js" > < / script >
< script src = "../gitbook/gitbook-plugin-splitter/splitter.js" > < / script >
< script src = "../gitbook/gitbook-plugin-page-toc-button/plugin.js" > < / script >
< script src = "../gitbook/gitbook-plugin-editlink/plugin.js" > < / script >
< script src = "../gitbook/gitbook-plugin-search-plus/jquery.mark.min.js" > < / script >
< script src = "../gitbook/gitbook-plugin-search-plus/search.js" > < / script >
< script src = "../gitbook/gitbook-plugin-sharing/buttons.js" > < / script >
< script src = "../gitbook/gitbook-plugin-fontsettings/fontsettings.js" > < / script >
< / body >
< / html >
