kubernetes-handbook/usecases/spark-standalone-on-kuberne...

1948 lines
86 KiB
HTML
Raw Normal View History

2017-08-30 14:20:52 +08:00
<!DOCTYPE HTML>
2017-09-19 22:01:07 +08:00
<html lang="zh-hans" >
2017-08-30 14:20:52 +08:00
<head>
<meta charset="UTF-8">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<title>5.2.1 Spark standalone on Kubernetes · Kubernetes Handbook</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="description" content="">
<meta name="generator" content="GitBook 3.2.2">
<meta name="author" content="Jimmy Song">
<link rel="stylesheet" href="../gitbook/style.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-splitter/splitter.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-page-toc-button/plugin.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-image-captions/image-captions.css">
2017-10-09 15:28:05 +08:00
<link rel="stylesheet" href="../gitbook/gitbook-plugin-back-to-top-button/plugin.css">
2017-08-30 14:20:52 +08:00
2017-10-09 15:28:05 +08:00
<link rel="stylesheet" href="../gitbook/gitbook-plugin-search-plus/search.css">
2017-09-19 21:38:03 +08:00
2017-10-09 15:28:05 +08:00
<link rel="stylesheet" href="../gitbook/gitbook-plugin-tbfed-pagefooter/footer.css">
2017-08-30 14:20:52 +08:00
<link rel="stylesheet" href="../gitbook/gitbook-plugin-highlight/website.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-fontsettings/website.css">
2017-10-09 15:28:05 +08:00
2017-08-30 14:20:52 +08:00
<meta name="HandheldFriendly" content="true"/>
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black">
<link rel="apple-touch-icon-precomposed" sizes="152x152" href="../gitbook/images/apple-touch-icon-precomposed-152.png">
<link rel="shortcut icon" href="../gitbook/images/favicon.ico" type="image/x-icon">
2017-09-14 15:57:50 +08:00
<link rel="next" href="running-spark-with-kubernetes-native-scheduler.html" />
2017-08-30 14:20:52 +08:00
<link rel="prev" href="big-data.html" />
2017-10-09 15:28:05 +08:00
<link rel="shortcut icon" href='../favicon.ico' type="image/x-icon">
<link rel="bookmark" href='../favicon.ico' type="image/x-icon">
<style>
@media only screen and (max-width: 640px) {
.book-header .hidden-mobile {
display: none;
}
}
</style>
<script>
window["gitbook-plugin-github-buttons"] = {"repo":"rootsongjc/kubernetes-handbook","types":["star"],"size":"small"};
</script>
2017-08-30 14:20:52 +08:00
</head>
<body>
<div class="book">
<div class="book-summary">
<div id="book-search-input" role="search">
2017-09-19 22:01:07 +08:00
<input type="text" placeholder="输入并搜索" />
2017-08-30 14:20:52 +08:00
</div>
<nav role="navigation">
<ul class="summary">
2017-10-09 15:28:05 +08:00
<li>
<a href="https://jimmysong.io" target="_blank" class="custom-link">Home</a>
</li>
2017-08-30 14:20:52 +08:00
2017-10-09 15:28:05 +08:00
<li class="divider"></li>
2017-08-30 14:20:52 +08:00
<li class="chapter " data-level="1.1" data-path="../">
<a href="../">
序言
2017-08-30 14:20:52 +08:00
</a>
</li>
<li class="chapter " data-level="1.2" data-path="../cloud-native/kubernetes-and-cloud-native-app-overview.html">
<a href="../cloud-native/kubernetes-and-cloud-native-app-overview.html">
1. Kubernetes与云原生应用概览
</a>
</li>
<li class="chapter " data-level="1.3" data-path="../concepts/">
2017-08-30 14:20:52 +08:00
<a href="../concepts/">
2. 概念原理
</a>
<ul class="articles">
<li class="chapter " data-level="1.3.1" data-path="../concepts/concepts.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/concepts.html">
2.1 设计理念
</a>
</li>
<li class="chapter " data-level="1.3.2" data-path="../concepts/objects.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/objects.html">
2017-09-03 13:29:38 +08:00
2.2 Objects
2017-08-30 14:20:52 +08:00
</a>
<ul class="articles">
<li class="chapter " data-level="1.3.2.1" data-path="../concepts/pod-overview.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/pod-overview.html">
2.2.1 Pod
</a>
<ul class="articles">
<li class="chapter " data-level="1.3.2.1.1" data-path="../concepts/pod.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/pod.html">
2.2.1.1 Pod解析
</a>
2017-08-31 23:28:33 +08:00
</li>
<li class="chapter " data-level="1.3.2.1.2" data-path="../concepts/init-containers.html">
2017-08-31 23:28:33 +08:00
<a href="../concepts/init-containers.html">
2.2.1.2 Init容器
</a>
2017-09-03 15:58:39 +08:00
</li>
<li class="chapter " data-level="1.3.2.1.3" data-path="../concepts/pod-security-policy.html">
2017-09-03 15:58:39 +08:00
<a href="../concepts/pod-security-policy.html">
2.2.1.3 Pod安全策略
</a>
2017-09-17 15:39:26 +08:00
</li>
<li class="chapter " data-level="1.3.2.1.4" data-path="../concepts/pod-lifecycle.html">
2017-09-17 15:39:26 +08:00
<a href="../concepts/pod-lifecycle.html">
2.2.1.4 Pod的生命周期
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
<li class="chapter " data-level="1.3.2.2" data-path="../concepts/node.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/node.html">
2.2.2 Node
</a>
</li>
<li class="chapter " data-level="1.3.2.3" data-path="../concepts/namespace.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/namespace.html">
2.2.3 Namespace
</a>
</li>
<li class="chapter " data-level="1.3.2.4" data-path="../concepts/service.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/service.html">
2.2.4 Service
</a>
</li>
<li class="chapter " data-level="1.3.2.5" data-path="../concepts/volume.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/volume.html">
2.2.5 Volume和Persistent Volume
</a>
</li>
<li class="chapter " data-level="1.3.2.6" data-path="../concepts/deployment.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/deployment.html">
2.2.6 Deployment
</a>
</li>
<li class="chapter " data-level="1.3.2.7" data-path="../concepts/secret.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/secret.html">
2.2.7 Secret
</a>
</li>
<li class="chapter " data-level="1.3.2.8" data-path="../concepts/statefulset.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/statefulset.html">
2.2.8 StatefulSet
</a>
</li>
<li class="chapter " data-level="1.3.2.9" data-path="../concepts/daemonset.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/daemonset.html">
2.2.9 DaemonSet
</a>
</li>
<li class="chapter " data-level="1.3.2.10" data-path="../concepts/serviceaccount.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/serviceaccount.html">
2.2.10 ServiceAccount
</a>
</li>
<li class="chapter " data-level="1.3.2.11" data-path="../concepts/replicaset.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/replicaset.html">
2.2.11 ReplicationController和ReplicaSet
</a>
</li>
<li class="chapter " data-level="1.3.2.12" data-path="../concepts/job.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/job.html">
2.2.12 Job
</a>
</li>
<li class="chapter " data-level="1.3.2.13" data-path="../concepts/cronjob.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/cronjob.html">
2.2.13 CronJob
</a>
</li>
<li class="chapter " data-level="1.3.2.14" data-path="../concepts/ingress.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/ingress.html">
2.2.14 Ingress
</a>
</li>
<li class="chapter " data-level="1.3.2.15" data-path="../concepts/configmap.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/configmap.html">
2.2.15 ConfigMap
</a>
</li>
<li class="chapter " data-level="1.3.2.16" data-path="../concepts/horizontal-pod-autoscaling.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/horizontal-pod-autoscaling.html">
2.2.16 Horizontal Pod Autoscaling
</a>
</li>
<li class="chapter " data-level="1.3.2.17" data-path="../concepts/label.html">
2017-08-30 14:20:52 +08:00
<a href="../concepts/label.html">
2.2.17 Label
</a>
2017-09-03 15:58:39 +08:00
</li>
<li class="chapter " data-level="1.3.2.18" data-path="../concepts/garbage-collection.html">
2017-09-03 15:58:39 +08:00
<a href="../concepts/garbage-collection.html">
2.2.18 垃圾收集
</a>
</li>
<li class="chapter " data-level="1.3.2.19" data-path="../concepts/network-policy.html">
2017-09-03 15:58:39 +08:00
<a href="../concepts/network-policy.html">
2.2.19 NetworkPolicy
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
</ul>
</li>
<li class="chapter " data-level="1.4" data-path="../guide/">
2017-08-30 14:20:52 +08:00
<a href="../guide/">
3. 用户指南
</a>
<ul class="articles">
<li class="chapter " data-level="1.4.1" data-path="../guide/resource-configuration.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/resource-configuration.html">
2017-09-28 21:20:49 +08:00
3.1 资源对象配置
2017-08-30 14:20:52 +08:00
</a>
<ul class="articles">
<li class="chapter " data-level="1.4.1.1" data-path="../guide/configure-liveness-readiness-probes.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/configure-liveness-readiness-probes.html">
3.1.1 配置Pod的liveness和readiness探针
</a>
</li>
<li class="chapter " data-level="1.4.1.2" data-path="../guide/configure-pod-service-account.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/configure-pod-service-account.html">
3.1.2 配置Pod的Service Account
</a>
2017-09-28 21:20:49 +08:00
</li>
<li class="chapter " data-level="1.4.1.3" data-path="../guide/secret-configuration.html">
2017-09-28 21:20:49 +08:00
<a href="../guide/secret-configuration.html">
3.1.3 Secret配置
</a>
2017-10-10 14:51:45 +08:00
</li>
<li class="chapter " data-level="1.4.1.4" data-path="../guide/resource-quota-management.html">
2017-10-10 14:51:45 +08:00
<a href="../guide/resource-quota-management.html">
3.2.3 管理namespace中的资源配额
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
<li class="chapter " data-level="1.4.2" data-path="../guide/command-usage.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/command-usage.html">
3.2 命令使用
</a>
<ul class="articles">
<li class="chapter " data-level="1.4.2.1" data-path="../guide/using-kubectl.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/using-kubectl.html">
3.2.1 使用kubectl
</a>
</li>
<li class="chapter " data-level="1.4.2.2" data-path="../guide/docker-cli-to-kubectl.html">
<a href="../guide/docker-cli-to-kubectl.html">
3.2.2 docker用户过度到kubectl命令行指南
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
<li class="chapter " data-level="1.4.3" data-path="../guide/cluster-security-management.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/cluster-security-management.html">
2017-08-30 14:20:52 +08:00
3.3 集群安全性管理
2017-08-30 14:20:52 +08:00
</a>
<ul class="articles">
<li class="chapter " data-level="1.4.3.1" data-path="../guide/managing-tls-in-a-cluster.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/managing-tls-in-a-cluster.html">
3.3.1 管理集群中的TLS
</a>
</li>
<li class="chapter " data-level="1.4.3.2" data-path="../guide/kubelet-authentication-authorization.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/kubelet-authentication-authorization.html">
3.3.2 kubelet的认证授权
</a>
</li>
<li class="chapter " data-level="1.4.3.3" data-path="../guide/tls-bootstrapping.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/tls-bootstrapping.html">
3.3.3 TLS bootstrap
</a>
2017-08-31 14:23:44 +08:00
</li>
<li class="chapter " data-level="1.4.3.4" data-path="../guide/kubectl-user-authentication-authorization.html">
2017-08-31 14:23:44 +08:00
<a href="../guide/kubectl-user-authentication-authorization.html">
3.3.4 kubectl的用户认证授权
</a>
</li>
<li class="chapter " data-level="1.4.3.5" data-path="../guide/rbac.html">
2017-08-31 14:23:44 +08:00
<a href="../guide/rbac.html">
3.3.5 RBAC——基于角色的访问控制
</a>
2017-09-07 14:13:59 +08:00
</li>
<li class="chapter " data-level="1.4.3.6" data-path="../guide/ip-masq-agent.html">
2017-09-07 14:13:59 +08:00
<a href="../guide/ip-masq-agent.html">
3.3.6 IP伪装代理
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
<li class="chapter " data-level="1.4.4" data-path="../guide/access-kubernetes-cluster.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/access-kubernetes-cluster.html">
3.4 访问 Kubernetes 集群
</a>
<ul class="articles">
<li class="chapter " data-level="1.4.4.1" data-path="../guide/access-cluster.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/access-cluster.html">
3.4.1 访问集群
</a>
</li>
<li class="chapter " data-level="1.4.4.2" data-path="../guide/authenticate-across-clusters-kubeconfig.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/authenticate-across-clusters-kubeconfig.html">
3.4.2 使用 kubeconfig 文件配置跨集群认证
</a>
</li>
<li class="chapter " data-level="1.4.4.3" data-path="../guide/connecting-to-applications-port-forward.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/connecting-to-applications-port-forward.html">
3.4.3 通过端口转发访问集群中的应用程序
</a>
</li>
<li class="chapter " data-level="1.4.4.4" data-path="../guide/service-access-application-cluster.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/service-access-application-cluster.html">
3.4.4 使用 service 访问群集中的应用程序
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="1.4.5" data-path="../guide/application-development-deployment-flow.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/application-development-deployment-flow.html">
3.5 在kubernetes中开发部署应用
</a>
<ul class="articles">
<li class="chapter " data-level="1.4.5.1" data-path="../guide/deploy-applications-in-kubernetes.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/deploy-applications-in-kubernetes.html">
3.5.1 适用于kubernetes的应用开发部署流程
</a>
</li>
<li class="chapter " data-level="1.4.5.2" data-path="../guide/migrating-hadoop-yarn-to-kubernetes.html">
2017-08-30 14:20:52 +08:00
<a href="../guide/migrating-hadoop-yarn-to-kubernetes.html">
3.5.2 迁移传统应用到kubernetes中——以Hadoop YARN为例
</a>
2017-10-23 13:40:58 +08:00
</li>
<li class="chapter " data-level="1.4.5.3" data-path="../guide/using-statefulset.html">
<a href="../guide/using-statefulset.html">
3.5.3 使用StatefulSet部署用状态应用
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
</ul>
</li>
<li class="chapter " data-level="1.5" data-path="../practice/">
2017-08-30 14:20:52 +08:00
<a href="../practice/">
4. 最佳实践
</a>
<ul class="articles">
<li class="chapter " data-level="1.5.1" data-path="../practice/install-kbernetes1.6-on-centos.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/install-kbernetes1.6-on-centos.html">
4.1 在CentOS上部署kubernetes1.6集群
</a>
<ul class="articles">
<li class="chapter " data-level="1.5.1.1" data-path="../practice/create-tls-and-secret-key.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/create-tls-and-secret-key.html">
4.1.1 创建TLS证书和秘钥
</a>
</li>
<li class="chapter " data-level="1.5.1.2" data-path="../practice/create-kubeconfig.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/create-kubeconfig.html">
4.1.2 创建kubeconfig文件
</a>
</li>
<li class="chapter " data-level="1.5.1.3" data-path="../practice/etcd-cluster-installation.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/etcd-cluster-installation.html">
4.1.3 创建高可用etcd集群
</a>
</li>
<li class="chapter " data-level="1.5.1.4" data-path="../practice/kubectl-installation.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/kubectl-installation.html">
4.1.4 安装kubectl命令行工具
</a>
</li>
<li class="chapter " data-level="1.5.1.5" data-path="../practice/master-installation.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/master-installation.html">
4.1.5 部署master节点
</a>
</li>
<li class="chapter " data-level="1.5.1.6" data-path="../practice/node-installation.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/node-installation.html">
4.1.6 部署node节点
</a>
</li>
<li class="chapter " data-level="1.5.1.7" data-path="../practice/kubedns-addon-installation.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/kubedns-addon-installation.html">
4.1.7 安装kubedns插件
</a>
</li>
<li class="chapter " data-level="1.5.1.8" data-path="../practice/dashboard-addon-installation.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/dashboard-addon-installation.html">
4.1.8 安装dashboard插件
</a>
</li>
<li class="chapter " data-level="1.5.1.9" data-path="../practice/heapster-addon-installation.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/heapster-addon-installation.html">
4.1.9 安装heapster插件
</a>
</li>
<li class="chapter " data-level="1.5.1.10" data-path="../practice/efk-addon-installation.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/efk-addon-installation.html">
4.1.10 安装EFK插件
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="1.5.2" data-path="../practice/service-discovery-and-loadbalancing.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/service-discovery-and-loadbalancing.html">
4.2 服务发现与负载均衡
</a>
<ul class="articles">
<li class="chapter " data-level="1.5.2.1" data-path="../practice/traefik-ingress-installation.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/traefik-ingress-installation.html">
4.2.1 安装Traefik ingress
</a>
</li>
<li class="chapter " data-level="1.5.2.2" data-path="../practice/distributed-load-test.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/distributed-load-test.html">
4.2.2 分布式负载测试
</a>
</li>
<li class="chapter " data-level="1.5.2.3" data-path="../practice/network-and-cluster-perfermance-test.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/network-and-cluster-perfermance-test.html">
4.2.3 网络和集群性能测试
</a>
</li>
<li class="chapter " data-level="1.5.2.4" data-path="../practice/edge-node-configuration.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/edge-node-configuration.html">
4.2.4 边缘节点配置
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="1.5.3" data-path="../practice/operation.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/operation.html">
4.3 运维管理
</a>
<ul class="articles">
<li class="chapter " data-level="1.5.3.1" data-path="../practice/service-rolling-update.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/service-rolling-update.html">
4.3.1 服务滚动升级
</a>
</li>
<li class="chapter " data-level="1.5.3.2" data-path="../practice/app-log-collection.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/app-log-collection.html">
4.3.2 应用日志收集
</a>
</li>
<li class="chapter " data-level="1.5.3.3" data-path="../practice/configuration-best-practice.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/configuration-best-practice.html">
4.3.3 配置最佳实践
</a>
</li>
<li class="chapter " data-level="1.5.3.4" data-path="../practice/monitor.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/monitor.html">
4.3.4 集群及应用监控
</a>
</li>
<li class="chapter " data-level="1.5.3.5" data-path="../practice/jenkins-ci-cd.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/jenkins-ci-cd.html">
4.3.5 使用Jenkins进行持续构建与发布
</a>
</li>
<li class="chapter " data-level="1.5.3.6" data-path="../practice/data-persistence-problem.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/data-persistence-problem.html">
4.3.6 数据持久化问题
</a>
2017-09-03 15:58:39 +08:00
</li>
<li class="chapter " data-level="1.5.3.7" data-path="../practice/manage-compute-resources-container.html">
2017-09-03 15:58:39 +08:00
<a href="../practice/manage-compute-resources-container.html">
4.3.7 管理容器的计算资源
</a>
</li>
<li class="chapter " data-level="1.5.3.8" data-path="../practice/using-prometheus-to-monitor-kuberentes-cluster.html">
<a href="../practice/using-prometheus-to-monitor-kuberentes-cluster.html">
4.3.8 使用Prometheus监控kubernetes集群
</a>
</li>
<li class="chapter " data-level="1.5.3.9" data-path="../practice/using-heapster-to-get-object-metrics.html">
<a href="../practice/using-heapster-to-get-object-metrics.html">
4.3.9 使用Heapster获取集群和对象的metric数据
</a>
2017-10-18 16:53:12 +08:00
</li>
<li class="chapter " data-level="1.5.3.10" data-path="../practice/manually-upgrade.html">
2017-10-18 16:53:12 +08:00
<a href="../practice/manually-upgrade.html">
4.3.10 手动集群升级
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
<li class="chapter " data-level="1.5.4" data-path="../practice/storage.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/storage.html">
4.4 存储管理
</a>
<ul class="articles">
<li class="chapter " data-level="1.5.4.1" data-path="../practice/glusterfs.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/glusterfs.html">
4.4.1 GlusterFS
</a>
<ul class="articles">
<li class="chapter " data-level="1.5.4.1.1" data-path="../practice/using-glusterfs-for-persistent-storage.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/using-glusterfs-for-persistent-storage.html">
4.4.1.1 使用GlusterFS做持久化存储
</a>
</li>
<li class="chapter " data-level="1.5.4.1.2" data-path="../practice/storage-for-containers-using-glusterfs-with-openshift.html">
2017-08-30 14:20:52 +08:00
<a href="../practice/storage-for-containers-using-glusterfs-with-openshift.html">
4.4.1.2 在OpenShift中使用GlusterFS做持久化存储
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="1.5.4.2" data-path="../practice/cephfs.html">
<a href="../practice/cephfs.html">
4.4.2 CephFS
</a>
<ul class="articles">
<li class="chapter " data-level="1.5.4.2.1" data-path="../practice/using-ceph-for-persistent-storage.html">
<a href="../practice/using-ceph-for-persistent-storage.html">
4.4.2.1 使用Ceph做持久化存储
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
</ul>
2017-10-19 15:31:08 +08:00
</li>
<li class="chapter " data-level="1.5.5" data-path="../practice/services-management-tool.html">
2017-10-19 15:31:08 +08:00
<a href="../practice/services-management-tool.html">
4.5 服务编排管理
</a>
2017-10-20 16:22:43 +08:00
<ul class="articles">
<li class="chapter " data-level="1.5.5.1" data-path="../practice/helm.html">
<a href="../practice/helm.html">
2017-10-25 17:12:37 +08:00
4.5.1 使用Helm管理kubernetes应用
2017-10-20 16:22:43 +08:00
</a>
</li>
</ul>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
<li class="chapter " data-level="1.6" data-path="./">
2017-08-30 14:20:52 +08:00
<a href="./">
5. 领域应用
</a>
<ul class="articles">
<li class="chapter " data-level="1.6.1" data-path="microservices.html">
2017-08-30 14:20:52 +08:00
<a href="microservices.html">
5.1 微服务架构
</a>
<ul class="articles">
<li class="chapter " data-level="1.6.1.1" data-path="service-discovery-in-microservices.html">
2017-09-20 21:55:19 +08:00
<a href="service-discovery-in-microservices.html">
5.1.1 微服务中的服务发现
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="1.6.2" data-path="service-mesh.html">
2017-09-20 21:55:19 +08:00
<a href="service-mesh.html">
5.2 Service Mesh 服务网格
</a>
<ul class="articles">
<li class="chapter " data-level="1.6.2.1" data-path="istio.html">
2017-08-30 14:20:52 +08:00
<a href="istio.html">
5.1.1 Istio
</a>
<ul class="articles">
<li class="chapter " data-level="1.6.2.1.1" data-path="istio-installation.html">
2017-08-30 14:20:52 +08:00
<a href="istio-installation.html">
5.1.1.1 安装istio
</a>
</li>
<li class="chapter " data-level="1.6.2.1.2" data-path="configuring-request-routing.html">
2017-08-30 14:20:52 +08:00
<a href="configuring-request-routing.html">
5.1.1.2 配置请求的路由规则
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="1.6.2.2" data-path="linkerd.html">
2017-08-30 14:20:52 +08:00
<a href="linkerd.html">
5.1.2 Linkerd
</a>
<ul class="articles">
<li class="chapter " data-level="1.6.2.2.1" data-path="linkerd-user-guide.html">
2017-08-30 14:20:52 +08:00
<a href="linkerd-user-guide.html">
5.1.2.1 Linkerd 使用指南
</a>
</li>
</ul>
</li>
</ul>
</li>
<li class="chapter " data-level="1.6.3" data-path="big-data.html">
2017-08-30 14:20:52 +08:00
<a href="big-data.html">
5.2 大数据
</a>
<ul class="articles">
<li class="chapter active" data-level="1.6.3.1" data-path="spark-standalone-on-kubernetes.html">
2017-08-30 14:20:52 +08:00
<a href="spark-standalone-on-kubernetes.html">
5.2.1 Spark standalone on Kubernetes
</a>
2017-08-31 14:23:44 +08:00
</li>
<li class="chapter " data-level="1.6.3.2" data-path="running-spark-with-kubernetes-native-scheduler.html">
2017-08-31 14:23:44 +08:00
2017-09-14 15:57:50 +08:00
<a href="running-spark-with-kubernetes-native-scheduler.html">
2017-08-31 14:23:44 +08:00
5.2.2 运行支持kubernetes原生调度的Spark程序
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
2017-08-30 16:52:33 +08:00
</li>
<li class="chapter " data-level="1.6.4" data-path="serverless.html">
2017-08-30 16:52:33 +08:00
<a href="serverless.html">
5.3 Serverless架构
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
<li class="chapter " data-level="1.7" data-path="../develop/">
2017-08-30 14:20:52 +08:00
<a href="../develop/">
6. 开发指南
</a>
<ul class="articles">
<li class="chapter " data-level="1.7.1" data-path="../develop/developing-environment.html">
2017-08-30 14:20:52 +08:00
<a href="../develop/developing-environment.html">
6.1 开发环境搭建
</a>
</li>
<li class="chapter " data-level="1.7.2" data-path="../develop/testing.html">
2017-08-30 14:20:52 +08:00
<a href="../develop/testing.html">
6.2 单元测试和集成测试
</a>
</li>
<li class="chapter " data-level="1.7.3" data-path="../develop/client-go-sample.html">
2017-08-30 14:20:52 +08:00
<a href="../develop/client-go-sample.html">
6.3 client-go示例
</a>
</li>
<li class="chapter " data-level="1.7.4" data-path="../develop/contribute.html">
2017-08-30 14:20:52 +08:00
<a href="../develop/contribute.html">
6.4 社区贡献
</a>
2017-10-23 19:09:41 +08:00
</li>
<li class="chapter " data-level="1.7.5" data-path="../develop/minikube.html">
<a href="../develop/minikube.html">
6.5 Minikube
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
<li class="chapter " data-level="1.8" data-path="../appendix/">
2017-08-30 14:20:52 +08:00
<a href="../appendix/">
7. 附录
</a>
<ul class="articles">
<li class="chapter " data-level="1.8.1" data-path="../appendix/docker-best-practice.html">
2017-08-30 14:20:52 +08:00
<a href="../appendix/docker-best-practice.html">
7.1 Docker最佳实践
</a>
</li>
<li class="chapter " data-level="1.8.2" data-path="../appendix/issues.html">
2017-08-30 14:20:52 +08:00
<a href="../appendix/issues.html">
7.2 问题记录
</a>
</li>
<li class="chapter " data-level="1.8.3" data-path="../appendix/tricks.html">
2017-08-30 14:20:52 +08:00
<a href="../appendix/tricks.html">
7.3 使用技巧
</a>
</li>
<li class="chapter " data-level="1.8.4" data-path="../appendix/debug-kubernetes-services.html">
<a href="../appendix/debug-kubernetes-services.html">
7.4 kubernetes service中的故障排查
</a>
2017-10-18 14:16:55 +08:00
</li>
<li class="chapter " data-level="1.8.5" data-path="../appendix/material-share.html">
2017-10-18 14:16:55 +08:00
<a href="../appendix/material-share.html">
2017-10-18 16:53:12 +08:00
7.5 Kubernetes相关资讯和情报链接
2017-10-18 14:16:55 +08:00
</a>
2017-08-30 14:20:52 +08:00
</li>
</ul>
</li>
<li class="divider"></li>
<li>
<a href="https://www.gitbook.com" target="blank" class="gitbook-link">
2017-09-19 22:01:07 +08:00
本书使用 GitBook 发布
2017-08-30 14:20:52 +08:00
</a>
</li>
</ul>
</nav>
</div>
<div class="book-body">
<div class="body-inner">
<div class="book-header" role="navigation">
<!-- Title -->
<h1>
<i class="fa fa-circle-o-notch fa-spin"></i>
<a href=".." >5.2.1 Spark standalone on Kubernetes</a>
</h1>
</div>
<div class="page-wrapper" tabindex="-1" role="main">
<div class="page-inner">
<div class="search-plus" id="book-search-results">
<div class="search-noresults">
<section class="normal markdown-section">
<h1 id="spark-standalone-on-kubernetes">Spark standalone on Kubernetes</h1>
<p>&#x8BE5;&#x9879;&#x76EE;&#x662F;&#x57FA;&#x4E8E; Spark standalone &#x6A21;&#x5F0F;&#xFF0C;&#x5BF9;&#x8D44;&#x6E90;&#x7684;&#x5206;&#x914D;&#x8C03;&#x5EA6;&#x8FD8;&#x6709;&#x4F5C;&#x4E1A;&#x72B6;&#x6001;&#x67E5;&#x8BE2;&#x7684;&#x529F;&#x80FD;&#x5B9E;&#x5728;&#x6709;&#x9650;&#xFF0C;&#x5BF9;&#x4E8E;&#x8BA9; spark &#x4F7F;&#x7528;&#x771F;&#x6B63;&#x539F;&#x751F;&#x7684; kubernetes &#x8D44;&#x6E90;&#x8C03;&#x5EA6;&#x63A8;&#x8350;&#x5927;&#x5BB6;&#x5C1D;&#x8BD5; <a href="https://github.com/apache-spark-on-k8s/" target="_blank">https://github.com/apache-spark-on-k8s/</a></p>
<p>&#x672C;&#x6587;&#x4E2D;&#x4F7F;&#x7528;&#x7684;&#x955C;&#x50CF;&#x6211;&#x5DF2;&#x7F16;&#x8BD1;&#x597D;&#x4E0A;&#x4F20;&#x5230;&#x4E86;&#x65F6;&#x901F;&#x4E91;&#x4E0A;&#xFF0C;&#x5927;&#x5BB6;&#x53EF;&#x4EE5;&#x76F4;&#x63A5;&#x4E0B;&#x8F7D;&#x3002;</p>
<pre><code>index.tenxcloud.com/jimmy/spark:1.5.2_v1
index.tenxcloud.com/jimmy/zeppelin:0.7.1
</code></pre><p>&#x4EE3;&#x7801;&#x548C;&#x4F7F;&#x7528;&#x6587;&#x6863;&#x89C1;Github&#x5730;&#x5740;&#xFF1A;<a href="https://github.com/rootsongjc/spark-on-kubernetes" target="_blank">https://github.com/rootsongjc/spark-on-kubernetes</a></p>
<p>&#x672C;&#x6587;&#x4E2D;&#x7528;&#x5230;&#x7684; yaml &#x6587;&#x4EF6;&#x53EF;&#x4EE5;&#x5728; <a href="../manifests/spark-standalone">../manifests/spark-standalone</a> &#x76EE;&#x5F55;&#x4E0B;&#x627E;&#x5230;&#xFF0C;&#x4E5F;&#x53EF;&#x4EE5;&#x5728;&#x4E0A;&#x9762;&#x7684; <a href="https://github.com/rootsongjc/spark-on-kubernetes/" target="_blank">https://github.com/rootsongjc/spark-on-kubernetes/</a> &#x9879;&#x76EE;&#x7684; manifests &#x76EE;&#x5F55;&#x4E0B;&#x627E;&#x5230;&#x3002;</p>
<p><strong>&#x6CE8;&#x610F;</strong>&#xFF1A;&#x65F6;&#x901F;&#x4E91;&#x4E0A;&#x672C;&#x6765;&#x5DF2;&#x7ECF;&#x63D0;&#x4F9B;&#x7684;&#x955C;&#x50CF; <code>index.tenxcloud.com/google_containers/spark:1.5.2_v1</code> &#xFF0C;&#x4F46;&#x662F;&#x8BE5;&#x955C;&#x50CF;&#x4F3C;&#x4E4E;&#x6709;&#x95EE;&#x9898;&#xFF0C;&#x4E0B;&#x8F7D;&#x603B;&#x662F;&#x5931;&#x8D25;&#x3002;</p>
<h2 id="&#x5728;kubernetes&#x4E0A;&#x542F;&#x52A8;spark">&#x5728;Kubernetes&#x4E0A;&#x542F;&#x52A8;spark</h2>
<p>&#x521B;&#x5EFA;&#x540D;&#x4E3A;spark-cluster&#x7684;namespace&#xFF0C;&#x6240;&#x6709;&#x64CD;&#x4F5C;&#x90FD;&#x5728;&#x8BE5;namespace&#x4E2D;&#x8FDB;&#x884C;&#x3002;</p>
<p>&#x6240;&#x6709;yaml&#x6587;&#x4EF6;&#x90FD;&#x5728;<code>manifests</code>&#x76EE;&#x5F55;&#x4E0B;&#x3002;</p>
<pre><code>$ kubectl create -f manifests/
</code></pre><p>&#x5C06;&#x4F1A;&#x542F;&#x52A8;&#x4E00;&#x4E2A;&#x62E5;&#x6709;&#x4E09;&#x4E2A;worker&#x7684;spark&#x96C6;&#x7FA4;&#x548C;zeppelin&#x3002;</p>
<p>&#x540C;&#x65F6;&#x5728;&#x8BE5;namespace&#x4E2D;&#x589E;&#x52A0;ingress&#x914D;&#x7F6E;&#xFF0C;&#x5C06;spark&#x7684;UI&#x548C;zeppelin&#x9875;&#x9762;&#x90FD;&#x66B4;&#x9732;&#x51FA;&#x6765;&#xFF0C;&#x53EF;&#x4EE5;&#x5728;&#x96C6;&#x7FA4;&#x5916;&#x90E8;&#x8BBF;&#x95EE;&#x3002;</p>
<p>&#x8BE5;ingress&#x540E;&#x7AEF;&#x4F7F;&#x7528;traefik&#x3002;</p>
<h2 id="&#x8BBF;&#x95EE;spark">&#x8BBF;&#x95EE;spark</h2>
<p>&#x901A;&#x8FC7;&#x4E0A;&#x9762;&#x5BF9;ingress&#x7684;&#x914D;&#x7F6E;&#x66B4;&#x9732;&#x670D;&#x52A1;&#xFF0C;&#x9700;&#x8981;&#x4FEE;&#x6539;&#x672C;&#x673A;&#x7684;/etc/hosts&#x6587;&#x4EF6;&#xFF0C;&#x589E;&#x52A0;&#x4EE5;&#x4E0B;&#x914D;&#x7F6E;&#xFF0C;&#x4F7F;&#x5176;&#x80FD;&#x591F;&#x89E3;&#x6790;&#x5230;&#x4E0A;&#x8FF0;service&#x3002;</p>
<pre><code>172.20.0.119 zeppelin.traefik.io
172.20.0.119 spark.traefik.io
</code></pre><p>172.20.0.119&#x662F;&#x6211;&#x8BBE;&#x7F6E;&#x7684;VIP&#x5730;&#x5740;&#xFF0C;VIP&#x7684;&#x8BBE;&#x7F6E;&#x548C;traefik&#x7684;&#x914D;&#x7F6E;&#x8BF7;&#x67E5;&#x770B;<a href="https://github.com/rootsongjc/kubernetes-handbook" target="_blank">kubernetes-handbook</a>&#x3002;</p>
<p><strong>spark ui</strong></p>
<p>&#x8BBF;&#x95EE;<a href="http://spark.traefik.io" target="_blank">http://spark.traefik.io</a></p>
<figure id="fig1.6.3.1.1"><img src="../images/spark-ui.jpg" alt="spark master ui"><figcaption>&#x56FE;&#x7247; - spark master ui</figcaption></figure>
2017-08-30 14:20:52 +08:00
<p><strong>zeppelin ui</strong></p>
<p>&#x8BBF;&#x95EE;<a href="http://zepellin.treafik.io" target="_blank">http://zepellin.treafik.io</a></p>
2017-10-09 15:28:05 +08:00
<p><img src="../images/zeppelin-ui.jpg" alt="zeppelin ui"> </p>
<footer class="page-footer"><span class="copyright">Copyright &#xA9; jimmysong.io 2017 all right reserved&#xFF0C;powered by Gitbook</span><span class="footer-modification">Updated:
2017-08-30 14:16:22
</span></footer>
2017-08-30 14:20:52 +08:00
</section>
</div>
<div class="search-results">
<div class="has-results">
<h1 class="search-results-title"><span class='search-results-count'></span> results matching "<span class='search-query'></span>"</h1>
<ul class="search-results-list"></ul>
</div>
<div class="no-results">
<h1 class="search-results-title">No results matching "<span class='search-query'></span>"</h1>
</div>
</div>
</div>
</div>
</div>
</div>
<a href="big-data.html" class="navigation navigation-prev " aria-label="Previous page: 5.2 大数据">
<i class="fa fa-angle-left"></i>
</a>
2017-09-14 15:57:50 +08:00
<a href="running-spark-with-kubernetes-native-scheduler.html" class="navigation navigation-next " aria-label="Next page: 5.2.2 运行支持kubernetes原生调度的Spark程序">
2017-08-30 14:20:52 +08:00
<i class="fa fa-angle-right"></i>
</a>
</div>
<script>
var gitbook = gitbook || [];
gitbook.push(function() {
2017-10-25 17:12:37 +08:00
gitbook.page.hasChanged({"page":{"title":"5.2.1 Spark standalone on Kubernetes","level":"1.6.3.1","depth":3,"next":{"title":"5.2.2 运行支持kubernetes原生调度的Spark程序","level":"1.6.3.2","depth":3,"path":"usecases/running-spark-with-kubernetes-native-scheduler.md","ref":"usecases/running-spark-with-kubernetes-native-scheduler.md","articles":[]},"previous":{"title":"5.2 大数据","level":"1.6.3","depth":2,"path":"usecases/big-data.md","ref":"usecases/big-data.md","articles":[{"title":"5.2.1 Spark standalone on Kubernetes","level":"1.6.3.1","depth":3,"path":"usecases/spark-standalone-on-kubernetes.md","ref":"usecases/spark-standalone-on-kubernetes.md","articles":[]},{"title":"5.2.2 运行支持kubernetes原生调度的Spark程序","level":"1.6.3.2","depth":3,"path":"usecases/running-spark-with-kubernetes-native-scheduler.md","ref":"usecases/running-spark-with-kubernetes-native-scheduler.md","articles":[]}]},"dir":"ltr"},"config":{"plugins":["github","codesnippet","splitter","page-toc-button","image-captions","editlink","back-to-top-button","-lunr","-search","search-plus","github-buttons@2.1.0","favicon@^0.0.2","tbfed-pagefooter@^0.0.1","3-ba"],"styles":{"website":"styles/website.css","pdf":"styles/pdf.css","epub":"styles/epub.css","mobi":"styles/mobi.css","ebook":"styles/ebook.css","print":"styles/print.css"},"pluginsConfig":{"tbfed-pagefooter":{"copyright":"Copyright © jimmysong.io 2017","modify_label":"Updated:","modify_format":"YYYY-MM-DD HH:mm:ss"},"github":{"url":"https://github.com/rootsongjc/kubernetes-handbook"},"editlink":{"label":"编辑本页","multilingual":false,"base":"https://github.com/rootsongjc/kubernetes-handbook/blob/master/"},"splitter":{},"codesnippet":{},"fontsettings":{"theme":"white","family":"sans","size":2},"highlight":{},"favicon":{"shortcut":"favicon.ico","bookmark":"favicon.ico"},"page-toc-button":{},"back-to-top-button":{},"github-buttons":{"repo":"rootsongjc/kubernetes-handbook","types":["star"],"size":"small"},"3-ba":{"configuration":"auto","token":"11f7d254cfa4e0ca44b175c66d379ecc"},"sharing":{"facebook":true,"twitter":true,"google":false,"weibo":false,"instapaper":false,"vk":false,"all":["facebook","google","twitter","weibo","instapaper"]},"theme-default":{"styles":{"website":"styles/website.css","pdf":"styles/pdf.css","epub":"styles/epub.css","mobi":"styles/mobi.css","ebook":"styles/ebook.css","print":"styles/print.css"},"showLevel":false},"search-plus":{},"image-captions":{"caption":"图片 - _CAPTION_","variable_name":"_pictures"}},"theme":"default","author":"Jimmy Song","pdf":{"pageNumbers":true,"fontSize":12,"fontFamily":"Arial","paperSize":"a4","chapterMark":"pagebreak","pageBreaksBefore":"/","margin":{"right":62,"left":62,"top":56,"bottom":56}},"structure":{"langs":"LANGS.md","readme":"README.md","glossary":"GLOSSARY.md","summary":"SUMMARY.md"},"variables":{"_pictures":[{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig1.2.1","level":"1.2","list_caption":"Figure: 云计算演进历程","alt":"云计算演进历程","nro":1,"url":"../images/cloud-computing-evolution-road.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"云计算演进历程","attributes":{},"skip":false,"key":"1.2.1"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig1.2.2","level":"1.2","list_caption":"Figure: Cloud native思维导图","alt":"Cloud native思维导图","nro":2,"url":"../images/cloud-native-architecutre-mindnode.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"Cloud native思维导图","attributes":{},"skip":false,"key":"1.2.2"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig1.2.3","level":"1.2","list_caption":"Figure: 十二因素应用","alt":"十二因素应用","nro":3,"url":"../images/12-factor-app.png","index":3,"caption_template":"图片 - _CAPTION_","label":"十二因素应用","attributes":{},"skip":false,"key":"1.2.3"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig1.2.4","level":"1.2","list_caption":"Figure: 使用Jenkins进行<E8BF9B><E8A18C>
2017-08-30 14:20:52 +08:00
});
</script>
</div>
<script src="../gitbook/gitbook.js"></script>
<script src="../gitbook/theme.js"></script>
<script src="../gitbook/gitbook-plugin-github/plugin.js"></script>
<script src="../gitbook/gitbook-plugin-splitter/splitter.js"></script>
<script src="../gitbook/gitbook-plugin-page-toc-button/plugin.js"></script>
<script src="../gitbook/gitbook-plugin-editlink/plugin.js"></script>
2017-09-19 21:38:03 +08:00
<script src="../gitbook/gitbook-plugin-back-to-top-button/plugin.js"></script>
2017-08-30 14:20:52 +08:00
<script src="../gitbook/gitbook-plugin-search-plus/jquery.mark.min.js"></script>
<script src="../gitbook/gitbook-plugin-search-plus/search.js"></script>
2017-10-09 15:28:05 +08:00
<script src="../gitbook/gitbook-plugin-github-buttons/plugin.js"></script>
2017-10-11 18:13:13 +08:00
<script src="../gitbook/gitbook-plugin-3-ba/plugin.js"></script>
2017-08-30 14:20:52 +08:00
<script src="../gitbook/gitbook-plugin-sharing/buttons.js"></script>
<script src="../gitbook/gitbook-plugin-fontsettings/fontsettings.js"></script>
</body>
</html>