kubernetes-handbook/usecases/running-spark-with-kubernet...

3292 lines
179 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

<!DOCTYPE HTML>
<html lang="zh-hans" >
<head>
<meta charset="UTF-8">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<title>运行支持Kubernetes原生调度的Spark程序 · Kubernetes Handbook - jimmysong.io</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="description" content="">
<meta name="generator" content="GitBook 3.2.2">
<meta name="author" content="Jimmy Song">
<link rel="stylesheet" href="../gitbook/style.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-splitter/splitter.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-page-toc-button/plugin.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-image-captions/image-captions.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-back-to-top-button/plugin.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-search-plus/search.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-tbfed-pagefooter/footer.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-highlight/website.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-fontsettings/website.css">
<meta name="HandheldFriendly" content="true"/>
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black">
<link rel="apple-touch-icon-precomposed" sizes="152x152" href="../gitbook/images/apple-touch-icon-precomposed-152.png">
<link rel="shortcut icon" href="../gitbook/images/favicon.ico" type="image/x-icon">
<link rel="next" href="serverless.html" />
<link rel="prev" href="spark-standalone-on-kubernetes.html" />
<link rel="shortcut icon" href='../favicon.ico' type="image/x-icon">
<link rel="bookmark" href='../favicon.ico' type="image/x-icon">
<style>
@media only screen and (max-width: 640px) {
.book-header .hidden-mobile {
display: none;
}
}
</style>
<script>
window["gitbook-plugin-github-buttons"] = {"repo":"rootsongjc/kubernetes-handbook","types":["star"],"size":"small"};
</script>
</head>
<body>
<div class="book">
<div class="book-summary">
<div id="book-search-input" role="search">
<input type="text" placeholder="输入并搜索" />
</div>
<nav role="navigation">
<ul class="summary">
<li>
<a href="https://jimmysong.io" target="_blank" class="custom-link">Jimmy Song</a>
</li>
<li class="divider"></li>
<li class="header">前言</li>
<li class="chapter " data-level="1.1" data-path="../">
<a href="../">
<b>1.1.</b>
序言
</a>
</li>
<li class="header">云原生</li>
<li class="chapter " data-level="2.1" data-path="../cloud-native/kubernetes-and-cloud-native-app-overview.html">
<a href="../cloud-native/kubernetes-and-cloud-native-app-overview.html">
<b>2.1.</b>
Kubernetes与云原生应用概览
</a>
</li>
<li class="chapter " data-level="2.2" data-path="../cloud-native/from-kubernetes-to-cloud-native.html">
<a href="../cloud-native/from-kubernetes-to-cloud-native.html">
<b>2.2.</b>
云原生应用之路——从Kubernetes到Cloud Native
</a>
</li>
<li class="header">概念与原理</li>
<li class="chapter " data-level="3.1" data-path="../concepts/">
<a href="../concepts/">
<b>3.1.</b>
Kubernetes架构
</a>
</li>
<li class="chapter " data-level="3.2" data-path="../concepts/concepts.html">
<a href="../concepts/concepts.html">
<b>3.2.</b>
设计理念
</a>
</li>
<li class="chapter " data-level="3.3" data-path="../concepts/objects.html">
<a href="../concepts/objects.html">
<b>3.3.</b>
资源对象与基本概念解析
</a>
<ul class="articles">
<li class="chapter " data-level="3.3.1" data-path="../concepts/pod-overview.html">
<a href="../concepts/pod-overview.html">
<b>3.3.1.</b>
Pod
</a>
<ul class="articles">
<li class="chapter " data-level="3.3.1.1" data-path="../concepts/pod.html">
<a href="../concepts/pod.html">
<b>3.3.1.1.</b>
Pod解析
</a>
</li>
<li class="chapter " data-level="3.3.1.2" data-path="../concepts/init-containers.html">
<a href="../concepts/init-containers.html">
<b>3.3.1.2.</b>
Init容器
</a>
</li>
<li class="chapter " data-level="3.3.1.3" data-path="../concepts/pod-security-policy.html">
<a href="../concepts/pod-security-policy.html">
<b>3.3.1.3.</b>
Pod安全策略
</a>
</li>
<li class="chapter " data-level="3.3.1.4" data-path="../concepts/pod-lifecycle.html">
<a href="../concepts/pod-lifecycle.html">
<b>3.3.1.4.</b>
Pod的生命周期
</a>
</li>
<li class="chapter " data-level="3.3.1.5" data-path="../concepts/pod-hook.html">
<a href="../concepts/pod-hook.html">
<b>3.3.1.5.</b>
Pod Hook
</a>
</li>
<li class="chapter " data-level="3.3.1.6" data-path="../concepts/pod-preset.html">
<a href="../concepts/pod-preset.html">
<b>3.3.1.6.</b>
Pod Preset
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="3.3.2" data-path="../concepts/node.html">
<a href="../concepts/node.html">
<b>3.3.2.</b>
Node
</a>
</li>
<li class="chapter " data-level="3.3.3" data-path="../concepts/namespace.html">
<a href="../concepts/namespace.html">
<b>3.3.3.</b>
Namespace
</a>
</li>
<li class="chapter " data-level="3.3.4" data-path="../concepts/service.html">
<a href="../concepts/service.html">
<b>3.3.4.</b>
Service
</a>
</li>
<li class="chapter " data-level="3.3.5" data-path="../concepts/volume.html">
<a href="../concepts/volume.html">
<b>3.3.5.</b>
Volume和Persistent Volume
</a>
</li>
<li class="chapter " data-level="3.3.6" data-path="../concepts/deployment.html">
<a href="../concepts/deployment.html">
<b>3.3.6.</b>
Deployment
</a>
</li>
<li class="chapter " data-level="3.3.7" data-path="../concepts/secret.html">
<a href="../concepts/secret.html">
<b>3.3.7.</b>
Secret
</a>
</li>
<li class="chapter " data-level="3.3.8" data-path="../concepts/statefulset.html">
<a href="../concepts/statefulset.html">
<b>3.3.8.</b>
StatefulSet
</a>
</li>
<li class="chapter " data-level="3.3.9" data-path="../concepts/daemonset.html">
<a href="../concepts/daemonset.html">
<b>3.3.9.</b>
DaemonSet
</a>
</li>
<li class="chapter " data-level="3.3.10" data-path="../concepts/serviceaccount.html">
<a href="../concepts/serviceaccount.html">
<b>3.3.10.</b>
ServiceAccount
</a>
</li>
<li class="chapter " data-level="3.3.11" data-path="../concepts/replicaset.html">
<a href="../concepts/replicaset.html">
<b>3.3.11.</b>
ReplicationController和ReplicaSet
</a>
</li>
<li class="chapter " data-level="3.3.12" data-path="../concepts/job.html">
<a href="../concepts/job.html">
<b>3.3.12.</b>
Job
</a>
</li>
<li class="chapter " data-level="3.3.13" data-path="../concepts/cronjob.html">
<a href="../concepts/cronjob.html">
<b>3.3.13.</b>
CronJob
</a>
</li>
<li class="chapter " data-level="3.3.14" data-path="../concepts/ingress.html">
<a href="../concepts/ingress.html">
<b>3.3.14.</b>
Ingress
</a>
<ul class="articles">
<li class="chapter " data-level="3.3.14.1" data-path="../concepts/traefik-ingress-controller.html">
<a href="../concepts/traefik-ingress-controller.html">
<b>3.3.14.1.</b>
Traefik Ingress Controller
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="3.3.15" data-path="../concepts/configmap.html">
<a href="../concepts/configmap.html">
<b>3.3.15.</b>
ConfigMap
</a>
<ul class="articles">
<li class="chapter " data-level="3.3.15.1" data-path="../concepts/configmap-hot-update.html">
<a href="../concepts/configmap-hot-update.html">
<b>3.3.15.1.</b>
ConfigMap的热更新
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="3.3.16" data-path="../concepts/horizontal-pod-autoscaling.html">
<a href="../concepts/horizontal-pod-autoscaling.html">
<b>3.3.16.</b>
Horizontal Pod Autoscaling
</a>
<ul class="articles">
<li class="chapter " data-level="3.3.16.1" data-path="../concepts/custom-metrics-hpa.html">
<a href="../concepts/custom-metrics-hpa.html">
<b>3.3.16.1.</b>
自定义指标HPA
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="3.3.17" data-path="../concepts/label.html">
<a href="../concepts/label.html">
<b>3.3.17.</b>
Label
</a>
</li>
<li class="chapter " data-level="3.3.18" data-path="../concepts/garbage-collection.html">
<a href="../concepts/garbage-collection.html">
<b>3.3.18.</b>
垃圾收集
</a>
</li>
<li class="chapter " data-level="3.3.19" data-path="../concepts/network-policy.html">
<a href="../concepts/network-policy.html">
<b>3.3.19.</b>
NetworkPolicy
</a>
</li>
<li class="chapter " data-level="3.3.20" data-path="../concepts/annotation.html">
<a href="../concepts/annotation.html">
<b>3.3.20.</b>
Annotation
</a>
</li>
<li class="chapter " data-level="3.3.21" data-path="../concepts/aggregated-api-server.html">
<a href="../concepts/aggregated-api-server.html">
<b>3.3.21.</b>
Aggregated API Server
</a>
</li>
<li class="chapter " data-level="3.3.22" data-path="../concepts/custom-resource.html">
<a href="../concepts/custom-resource.html">
<b>3.3.22.</b>
使用自定义资源扩展API
</a>
</li>
<li class="chapter " data-level="3.3.23" data-path="../concepts/apiservice.html">
<a href="../concepts/apiservice.html">
<b>3.3.23.</b>
APIService
</a>
</li>
</ul>
</li>
<li class="header">用户指南</li>
<li class="chapter " data-level="4.1" data-path="../guide/">
<a href="../guide/">
<b>4.1.</b>
用户指南
</a>
</li>
<li class="chapter " data-level="4.2" data-path="../guide/resource-configuration.html">
<a href="../guide/resource-configuration.html">
<b>4.2.</b>
资源对象配置
</a>
<ul class="articles">
<li class="chapter " data-level="4.2.1" data-path="../guide/configure-liveness-readiness-probes.html">
<a href="../guide/configure-liveness-readiness-probes.html">
<b>4.2.1.</b>
配置Pod的liveness和readiness探针
</a>
</li>
<li class="chapter " data-level="4.2.2" data-path="../guide/configure-pod-service-account.html">
<a href="../guide/configure-pod-service-account.html">
<b>4.2.2.</b>
配置Pod的Service Account
</a>
</li>
<li class="chapter " data-level="4.2.3" data-path="../guide/secret-configuration.html">
<a href="../guide/secret-configuration.html">
<b>4.2.3.</b>
Secret配置
</a>
</li>
<li class="chapter " data-level="4.2.4" data-path="../guide/resource-quota-management.html">
<a href="../guide/resource-quota-management.html">
<b>4.2.4.</b>
管理namespace中的资源配额
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="4.3" data-path="../guide/command-usage.html">
<a href="../guide/command-usage.html">
<b>4.3.</b>
命令使用
</a>
<ul class="articles">
<li class="chapter " data-level="4.3.1" data-path="../guide/docker-cli-to-kubectl.html">
<a href="../guide/docker-cli-to-kubectl.html">
<b>4.3.1.</b>
docker用户过度到kubectl命令行指南
</a>
</li>
<li class="chapter " data-level="4.3.2" data-path="../guide/using-kubectl.html">
<a href="../guide/using-kubectl.html">
<b>4.3.2.</b>
kubectl命令概览
</a>
</li>
<li class="chapter " data-level="4.3.3" data-path="../guide/kubectl-cheatsheet.html">
<a href="../guide/kubectl-cheatsheet.html">
<b>4.3.3.</b>
kubectl命令技巧大全
</a>
</li>
<li class="chapter " data-level="4.3.4" data-path="../guide/using-etcdctl-to-access-kubernetes-data.html">
<a href="../guide/using-etcdctl-to-access-kubernetes-data.html">
<b>4.3.4.</b>
使用etcdctl访问kubernetes数据
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="4.4" data-path="../guide/cluster-security-management.html">
<a href="../guide/cluster-security-management.html">
<b>4.4.</b>
集群安全性管理
</a>
<ul class="articles">
<li class="chapter " data-level="4.4.1" data-path="../guide/managing-tls-in-a-cluster.html">
<a href="../guide/managing-tls-in-a-cluster.html">
<b>4.4.1.</b>
管理集群中的TLS
</a>
</li>
<li class="chapter " data-level="4.4.2" data-path="../guide/kubelet-authentication-authorization.html">
<a href="../guide/kubelet-authentication-authorization.html">
<b>4.4.2.</b>
kubelet的认证授权
</a>
</li>
<li class="chapter " data-level="4.4.3" data-path="../guide/tls-bootstrapping.html">
<a href="../guide/tls-bootstrapping.html">
<b>4.4.3.</b>
TLS bootstrap
</a>
</li>
<li class="chapter " data-level="4.4.4" data-path="../guide/kubectl-user-authentication-authorization.html">
<a href="../guide/kubectl-user-authentication-authorization.html">
<b>4.4.4.</b>
创建用户认证授权的kubeconfig文件
</a>
</li>
<li class="chapter " data-level="4.4.5" data-path="../guide/rbac.html">
<a href="../guide/rbac.html">
<b>4.4.5.</b>
RBAC——基于角色的访问控制
</a>
</li>
<li class="chapter " data-level="4.4.6" data-path="../guide/ip-masq-agent.html">
<a href="../guide/ip-masq-agent.html">
<b>4.4.6.</b>
IP伪装代理
</a>
</li>
<li class="chapter " data-level="4.4.7" data-path="../guide/auth-with-kubeconfig-or-token.html">
<a href="../guide/auth-with-kubeconfig-or-token.html">
<b>4.4.7.</b>
使用kubeconfig或token进行用户身份认证
</a>
</li>
<li class="chapter " data-level="4.4.8" data-path="../guide/authentication.html">
<a href="../guide/authentication.html">
<b>4.4.8.</b>
Kubernetes中的用户与身份认证授权
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="4.5" data-path="../guide/access-kubernetes-cluster.html">
<a href="../guide/access-kubernetes-cluster.html">
<b>4.5.</b>
访问Kubernetes集群
</a>
<ul class="articles">
<li class="chapter " data-level="4.5.1" data-path="../guide/access-cluster.html">
<a href="../guide/access-cluster.html">
<b>4.5.1.</b>
访问集群
</a>
</li>
<li class="chapter " data-level="4.5.2" data-path="../guide/authenticate-across-clusters-kubeconfig.html">
<a href="../guide/authenticate-across-clusters-kubeconfig.html">
<b>4.5.2.</b>
使用kubeconfig文件配置跨集群认证
</a>
</li>
<li class="chapter " data-level="4.5.3" data-path="../guide/connecting-to-applications-port-forward.html">
<a href="../guide/connecting-to-applications-port-forward.html">
<b>4.5.3.</b>
通过端口转发访问集群中的应用程序
</a>
</li>
<li class="chapter " data-level="4.5.4" data-path="../guide/service-access-application-cluster.html">
<a href="../guide/service-access-application-cluster.html">
<b>4.5.4.</b>
使用service访问群集中的应用程序
</a>
</li>
<li class="chapter " data-level="4.5.5" data-path="../guide/accessing-kubernetes-pods-from-outside-of-the-cluster.html">
<a href="../guide/accessing-kubernetes-pods-from-outside-of-the-cluster.html">
<b>4.5.5.</b>
从外部访问Kubernetes中的Pod
</a>
</li>
<li class="chapter " data-level="4.5.6" data-path="../guide/carbin-mobile-dashboard-for-kubernetes.html">
<a href="../guide/carbin-mobile-dashboard-for-kubernetes.html">
<b>4.5.6.</b>
Carbin - Kubernetes手机客户端
</a>
</li>
<li class="chapter " data-level="4.5.7" data-path="../guide/kubernetes-desktop-client.html">
<a href="../guide/kubernetes-desktop-client.html">
<b>4.5.7.</b>
Kubernetic - Kubernetes桌面客户端
</a>
</li>
<li class="chapter " data-level="4.5.8" data-path="../guide/kubernator-kubernetes-ui.html">
<a href="../guide/kubernator-kubernetes-ui.html">
<b>4.5.8.</b>
Kubernator - 更底层的Kubernetes UI
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="4.6" data-path="../guide/application-development-deployment-flow.html">
<a href="../guide/application-development-deployment-flow.html">
<b>4.6.</b>
在Kubernetes中开发部署应用
</a>
<ul class="articles">
<li class="chapter " data-level="4.6.1" data-path="../guide/deploy-applications-in-kubernetes.html">
<a href="../guide/deploy-applications-in-kubernetes.html">
<b>4.6.1.</b>
适用于kubernetes的应用开发部署流程
</a>
</li>
<li class="chapter " data-level="4.6.2" data-path="../guide/migrating-hadoop-yarn-to-kubernetes.html">
<a href="../guide/migrating-hadoop-yarn-to-kubernetes.html">
<b>4.6.2.</b>
迁移传统应用到Kubernetes中——以Hadoop YARN为例
</a>
</li>
<li class="chapter " data-level="4.6.3" data-path="../guide/using-statefulset.html">
<a href="../guide/using-statefulset.html">
<b>4.6.3.</b>
使用StatefulSet部署用状态应用
</a>
</li>
</ul>
</li>
<li class="header">最佳实践</li>
<li class="chapter " data-level="5.1" data-path="../practice/">
<a href="../practice/">
<b>5.1.</b>
最佳实践概览
</a>
</li>
<li class="chapter " data-level="5.2" data-path="../practice/install-kubernetes-on-centos.html">
<a href="../practice/install-kubernetes-on-centos.html">
<b>5.2.</b>
在CentOS上部署Kubernetes集群
</a>
<ul class="articles">
<li class="chapter " data-level="5.2.1" data-path="../practice/create-tls-and-secret-key.html">
<a href="../practice/create-tls-and-secret-key.html">
<b>5.2.1.</b>
创建TLS证书和秘钥
</a>
</li>
<li class="chapter " data-level="5.2.2" data-path="../practice/create-kubeconfig.html">
<a href="../practice/create-kubeconfig.html">
<b>5.2.2.</b>
创建kubeconfig文件
</a>
</li>
<li class="chapter " data-level="5.2.3" data-path="../practice/etcd-cluster-installation.html">
<a href="../practice/etcd-cluster-installation.html">
<b>5.2.3.</b>
创建高可用etcd集群
</a>
</li>
<li class="chapter " data-level="5.2.4" data-path="../practice/kubectl-installation.html">
<a href="../practice/kubectl-installation.html">
<b>5.2.4.</b>
安装kubectl命令行工具
</a>
</li>
<li class="chapter " data-level="5.2.5" data-path="../practice/master-installation.html">
<a href="../practice/master-installation.html">
<b>5.2.5.</b>
部署master节点
</a>
</li>
<li class="chapter " data-level="5.2.6" data-path="../practice/flannel-installation.html">
<a href="../practice/flannel-installation.html">
<b>5.2.6.</b>
安装flannel网络插件
</a>
</li>
<li class="chapter " data-level="5.2.7" data-path="../practice/node-installation.html">
<a href="../practice/node-installation.html">
<b>5.2.7.</b>
部署node节点
</a>
</li>
<li class="chapter " data-level="5.2.8" data-path="../practice/kubedns-addon-installation.html">
<a href="../practice/kubedns-addon-installation.html">
<b>5.2.8.</b>
安装kubedns插件
</a>
</li>
<li class="chapter " data-level="5.2.9" data-path="../practice/dashboard-addon-installation.html">
<a href="../practice/dashboard-addon-installation.html">
<b>5.2.9.</b>
安装dashboard插件
</a>
</li>
<li class="chapter " data-level="5.2.10" data-path="../practice/heapster-addon-installation.html">
<a href="../practice/heapster-addon-installation.html">
<b>5.2.10.</b>
安装heapster插件
</a>
</li>
<li class="chapter " data-level="5.2.11" data-path="../practice/efk-addon-installation.html">
<a href="../practice/efk-addon-installation.html">
<b>5.2.11.</b>
安装EFK插件
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="5.3" data-path="../practice/service-discovery-and-loadbalancing.html">
<a href="../practice/service-discovery-and-loadbalancing.html">
<b>5.3.</b>
服务发现与负载均衡
</a>
<ul class="articles">
<li class="chapter " data-level="5.3.1" data-path="../practice/traefik-ingress-installation.html">
<a href="../practice/traefik-ingress-installation.html">
<b>5.3.1.</b>
安装Traefik ingress
</a>
</li>
<li class="chapter " data-level="5.3.2" data-path="../practice/distributed-load-test.html">
<a href="../practice/distributed-load-test.html">
<b>5.3.2.</b>
分布式负载测试
</a>
</li>
<li class="chapter " data-level="5.3.3" data-path="../practice/network-and-cluster-perfermance-test.html">
<a href="../practice/network-and-cluster-perfermance-test.html">
<b>5.3.3.</b>
网络和集群性能测试
</a>
</li>
<li class="chapter " data-level="5.3.4" data-path="../practice/edge-node-configuration.html">
<a href="../practice/edge-node-configuration.html">
<b>5.3.4.</b>
边缘节点配置
</a>
</li>
<li class="chapter " data-level="5.3.5" data-path="../practice/nginx-ingress-installation.html">
<a href="../practice/nginx-ingress-installation.html">
<b>5.3.5.</b>
安装Nginx ingress
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="5.4" data-path="../practice/operation.html">
<a href="../practice/operation.html">
<b>5.4.</b>
运维管理
</a>
<ul class="articles">
<li class="chapter " data-level="5.4.1" data-path="../practice/service-rolling-update.html">
<a href="../practice/service-rolling-update.html">
<b>5.4.1.</b>
服务滚动升级
</a>
</li>
<li class="chapter " data-level="5.4.2" data-path="../practice/app-log-collection.html">
<a href="../practice/app-log-collection.html">
<b>5.4.2.</b>
应用日志收集
</a>
</li>
<li class="chapter " data-level="5.4.3" data-path="../practice/configuration-best-practice.html">
<a href="../practice/configuration-best-practice.html">
<b>5.4.3.</b>
配置最佳实践
</a>
</li>
<li class="chapter " data-level="5.4.4" data-path="../practice/monitor.html">
<a href="../practice/monitor.html">
<b>5.4.4.</b>
集群及应用监控
</a>
</li>
<li class="chapter " data-level="5.4.5" data-path="../practice/data-persistence-problem.html">
<a href="../practice/data-persistence-problem.html">
<b>5.4.5.</b>
数据持久化问题
</a>
</li>
<li class="chapter " data-level="5.4.6" data-path="../practice/manage-compute-resources-container.html">
<a href="../practice/manage-compute-resources-container.html">
<b>5.4.6.</b>
管理容器的计算资源
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="5.5" data-path="../practice/storage.html">
<a href="../practice/storage.html">
<b>5.5.</b>
存储管理
</a>
<ul class="articles">
<li class="chapter " data-level="5.5.1" data-path="../practice/glusterfs.html">
<a href="../practice/glusterfs.html">
<b>5.5.1.</b>
GlusterFS
</a>
<ul class="articles">
<li class="chapter " data-level="5.5.1.1" data-path="../practice/using-glusterfs-for-persistent-storage.html">
<a href="../practice/using-glusterfs-for-persistent-storage.html">
<b>5.5.1.1.</b>
使用GlusterFS做持久化存储
</a>
</li>
<li class="chapter " data-level="5.5.1.2" data-path="../practice/storage-for-containers-using-glusterfs-with-openshift.html">
<a href="../practice/storage-for-containers-using-glusterfs-with-openshift.html">
<b>5.5.1.2.</b>
在OpenShift中使用GlusterFS做持久化存储
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="5.5.2" data-path="../practice/cephfs.html">
<a href="../practice/cephfs.html">
<b>5.5.2.</b>
CephFS
</a>
<ul class="articles">
<li class="chapter " data-level="5.5.2.1" data-path="../practice/using-ceph-for-persistent-storage.html">
<a href="../practice/using-ceph-for-persistent-storage.html">
<b>5.5.2.1.</b>
使用Ceph做持久化存储
</a>
</li>
</ul>
</li>
</ul>
</li>
<li class="chapter " data-level="5.6" data-path="../practice/monitoring.html">
<a href="../practice/monitoring.html">
<b>5.6.</b>
集群与应用监控
</a>
<ul class="articles">
<li class="chapter " data-level="5.6.1" data-path="../practice/heapster.html">
<a href="../practice/heapster.html">
<b>5.6.1.</b>
Heapster
</a>
<ul class="articles">
<li class="chapter " data-level="5.6.1.1" data-path="../practice/using-heapster-to-get-object-metrics.html">
<a href="../practice/using-heapster-to-get-object-metrics.html">
<b>5.6.1.1.</b>
使用Heapster获取集群和对象的metric数据
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="5.6.2" data-path="../practice/prometheus.html">
<a href="../practice/prometheus.html">
<b>5.6.2.</b>
Prometheus
</a>
<ul class="articles">
<li class="chapter " data-level="5.6.2.1" data-path="../practice/using-prometheus-to-monitor-kuberentes-cluster.html">
<a href="../practice/using-prometheus-to-monitor-kuberentes-cluster.html">
<b>5.6.2.1.</b>
使用Prometheus监控kubernetes集群
</a>
</li>
</ul>
</li>
</ul>
</li>
<li class="chapter " data-level="5.7" data-path="../practice/services-management-tool.html">
<a href="../practice/services-management-tool.html">
<b>5.7.</b>
服务编排管理
</a>
<ul class="articles">
<li class="chapter " data-level="5.7.1" data-path="../practice/helm.html">
<a href="../practice/helm.html">
<b>5.7.1.</b>
使用Helm管理kubernetes应用
</a>
</li>
<li class="chapter " data-level="5.7.2" data-path="../practice/create-private-charts-repo.html">
<a href="../practice/create-private-charts-repo.html">
<b>5.7.2.</b>
构建私有Chart仓库
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="5.8" data-path="../practice/ci-cd.html">
<a href="../practice/ci-cd.html">
<b>5.8.</b>
持续集成与发布
</a>
<ul class="articles">
<li class="chapter " data-level="5.8.1" data-path="../practice/jenkins-ci-cd.html">
<a href="../practice/jenkins-ci-cd.html">
<b>5.8.1.</b>
使用Jenkins进行持续集成与发布
</a>
</li>
<li class="chapter " data-level="5.8.2" data-path="../practice/drone-ci-cd.html">
<a href="../practice/drone-ci-cd.html">
<b>5.8.2.</b>
使用Drone进行持续集成与发布
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="5.9" data-path="../practice/update-and-upgrade.html">
<a href="../practice/update-and-upgrade.html">
<b>5.9.</b>
更新与升级
</a>
<ul class="articles">
<li class="chapter " data-level="5.9.1" data-path="../practice/manually-upgrade.html">
<a href="../practice/manually-upgrade.html">
<b>5.9.1.</b>
手动升级Kubernetes集群
</a>
</li>
<li class="chapter " data-level="5.9.2" data-path="../practice/dashboard-upgrade.html">
<a href="../practice/dashboard-upgrade.html">
<b>5.9.2.</b>
升级dashboard
</a>
</li>
</ul>
</li>
<li class="header">领域应用</li>
<li class="chapter " data-level="6.1" data-path="./">
<a href="./">
<b>6.1.</b>
领域应用概览
</a>
</li>
<li class="chapter " data-level="6.2" data-path="microservices.html">
<a href="microservices.html">
<b>6.2.</b>
微服务架构
</a>
<ul class="articles">
<li class="chapter " data-level="6.2.1" data-path="service-discovery-in-microservices.html">
<a href="service-discovery-in-microservices.html">
<b>6.2.1.</b>
微服务中的服务发现
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="6.3" data-path="service-mesh.html">
<a href="service-mesh.html">
<b>6.3.</b>
Service Mesh 服务网格
</a>
<ul class="articles">
<li class="chapter " data-level="6.3.1" data-path="istio.html">
<a href="istio.html">
<b>6.3.1.</b>
Istio
</a>
<ul class="articles">
<li class="chapter " data-level="6.3.1.1" data-path="istio-installation.html">
<a href="istio-installation.html">
<b>6.3.1.1.</b>
安装并试用Istio service mesh
</a>
</li>
<li class="chapter " data-level="6.3.1.2" data-path="configuring-request-routing.html">
<a href="configuring-request-routing.html">
<b>6.3.1.2.</b>
配置请求的路由规则
</a>
</li>
<li class="chapter " data-level="6.3.1.3" data-path="install-and-expand-istio-mesh.html">
<a href="install-and-expand-istio-mesh.html">
<b>6.3.1.3.</b>
安装和拓展Istio service mesh
</a>
</li>
<li class="chapter " data-level="6.3.1.4" data-path="integrating-vms.html">
<a href="integrating-vms.html">
<b>6.3.1.4.</b>
集成虚拟机
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="6.3.2" data-path="linkerd.html">
<a href="linkerd.html">
<b>6.3.2.</b>
Linkerd
</a>
<ul class="articles">
<li class="chapter " data-level="6.3.2.1" data-path="linkerd-user-guide.html">
<a href="linkerd-user-guide.html">
<b>6.3.2.1.</b>
Linkerd 使用指南
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="6.3.3" data-path="conduit.html">
<a href="conduit.html">
<b>6.3.3.</b>
Conduit
</a>
<ul class="articles">
<li class="chapter " data-level="6.3.3.1" data-path="conduit-overview.html">
<a href="conduit-overview.html">
<b>6.3.3.1.</b>
Condiut概览
</a>
</li>
<li class="chapter " data-level="6.3.3.2" data-path="conduit-installation.html">
<a href="conduit-installation.html">
<b>6.3.3.2.</b>
安装Conduit
</a>
</li>
</ul>
</li>
</ul>
</li>
<li class="chapter " data-level="6.4" data-path="big-data.html">
<a href="big-data.html">
<b>6.4.</b>
大数据
</a>
<ul class="articles">
<li class="chapter " data-level="6.4.1" data-path="spark-standalone-on-kubernetes.html">
<a href="spark-standalone-on-kubernetes.html">
<b>6.4.1.</b>
Spark standalone on Kubernetes
</a>
</li>
<li class="chapter active" data-level="6.4.2" data-path="running-spark-with-kubernetes-native-scheduler.html">
<a href="running-spark-with-kubernetes-native-scheduler.html">
<b>6.4.2.</b>
运行支持Kubernetes原生调度的Spark程序
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="6.5" data-path="serverless.html">
<a href="serverless.html">
<b>6.5.</b>
Serverless架构
</a>
<ul class="articles">
<li class="chapter " data-level="6.5.1" data-path="understanding-serverless.html">
<a href="understanding-serverless.html">
<b>6.5.1.</b>
理解Serverless
</a>
</li>
<li class="chapter " data-level="6.5.2" data-path="faas.html">
<a href="faas.html">
<b>6.5.2.</b>
FaaS-函数即服务
</a>
<ul class="articles">
<li class="chapter " data-level="6.5.2.1" data-path="openfaas-quick-start.html">
<a href="openfaas-quick-start.html">
<b>6.5.2.1.</b>
OpenFaaS快速入门指南
</a>
</li>
</ul>
</li>
</ul>
</li>
<li class="chapter " data-level="6.6" data-path="edge-computing.html">
<a href="edge-computing.html">
<b>6.6.</b>
边缘计算
</a>
</li>
<li class="header">开发指南</li>
<li class="chapter " data-level="7.1" data-path="../develop/">
<a href="../develop/">
<b>7.1.</b>
开发指南概览
</a>
</li>
<li class="chapter " data-level="7.2" data-path="../develop/sigs-and-working-group.html">
<a href="../develop/sigs-and-working-group.html">
<b>7.2.</b>
SIG和工作组
</a>
</li>
<li class="chapter " data-level="7.3" data-path="../develop/developing-environment.html">
<a href="../develop/developing-environment.html">
<b>7.3.</b>
开发环境搭建
</a>
</li>
<li class="chapter " data-level="7.4" data-path="../develop/testing.html">
<a href="../develop/testing.html">
<b>7.4.</b>
单元测试和集成测试
</a>
</li>
<li class="chapter " data-level="7.5" data-path="../develop/client-go-sample.html">
<a href="../develop/client-go-sample.html">
<b>7.5.</b>
client-go示例
</a>
</li>
<li class="chapter " data-level="7.6" data-path="../develop/contribute.html">
<a href="../develop/contribute.html">
<b>7.6.</b>
社区贡献
</a>
</li>
<li class="chapter " data-level="7.7" data-path="../develop/minikube.html">
<a href="../develop/minikube.html">
<b>7.7.</b>
Minikube
</a>
</li>
<li class="header">附录</li>
<li class="chapter " data-level="8.1" data-path="../appendix/">
<a href="../appendix/">
<b>8.1.</b>
附录说明
</a>
</li>
<li class="chapter " data-level="8.2" data-path="../appendix/debug-kubernetes-services.html">
<a href="../appendix/debug-kubernetes-services.html">
<b>8.2.</b>
Kubernetes中的应用故障排查
</a>
</li>
<li class="chapter " data-level="8.3" data-path="../appendix/material-share.html">
<a href="../appendix/material-share.html">
<b>8.3.</b>
Kubernetes相关资讯和情报链接
</a>
</li>
<li class="chapter " data-level="8.4" data-path="../appendix/docker-best-practice.html">
<a href="../appendix/docker-best-practice.html">
<b>8.4.</b>
Docker最佳实践
</a>
</li>
<li class="chapter " data-level="8.5" data-path="../appendix/tricks.html">
<a href="../appendix/tricks.html">
<b>8.5.</b>
使用技巧
</a>
</li>
<li class="chapter " data-level="8.6" data-path="../appendix/issues.html">
<a href="../appendix/issues.html">
<b>8.6.</b>
问题记录
</a>
</li>
<li class="chapter " data-level="8.7" data-path="../appendix/kubernetes-changelog.html">
<a href="../appendix/kubernetes-changelog.html">
<b>8.7.</b>
Kubernetes版本更新日志
</a>
<ul class="articles">
<li class="chapter " data-level="8.7.1" data-path="../appendix/kubernetes-1.7-changelog.html">
<a href="../appendix/kubernetes-1.7-changelog.html">
<b>8.7.1.</b>
Kubernetes1.7更新日志
</a>
</li>
<li class="chapter " data-level="8.7.2" data-path="../appendix/kubernetes-1.8-changelog.html">
<a href="../appendix/kubernetes-1.8-changelog.html">
<b>8.7.2.</b>
Kubernetes1.8更新日志
</a>
</li>
<li class="chapter " data-level="8.7.3" data-path="../appendix/kubernetes-1.9-changelog.html">
<a href="../appendix/kubernetes-1.9-changelog.html">
<b>8.7.3.</b>
Kubernetes1.9更新日志
</a>
</li>
</ul>
</li>
<li class="chapter " data-level="8.8" data-path="../appendix/summary-and-outlook.html">
<a href="../appendix/summary-and-outlook.html">
<b>8.8.</b>
Kubernetes及云原生年度总结及展望
</a>
<ul class="articles">
<li class="chapter " data-level="8.8.1" data-path="../appendix/kubernetes-and-cloud-native-summary-in-2017-and-outlook-for-2018.html">
<a href="../appendix/kubernetes-and-cloud-native-summary-in-2017-and-outlook-for-2018.html">
<b>8.8.1.</b>
Kubernetes与云原生2017年年终总结及2018年展望
</a>
</li>
</ul>
</li>
<li class="divider"></li>
<li>
<a href="https://www.gitbook.com" target="blank" class="gitbook-link">
本书使用 GitBook 发布
</a>
</li>
</ul>
</nav>
</div>
<div class="book-body">
<div class="body-inner">
<div class="book-header" role="navigation">
<!-- Title -->
<h1>
<i class="fa fa-circle-o-notch fa-spin"></i>
<a href=".." >运行支持Kubernetes原生调度的Spark程序</a>
</h1>
</div>
<div class="page-wrapper" tabindex="-1" role="main">
<div class="page-inner">
<div class="search-plus" id="book-search-results">
<div class="search-noresults">
<section class="normal markdown-section">
<h1 id="&#x8FD0;&#x884C;&#x652F;&#x6301;kubernetes&#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684;spark&#x7A0B;&#x5E8F;">&#x8FD0;&#x884C;&#x652F;&#x6301;kubernetes&#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684;Spark&#x7A0B;&#x5E8F;</h1>
<p>TL;DR &#x8FD9;&#x4E2A;&#x4E3B;&#x9898;&#x6BD4;&#x8F83;&#x5927;&#xFF0C;&#x8BE5;&#x5F00;&#x6E90;&#x9879;&#x76EE;&#x4E5F;&#x8FD8;&#x5728;&#x4E0D;&#x65AD;&#x8FDB;&#x884C;&#x4E2D;&#xFF0C;&#x6211;&#x5355;&#x72EC;&#x505A;&#x4E86;&#x4E00;&#x4E2A; web &#x7528;&#x6765;&#x8BB0;&#x5F55; spark on kubernetes &#x7684;&#x7814;&#x7A76;&#x548C;&#x6700;&#x65B0;&#x8FDB;&#x5C55;&#x89C1;: <a href="https://jimmysong.io/spark-on-k8s" target="_blank">https://jimmysong.io/spark-on-k8s</a></p>
<p>&#x6211;&#x4EEC;&#x4E4B;&#x524D;&#x5C31;&#x5728; kubernetes &#x4E2D;&#x8FD0;&#x884C;&#x8FC7; standalone &#x65B9;&#x5F0F;&#x7684; spark &#x96C6;&#x7FA4;&#xFF0C;&#x89C1; <a href="spark-standalone-on-kubernetes.html">Spark standalone on kubernetes</a>&#x3002;</p>
<p>&#x76EE;&#x524D;&#x8FD0;&#x884C;&#x652F;&#x6301; kubernetes &#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684; spark &#x7A0B;&#x5E8F;&#x7531; Google &#x4E3B;&#x5BFC;&#xFF0C;&#x76EE;&#x524D;&#x8FD0;&#x884C;&#x652F;&#x6301; kubernetes &#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684; spark &#x7A0B;&#x5E8F;&#x7531; Google &#x4E3B;&#x5BFC;&#xFF0C;fork &#x81EA; spark &#x7684;&#x5B98;&#x65B9;&#x4EE3;&#x7801;&#x5E93;&#xFF0C;&#x89C1;<a href="https://github.com/apache-spark-on-k8s/spark/" target="_blank">https://github.com/apache-spark-on-k8s/spark/</a> &#xFF0C;&#x5C5E;&#x4E8E;Big Data SIG&#x3002;</p>
<p>&#x53C2;&#x4E0E;&#x5230;&#x8BE5;&#x9879;&#x76EE;&#x7684;&#x516C;&#x53F8;&#x6709;&#xFF1A;</p>
<ul>
<li>Bloomberg</li>
<li>Google</li>
<li>Haiwen</li>
<li>Hyperpilot</li>
<li>Intel</li>
<li>Palantir</li>
<li>Pepperdata</li>
<li>Red Hat</li>
</ul>
<h2 id="&#x4E3A;&#x4F55;&#x4F7F;&#x7528;-spark-on-kubernetes">&#x4E3A;&#x4F55;&#x4F7F;&#x7528; spark on kubernetes</h2>
<p>&#x4F7F;&#x7528;kubernetes&#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684;spark on kubernetes&#x662F;&#x5BF9;&#x73B0;&#x6709;&#x7684;spark on yarn/mesos&#x7684;&#x8D44;&#x6E90;&#x4F7F;&#x7528;&#x65B9;&#x5F0F;&#x7684;&#x9769;&#x547D;&#x6027;&#x7684;&#x6539;&#x8FDB;&#xFF0C;&#x4E3B;&#x8981;&#x8868;&#x73B0;&#x5728;&#x4EE5;&#x4E0B;&#x51E0;&#x70B9;&#xFF1A;</p>
<ol>
<li>Kubernetes&#x539F;&#x751F;&#x8C03;&#x5EA6;&#xFF1A;&#x4E0D;&#x518D;&#x9700;&#x8981;&#x4E8C;&#x5C42;&#x8C03;&#x5EA6;&#xFF0C;&#x76F4;&#x63A5;&#x4F7F;&#x7528;kubernetes&#x7684;&#x8D44;&#x6E90;&#x8C03;&#x5EA6;&#x529F;&#x80FD;&#xFF0C;&#x8DDF;&#x5176;&#x4ED6;&#x5E94;&#x7528;&#x5171;&#x7528;&#x6574;&#x4E2A;kubernetes&#x7BA1;&#x7406;&#x7684;&#x8D44;&#x6E90;&#x6C60;&#xFF1B;</li>
<li>&#x8D44;&#x6E90;&#x9694;&#x79BB;&#xFF0C;&#x7C92;&#x5EA6;&#x66F4;&#x7EC6;&#xFF1A;&#x539F;&#x5148;yarn&#x4E2D;&#x7684;queue&#x5728;spark on kubernetes&#x4E2D;&#x5DF2;&#x4E0D;&#x5B58;&#x5728;&#xFF0C;&#x53D6;&#x800C;&#x4EE3;&#x4E4B;&#x7684;&#x662F;kubernetes&#x4E2D;&#x539F;&#x751F;&#x7684;namespace&#xFF0C;&#x53EF;&#x4EE5;&#x4E3A;&#x6BCF;&#x4E2A;&#x7528;&#x6237;&#x5206;&#x522B;&#x6307;&#x5B9A;&#x4E00;&#x4E2A;namespace&#xFF0C;&#x9650;&#x5236;&#x7528;&#x6237;&#x7684;&#x8D44;&#x6E90;quota&#xFF1B;</li>
<li>&#x7EC6;&#x7C92;&#x5EA6;&#x7684;&#x8D44;&#x6E90;&#x5206;&#x914D;&#xFF1A;&#x53EF;&#x4EE5;&#x7ED9;&#x6BCF;&#x4E2A;spark&#x4EFB;&#x52A1;&#x6307;&#x5B9A;&#x8D44;&#x6E90;&#x9650;&#x5236;&#xFF0C;&#x5B9E;&#x9645;&#x6307;&#x5B9A;&#x591A;&#x5C11;&#x8D44;&#x6E90;&#x5C31;&#x4F7F;&#x7528;&#x591A;&#x5C11;&#x8D44;&#x6E90;&#xFF0C;&#x56E0;&#x4E3A;&#x6CA1;&#x6709;&#x4E86;&#x50CF;yarn&#x90A3;&#x6837;&#x7684;&#x4E8C;&#x5C42;&#x8C03;&#x5EA6;&#xFF08;&#x5708;&#x5730;&#x5F0F;&#x7684;&#xFF09;&#xFF0C;&#x6240;&#x4EE5;&#x53EF;&#x4EE5;&#x66F4;&#x9AD8;&#x6548;&#x548C;&#x7EC6;&#x7C92;&#x5EA6;&#x7684;&#x4F7F;&#x7528;&#x8D44;&#x6E90;&#xFF1B;</li>
<li>&#x76D1;&#x63A7;&#x7684;&#x53D8;&#x9769;&#xFF1A;&#x56E0;&#x4E3A;&#x505A;&#x5230;&#x4E86;&#x7EC6;&#x7C92;&#x5EA6;&#x7684;&#x8D44;&#x6E90;&#x5206;&#x914D;&#xFF0C;&#x6240;&#x4EE5;&#x53EF;&#x4EE5;&#x5BF9;&#x7528;&#x6237;&#x63D0;&#x4EA4;&#x7684;&#x6BCF;&#x4E00;&#x4E2A;&#x4EFB;&#x52A1;&#x505A;&#x5230;&#x8D44;&#x6E90;&#x4F7F;&#x7528;&#x7684;&#x76D1;&#x63A7;&#xFF0C;&#x4ECE;&#x800C;&#x5224;&#x65AD;&#x7528;&#x6237;&#x7684;&#x8D44;&#x6E90;&#x4F7F;&#x7528;&#x60C5;&#x51B5;&#xFF0C;&#x6240;&#x6709;&#x7684;metric&#x90FD;&#x8BB0;&#x5F55;&#x5728;&#x6570;&#x636E;&#x5E93;&#x4E2D;&#xFF0C;&#x751A;&#x81F3;&#x53EF;&#x4EE5;&#x4E3A;&#x6BCF;&#x4E2A;&#x7528;&#x6237;&#x7684;&#x6BCF;&#x6B21;&#x4EFB;&#x52A1;&#x63D0;&#x4EA4;&#x8BA1;&#x91CF;&#xFF1B;</li>
<li>&#x65E5;&#x5FD7;&#x7684;&#x53D8;&#x9769;&#xFF1A;&#x7528;&#x6237;&#x4E0D;&#x518D;&#x901A;&#x8FC7;yarn&#x7684;web&#x9875;&#x9762;&#x6765;&#x67E5;&#x770B;&#x4EFB;&#x52A1;&#x72B6;&#x6001;&#xFF0C;&#x800C;&#x662F;&#x901A;&#x8FC7;pod&#x7684;log&#x6765;&#x67E5;&#x770B;&#xFF0C;&#x53EF;&#x5C06;&#x6240;&#x6709;&#x7684;kuberentes&#x4E2D;&#x7684;&#x5E94;&#x7528;&#x7684;&#x65E5;&#x5FD7;&#x7B49;&#x540C;&#x770B;&#x5F85;&#x6536;&#x96C6;&#x8D77;&#x6765;&#xFF0C;&#x7136;&#x540E;&#x53EF;&#x4EE5;&#x6839;&#x636E;&#x6807;&#x7B7E;&#x67E5;&#x770B;&#x5BF9;&#x5E94;&#x5E94;&#x7528;&#x7684;&#x65E5;&#x5FD7;&#xFF1B;</li>
</ol>
<p>&#x6240;&#x6709;&#x8FD9;&#x4E9B;&#x53D8;&#x9769;&#x90FD;&#x53EF;&#x4EE5;&#x8BA9;&#x6211;&#x4EEC;&#x66F4;&#x9AD8;&#x6548;&#x7684;&#x83B7;&#x53D6;&#x8D44;&#x6E90;&#x3001;&#x66F4;&#x6709;&#x6548;&#x7387;&#x7684;&#x83B7;&#x53D6;&#x8D44;&#x6E90;&#xFF01;</p>
<h2 id="spark-&#x6982;&#x5FF5;&#x8BF4;&#x660E;">Spark &#x6982;&#x5FF5;&#x8BF4;&#x660E;</h2>
<p><a href="http://spark.apache.org" target="_blank">Apache Spark</a> &#x662F;&#x4E00;&#x4E2A;&#x56F4;&#x7ED5;&#x901F;&#x5EA6;&#x3001;&#x6613;&#x7528;&#x6027;&#x548C;&#x590D;&#x6742;&#x5206;&#x6790;&#x6784;&#x5EFA;&#x7684;&#x5927;&#x6570;&#x636E;&#x5904;&#x7406;&#x6846;&#x67B6;&#x3002;&#x6700;&#x521D;&#x5728;2009&#x5E74;&#x7531;&#x52A0;&#x5DDE;&#x5927;&#x5B66;&#x4F2F;&#x514B;&#x5229;&#x5206;&#x6821;&#x7684;AMPLab&#x5F00;&#x53D1;&#xFF0C;&#x5E76;&#x4E8E;2010&#x5E74;&#x6210;&#x4E3A;Apache&#x7684;&#x5F00;&#x6E90;&#x9879;&#x76EE;&#x4E4B;&#x4E00;&#x3002;</p>
<p>&#x5728; Spark &#x4E2D;&#x5305;&#x62EC;&#x5982;&#x4E0B;&#x7EC4;&#x4EF6;&#x6216;&#x6982;&#x5FF5;&#xFF1A;</p>
<ul>
<li><strong>Application</strong>&#xFF1A;Spark Application &#x7684;&#x6982;&#x5FF5;&#x548C; Hadoop &#x4E2D;&#x7684; MapReduce &#x7C7B;&#x4F3C;&#xFF0C;&#x6307;&#x7684;&#x662F;&#x7528;&#x6237;&#x7F16;&#x5199;&#x7684; Spark &#x5E94;&#x7528;&#x7A0B;&#x5E8F;&#xFF0C;&#x5305;&#x542B;&#x4E86;&#x4E00;&#x4E2A; Driver &#x529F;&#x80FD;&#x7684;&#x4EE3;&#x7801;&#x548C;&#x5206;&#x5E03;&#x5728;&#x96C6;&#x7FA4;&#x4E2D;&#x591A;&#x4E2A;&#x8282;&#x70B9;&#x4E0A;&#x8FD0;&#x884C;&#x7684; Executor &#x4EE3;&#x7801;&#xFF1B;</li>
<li><strong>Driver</strong>&#xFF1A;Spark &#x4E2D;&#x7684; Driver &#x5373;&#x8FD0;&#x884C;&#x4E0A;&#x8FF0; Application &#x7684; main() &#x51FD;&#x6570;&#x5E76;&#x4E14;&#x521B;&#x5EFA; SparkContext&#xFF0C;&#x5176;&#x4E2D;&#x521B;&#x5EFA; SparkContext &#x7684;&#x76EE;&#x7684;&#x662F;&#x4E3A;&#x4E86;&#x51C6;&#x5907;Spark&#x5E94;&#x7528;&#x7A0B;&#x5E8F;&#x7684;&#x8FD0;&#x884C;&#x73AF;&#x5883;&#x3002;&#x5728; Spark &#x4E2D;&#x7531; SparkContext &#x8D1F;&#x8D23;&#x548C; ClusterManager &#x901A;&#x4FE1;&#xFF0C;&#x8FDB;&#x884C;&#x8D44;&#x6E90;&#x7684;&#x7533;&#x8BF7;&#x3001;&#x4EFB;&#x52A1;&#x7684;&#x5206;&#x914D;&#x548C;&#x76D1;&#x63A7;&#x7B49;&#xFF1B;&#x5F53; Executor &#x90E8;&#x5206;&#x8FD0;&#x884C;&#x5B8C;&#x6BD5;&#x540E;&#xFF0C;Driver&#x8D1F;&#x8D23;&#x5C06;SparkContext &#x5173;&#x95ED;&#x3002;&#x901A;&#x5E38;&#x7528; SparkContext &#x4EE3;&#x8868; Driver&#xFF1B;</li>
<li><strong>Executor</strong>&#xFF1A;Application&#x8FD0;&#x884C;&#x5728;Worker &#x8282;&#x70B9;&#x4E0A;&#x7684;&#x4E00;&#x4E2A;&#x8FDB;&#x7A0B;&#xFF0C;&#x8BE5;&#x8FDB;&#x7A0B;&#x8D1F;&#x8D23;&#x8FD0;&#x884C;Task&#xFF0C;&#x5E76;&#x4E14;&#x8D1F;&#x8D23;&#x5C06;&#x6570;&#x636E;&#x5B58;&#x5728;&#x5185;&#x5B58;&#x6216;&#x8005;&#x78C1;&#x76D8;&#x4E0A;&#xFF0C;&#x6BCF;&#x4E2A;Application&#x90FD;&#x6709;&#x5404;&#x81EA;&#x72EC;&#x7ACB;&#x7684;&#x4E00;&#x6279;Executor&#x3002;&#x5728;Spark on Yarn&#x6A21;&#x5F0F;&#x4E0B;&#xFF0C;&#x5176;&#x8FDB;&#x7A0B;&#x540D;&#x79F0;&#x4E3A;<code>CoarseGrainedExecutorBackend</code>&#xFF0C;&#x7C7B;&#x4F3C;&#x4E8E; Hadoop MapReduce &#x4E2D;&#x7684; YarnChild&#x3002;&#x4E00;&#x4E2A; <code>CoarseGrainedExecutorBackend</code> &#x8FDB;&#x7A0B;&#x6709;&#x4E14;&#x4EC5;&#x6709;&#x4E00;&#x4E2A; executor &#x5BF9;&#x8C61;&#xFF0C;&#x5B83;&#x8D1F;&#x8D23;&#x5C06; Task &#x5305;&#x88C5;&#x6210; taskRunner&#xFF0C;&#x5E76;&#x4ECE;&#x7EBF;&#x7A0B;&#x6C60;&#x4E2D;&#x62BD;&#x53D6;&#x51FA;&#x4E00;&#x4E2A;&#x7A7A;&#x95F2;&#x7EBF;&#x7A0B;&#x8FD0;&#x884C; Task&#x3002;&#x6BCF;&#x4E2A; <code>CoarseGrainedExecutorBackend</code> &#x80FD;&#x5E76;&#x884C;&#x8FD0;&#x884C; Task &#x7684;&#x6570;&#x91CF;&#x5C31;&#x53D6;&#x51B3;&#x4E8E;&#x5206;&#x914D;&#x7ED9;&#x5B83;&#x7684; CPU &#x7684;&#x4E2A;&#x6570;&#x4E86;&#xFF1B;</li>
<li><strong>Cluster Manager</strong>&#xFF1A;&#x6307;&#x7684;&#x662F;&#x5728;&#x96C6;&#x7FA4;&#x4E0A;&#x83B7;&#x53D6;&#x8D44;&#x6E90;&#x7684;&#x5916;&#x90E8;&#x670D;&#x52A1;&#xFF0C;&#x76EE;&#x524D;&#x6709;&#xFF1A;<ul>
<li>Standalone&#xFF1A;Spark&#x539F;&#x751F;&#x7684;&#x8D44;&#x6E90;&#x7BA1;&#x7406;&#xFF0C;&#x7531;Master&#x8D1F;&#x8D23;&#x8D44;&#x6E90;&#x7684;&#x5206;&#x914D;&#xFF1B;</li>
<li>Hadoop Yarn&#xFF1A;&#x7531;YARN&#x4E2D;&#x7684;ResourceManager&#x8D1F;&#x8D23;&#x8D44;&#x6E90;&#x7684;&#x5206;&#x914D;&#xFF1B;</li>
</ul>
</li>
<li><strong>Worker</strong>&#xFF1A;&#x96C6;&#x7FA4;&#x4E2D;&#x4EFB;&#x4F55;&#x53EF;&#x4EE5;&#x8FD0;&#x884C;Application&#x4EE3;&#x7801;&#x7684;&#x8282;&#x70B9;&#xFF0C;&#x7C7B;&#x4F3C;&#x4E8E;YARN&#x4E2D;&#x7684;NodeManager&#x8282;&#x70B9;&#x3002;&#x5728;Standalone&#x6A21;&#x5F0F;&#x4E2D;&#x6307;&#x7684;&#x5C31;&#x662F;&#x901A;&#x8FC7;Slave&#x6587;&#x4EF6;&#x914D;&#x7F6E;&#x7684;Worker&#x8282;&#x70B9;&#xFF0C;&#x5728;Spark on Yarn&#x6A21;&#x5F0F;&#x4E2D;&#x6307;&#x7684;&#x5C31;&#x662F;NodeManager&#x8282;&#x70B9;&#xFF1B;</li>
<li><strong>&#x4F5C;&#x4E1A;&#xFF08;Job&#xFF09;</strong>&#xFF1A;&#x5305;&#x542B;&#x591A;&#x4E2A;Task&#x7EC4;&#x6210;&#x7684;&#x5E76;&#x884C;&#x8BA1;&#x7B97;&#xFF0C;&#x5F80;&#x5F80;&#x7531;Spark Action&#x50AC;&#x751F;&#xFF0C;&#x4E00;&#x4E2A;JOB&#x5305;&#x542B;&#x591A;&#x4E2A;RDD&#x53CA;&#x4F5C;&#x7528;&#x4E8E;&#x76F8;&#x5E94;RDD&#x4E0A;&#x7684;&#x5404;&#x79CD;Operation&#xFF1B;</li>
<li><strong>&#x9636;&#x6BB5;&#xFF08;Stage&#xFF09;</strong>&#xFF1A;&#x6BCF;&#x4E2A;Job&#x4F1A;&#x88AB;&#x62C6;&#x5206;&#x5F88;&#x591A;&#x7EC4; Task&#xFF0C;&#x6BCF;&#x7EC4;&#x4EFB;&#x52A1;&#x88AB;&#x79F0;&#x4E3A;Stage&#xFF0C;&#x4E5F;&#x53EF;&#x79F0;TaskSet&#xFF0C;&#x4E00;&#x4E2A;&#x4F5C;&#x4E1A;&#x5206;&#x4E3A;&#x591A;&#x4E2A;&#x9636;&#x6BB5;&#xFF0C;&#x6BCF;&#x4E00;&#x4E2A;stage&#x7684;&#x5206;&#x5272;&#x70B9;&#x662F;action&#x3002;&#x6BD4;&#x5982;&#x4E00;&#x4E2A;job&#x662F;&#xFF1A;&#xFF08;transformation1 -&gt; transformation1 -&gt; action1 -&gt; transformation3 -&gt; action2&#xFF09;&#xFF0C;&#x8FD9;&#x4E2A;job&#x5C31;&#x4F1A;&#x88AB;&#x5206;&#x4E3A;&#x4E24;&#x4E2A;stage&#xFF0C;&#x5206;&#x5272;&#x70B9;&#x662F;action1&#x548C;action2&#x3002;</li>
<li><p><strong>&#x4EFB;&#x52A1;&#xFF08;Task&#xFF09;</strong>&#xFF1A; &#x88AB;&#x9001;&#x5230;&#x67D0;&#x4E2A;Executor&#x4E0A;&#x7684;&#x5DE5;&#x4F5C;&#x4EFB;&#x52A1;&#xFF1B;</p>
</li>
<li><p><strong>Context</strong>&#xFF1A;&#x542F;&#x52A8;spark application&#x7684;&#x65F6;&#x5019;&#x521B;&#x5EFA;&#xFF0C;&#x4F5C;&#x4E3A;Spark &#x8FD0;&#x884C;&#x65F6;&#x73AF;&#x5883;&#x3002;</p>
</li>
<li><strong>Dynamic Allocation&#xFF08;&#x52A8;&#x6001;&#x8D44;&#x6E90;&#x5206;&#x914D;&#xFF09;</strong>&#xFF1A;&#x4E00;&#x4E2A;&#x914D;&#x7F6E;&#x9009;&#x9879;&#xFF0C;&#x53EF;&#x4EE5;&#x5C06;&#x5176;&#x6253;&#x5F00;&#x3002;&#x4ECE;Spark1.2&#x4E4B;&#x540E;&#xFF0C;&#x5BF9;&#x4E8E;On Yarn&#x6A21;&#x5F0F;&#xFF0C;&#x5DF2;&#x7ECF;&#x652F;&#x6301;&#x52A8;&#x6001;&#x8D44;&#x6E90;&#x5206;&#x914D;&#xFF08;Dynamic Resource Allocation&#xFF09;&#xFF0C;&#x8FD9;&#x6837;&#xFF0C;&#x5C31;&#x53EF;&#x4EE5;&#x6839;&#x636E;Application&#x7684;&#x8D1F;&#x8F7D;&#xFF08;Task&#x60C5;&#x51B5;&#xFF09;&#xFF0C;&#x52A8;&#x6001;&#x7684;&#x589E;&#x52A0;&#x548C;&#x51CF;&#x5C11;executors&#xFF0C;&#x8FD9;&#x79CD;&#x7B56;&#x7565;&#x975E;&#x5E38;&#x9002;&#x5408;&#x5728;YARN&#x4E0A;&#x4F7F;&#x7528;spark-sql&#x505A;&#x6570;&#x636E;&#x5F00;&#x53D1;&#x548C;&#x5206;&#x6790;&#xFF0C;&#x4EE5;&#x53CA;&#x5C06;spark-sql&#x4F5C;&#x4E3A;&#x957F;&#x670D;&#x52A1;&#x6765;&#x4F7F;&#x7528;&#x7684;&#x573A;&#x666F;&#x3002;Executor &#x7684;&#x52A8;&#x6001;&#x5206;&#x914D;&#x9700;&#x8981;&#x5728; cluster mode &#x4E0B;&#x542F;&#x7528; &quot;external shuffle service&quot;&#x3002;</li>
<li><strong>&#x52A8;&#x6001;&#x8D44;&#x6E90;&#x5206;&#x914D;&#x7B56;&#x7565;</strong>&#xFF1A;&#x5F00;&#x542F;&#x52A8;&#x6001;&#x5206;&#x914D;&#x7B56;&#x7565;&#x540E;&#xFF0C;application&#x4F1A;&#x5728;task&#x56E0;&#x6CA1;&#x6709;&#x8DB3;&#x591F;&#x8D44;&#x6E90;&#x88AB;&#x6302;&#x8D77;&#x7684;&#x65F6;&#x5019;&#x53BB;&#x52A8;&#x6001;&#x7533;&#x8BF7;&#x8D44;&#x6E90;&#xFF0C;&#x8FD9;&#x610F;&#x5473;&#x7740;&#x8BE5;application&#x73B0;&#x6709;&#x7684;executor&#x65E0;&#x6CD5;&#x6EE1;&#x8DB3;&#x6240;&#x6709;task&#x5E76;&#x884C;&#x8FD0;&#x884C;&#x3002;spark&#x4E00;&#x8F6E;&#x4E00;&#x8F6E;&#x7684;&#x7533;&#x8BF7;&#x8D44;&#x6E90;&#xFF0C;&#x5F53;&#x6709;task&#x6302;&#x8D77;&#x6216;&#x7B49;&#x5F85; <code>spark.dynamicAllocation.schedulerBacklogTimeout</code> (&#x9ED8;&#x8BA4;1s)&#x65F6;&#x95F4;&#x7684;&#x65F6;&#x5019;&#xFF0C;&#x4F1A;&#x5F00;&#x59CB;&#x52A8;&#x6001;&#x8D44;&#x6E90;&#x5206;&#x914D;&#xFF1B;&#x4E4B;&#x540E;&#x4F1A;&#x6BCF;&#x9694; <code>spark.dynamicAllocation.sustainedSchedulerBacklogTimeout</code> (&#x9ED8;&#x8BA4;1s)&#x65F6;&#x95F4;&#x7533;&#x8BF7;&#x4E00;&#x6B21;&#xFF0C;&#x76F4;&#x5230;&#x7533;&#x8BF7;&#x5230;&#x8DB3;&#x591F;&#x7684;&#x8D44;&#x6E90;&#x3002;&#x6BCF;&#x6B21;&#x7533;&#x8BF7;&#x7684;&#x8D44;&#x6E90;&#x91CF;&#x662F;&#x6307;&#x6570;&#x589E;&#x957F;&#x7684;&#xFF0C;&#x5373;1,2,4,8&#x7B49;&#x3002;&#x4E4B;&#x6240;&#x4EE5;&#x91C7;&#x7528;&#x6307;&#x6570;&#x589E;&#x957F;&#xFF0C;&#x51FA;&#x4E8E;&#x4E24;&#x65B9;&#x9762;&#x8003;&#x8651;&#xFF1A;&#x5176;&#x4E00;&#xFF0C;&#x5F00;&#x59CB;&#x7533;&#x8BF7;&#x7684;&#x5C11;&#x662F;&#x8003;&#x8651;&#x5230;&#x53EF;&#x80FD;application&#x4F1A;&#x9A6C;&#x4E0A;&#x5F97;&#x5230;&#x6EE1;&#x8DB3;&#xFF1B;&#x5176;&#x6B21;&#x8981;&#x6210;&#x500D;&#x589E;&#x52A0;&#xFF0C;&#x662F;&#x4E3A;&#x4E86;&#x9632;&#x6B62;application&#x9700;&#x8981;&#x5F88;&#x591A;&#x8D44;&#x6E90;&#xFF0C;&#x800C;&#x8BE5;&#x65B9;&#x5F0F;&#x53EF;&#x4EE5;&#x5728;&#x5F88;&#x5C11;&#x6B21;&#x6570;&#x7684;&#x7533;&#x8BF7;&#x4E4B;&#x540E;&#x5F97;&#x5230;&#x6EE1;&#x8DB3;&#x3002;</li>
</ul>
<h2 id="&#x67B6;&#x6784;&#x8BBE;&#x8BA1;">&#x67B6;&#x6784;&#x8BBE;&#x8BA1;</h2>
<p>&#x5173;&#x4E8E; spark standalone &#x7684;&#x5C40;&#x9650;&#x6027;&#x4E0E; kubernetes native spark &#x67B6;&#x6784;&#x4E4B;&#x95F4;&#x7684;&#x533A;&#x522B;&#x8BF7;&#x53C2;&#x8003; Anirudh Ramanathan &#x5728; 2016&#x5E74;10&#x6708;8&#x65E5;&#x63D0;&#x4EA4;&#x7684; issue <a href="https://github.com/kubernetes/kubernetes/issues/34377" target="_blank">Support Spark natively in Kubernetes #34377</a>&#x3002;</p>
<p>&#x7B80;&#x800C;&#x8A00;&#x4E4B;&#xFF0C;spark standalone on kubernetes &#x6709;&#x5982;&#x4E0B;&#x51E0;&#x4E2A;&#x7F3A;&#x70B9;&#xFF1A;</p>
<ul>
<li>&#x65E0;&#x6CD5;&#x5BF9;&#x4E8E;&#x591A;&#x79DF;&#x6237;&#x505A;&#x9694;&#x79BB;&#xFF0C;&#x6BCF;&#x4E2A;&#x7528;&#x6237;&#x90FD;&#x60F3;&#x7ED9; pod &#x7533;&#x8BF7; node &#x8282;&#x70B9;&#x53EF;&#x7528;&#x7684;&#x6700;&#x5927;&#x7684;&#x8D44;&#x6E90;&#x3002;</li>
<li>Spark &#x7684; master&#xFF0F;worker &#x672C;&#x6765;&#x4E0D;&#x662F;&#x8BBE;&#x8BA1;&#x6210;&#x4F7F;&#x7528; kubernetes &#x7684;&#x8D44;&#x6E90;&#x8C03;&#x5EA6;&#xFF0C;&#x8FD9;&#x6837;&#x4F1A;&#x5B58;&#x5728;&#x4E24;&#x5C42;&#x7684;&#x8D44;&#x6E90;&#x8C03;&#x5EA6;&#x95EE;&#x9898;&#xFF0C;&#x4E0D;&#x5229;&#x4E8E;&#x4E0E; kuberentes &#x96C6;&#x6210;&#x3002;</li>
</ul>
<p>&#x800C; kubernetes native spark &#x96C6;&#x7FA4;&#x4E2D;&#xFF0C;spark &#x53EF;&#x4EE5;&#x8C03;&#x7528; kubernetes API &#x83B7;&#x53D6;&#x96C6;&#x7FA4;&#x8D44;&#x6E90;&#x548C;&#x8C03;&#x5EA6;&#x3002;&#x8981;&#x5B9E;&#x73B0; kubernetes native spark &#x9700;&#x8981;&#x4E3A; spark &#x63D0;&#x4F9B;&#x4E00;&#x4E2A;&#x96C6;&#x7FA4;&#x5916;&#x90E8;&#x7684; manager &#x53EF;&#x4EE5;&#x7528;&#x6765;&#x8DDF; kubernetes API &#x4EA4;&#x4E92;&#x3002;</p>
<h3 id="&#x8C03;&#x5EA6;&#x5668;&#x540E;&#x53F0;">&#x8C03;&#x5EA6;&#x5668;&#x540E;&#x53F0;</h3>
<p>&#x4F7F;&#x7528; kubernetes &#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684; spark &#x7684;&#x57FA;&#x672C;&#x8BBE;&#x8BA1;&#x601D;&#x8DEF;&#x662F;&#x5C06; spark &#x7684; driver &#x548C; executor &#x90FD;&#x653E;&#x5728; kubernetes &#x7684; pod &#x4E2D;&#x8FD0;&#x884C;&#xFF0C;&#x53E6;&#x5916;&#x8FD8;&#x6709;&#x4E24;&#x4E2A;&#x9644;&#x52A0;&#x7684;&#x7EC4;&#x4EF6;&#xFF1A;<code>ResourceStagingServer</code> &#x548C; <code>KubernetesExternalShuffleService</code>&#x3002;</p>
<p>Spark driver &#x5176;&#x5B9E;&#x53EF;&#x4EE5;&#x8FD0;&#x884C;&#x5728; kubernetes &#x96C6;&#x7FA4;&#x5185;&#x90E8;&#xFF08;cluster mode&#xFF09;&#x53EF;&#x4EE5;&#x8FD0;&#x884C;&#x5728;&#x5916;&#x90E8;&#xFF08;client mode&#xFF09;&#xFF0C;executor &#x53EA;&#x80FD;&#x8FD0;&#x884C;&#x5728;&#x96C6;&#x7FA4;&#x5185;&#x90E8;&#xFF0C;&#x5F53;&#x6709; spark &#x4F5C;&#x4E1A;&#x63D0;&#x4EA4;&#x5230; kubernetes &#x96C6;&#x7FA4;&#x4E0A;&#x65F6;&#xFF0C;&#x8C03;&#x5EA6;&#x5668;&#x540E;&#x53F0;&#x5C06;&#x4F1A;&#x4E3A; executor pod &#x8BBE;&#x7F6E;&#x5982;&#x4E0B;&#x5C5E;&#x6027;&#xFF1A;</p>
<ul>
<li>&#x4F7F;&#x7528;&#x6211;&#x4EEC;&#x9884;&#x5148;&#x7F16;&#x8BD1;&#x597D;&#x7684;&#x5305;&#x542B; kubernetes &#x652F;&#x6301;&#x7684; spark &#x955C;&#x50CF;&#xFF0C;&#x7136;&#x540E;&#x8C03;&#x7528; <code>CoarseGrainedExecutorBackend</code> main class &#x542F;&#x52A8; JVM&#x3002;</li>
<li>&#x8C03;&#x5EA6;&#x5668;&#x540E;&#x53F0;&#x4E3A; executor pod &#x7684;&#x8FD0;&#x884C;&#x65F6;&#x6CE8;&#x5165;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#xFF0C;&#x4F8B;&#x5982;&#x5404;&#x79CD; JVM &#x53C2;&#x6570;&#xFF0C;&#x5305;&#x62EC;&#x7528;&#x6237;&#x5728; <code>spark-submit</code> &#x65F6;&#x6307;&#x5B9A;&#x7684;&#x90A3;&#x4E9B;&#x53C2;&#x6570;&#x3002;</li>
<li>Executor &#x7684; CPU&#x3001;&#x5185;&#x5B58;&#x9650;&#x5236;&#x6839;&#x636E;&#x8FD9;&#x4E9B;&#x6CE8;&#x5165;&#x7684;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#x4FDD;&#x5B58;&#x5230;&#x5E94;&#x7528;&#x7A0B;&#x5E8F;&#x7684; <code>SparkConf</code> &#x4E2D;&#x3002;</li>
<li>&#x53EF;&#x4EE5;&#x5728;&#x914D;&#x7F6E;&#x4E2D;&#x6307;&#x5B9A; spark &#x8FD0;&#x884C;&#x5728;&#x6307;&#x5B9A;&#x7684; namespace &#x4E2D;&#x3002;</li>
</ul>
<p>&#x53C2;&#x8003;&#xFF1A;<a href="https://github.com/apache-spark-on-k8s/spark/blob/branch-2.2-kubernetes/resource-managers/kubernetes/architecture-docs/scheduler-backend.md" target="_blank">Scheduler backend &#x6587;&#x6863;</a></p>
<h2 id="&#x5B89;&#x88C5;&#x6307;&#x5357;">&#x5B89;&#x88C5;&#x6307;&#x5357;</h2>
<p>&#x6211;&#x4EEC;&#x53EF;&#x4EE5;&#x76F4;&#x63A5;&#x4F7F;&#x7528;&#x5B98;&#x65B9;&#x5DF2;&#x7F16;&#x8BD1;&#x597D;&#x7684; docker &#x955C;&#x50CF;&#x6765;&#x90E8;&#x7F72;&#xFF0C;&#x4E0B;&#x9762;&#x662F;&#x5B98;&#x65B9;&#x53D1;&#x5E03;&#x7684;&#x955C;&#x50CF;&#xFF1A;</p>
<table>
<thead>
<tr>
<th>&#x7EC4;&#x4EF6;</th>
<th>&#x955C;&#x50CF;</th>
</tr>
</thead>
<tbody>
<tr>
<td>Spark Driver Image</td>
<td><code>kubespark/spark-driver:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
<tr>
<td>Spark Executor Image</td>
<td><code>kubespark/spark-executor:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
<tr>
<td>Spark Initialization Image</td>
<td><code>kubespark/spark-init:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
<tr>
<td>Spark Staging Server Image</td>
<td><code>kubespark/spark-resource-staging-server:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
<tr>
<td>PySpark Driver Image</td>
<td><code>kubespark/driver-py:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
<tr>
<td>PySpark Executor Image</td>
<td><code>kubespark/executor-py:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
</tbody>
</table>
<p>&#x6211;&#x5C06;&#x8FD9;&#x4E9B;&#x955C;&#x50CF;&#x653E;&#x5230;&#x4E86;&#x6211;&#x7684;&#x79C1;&#x6709;&#x955C;&#x50CF;&#x4ED3;&#x5E93;&#x4E2D;&#x4E86;&#x3002;</p>
<p>&#x8FD8;&#x9700;&#x8981;&#x5B89;&#x88C5;&#x652F;&#x6301; kubernetes &#x7684; spark &#x5BA2;&#x6237;&#x7AEF;&#xFF0C;&#x5728;&#x8FD9;&#x91CC;&#x4E0B;&#x8F7D;&#xFF1A;<a href="https://github.com/apache-spark-on-k8s/spark/releases" target="_blank">https://github.com/apache-spark-on-k8s/spark/releases</a></p>
<p>&#x6839;&#x636E;&#x4F7F;&#x7528;&#x7684;&#x955C;&#x50CF;&#x7248;&#x672C;&#xFF0C;&#x6211;&#x4E0B;&#x8F7D;&#x7684;&#x662F; <a href="https://github.com/apache-spark-on-k8s/spark/releases/tag/v2.1.0-kubernetes-0.3.1" target="_blank">v2.1.0-kubernetes-0.3.1</a> </p>
<p><strong>&#x8FD0;&#x884C; SparkPi &#x6D4B;&#x8BD5;</strong></p>
<p>&#x6211;&#x4EEC;&#x5C06;&#x4EFB;&#x52A1;&#x8FD0;&#x884C;&#x5728; <code>spark-cluster</code> &#x7684; namespace &#x4E2D;&#xFF0C;&#x542F;&#x52A8; 5 &#x4E2A; executor &#x5B9E;&#x4F8B;&#x3002;</p>
<pre><code class="lang-bash">./bin/spark-submit \
--deploy-mode cluster \
--class org.apache.spark.examples.SparkPi \
--master k8s://https://172.20.0.113:6443 \
--kubernetes-namespace spark-cluster \
--conf spark.executor.instances=5 \
--conf spark.app.name=spark-pi \
--conf spark.kubernetes.driver.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver:v2.1.0-kubernetes-0.3.1 \
--conf spark.kubernetes.executor.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-executor:v2.1.0-kubernetes-0.3.1 \
--conf spark.kubernetes.initcontainer.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-init:v2.1.0-kubernetes-0.3.1 \
<span class="hljs-built_in">local</span>:///opt/spark/examples/jars/spark-examples_2.11-2.1.0-k8s-0.3.1-SNAPSHOT.jar
</code></pre>
<p>&#x5173;&#x4E8E;&#x8BE5;&#x547D;&#x4EE4;&#x53C2;&#x6570;&#x7684;&#x4ECB;&#x7ECD;&#x8BF7;&#x53C2;&#x8003;&#xFF1A;<a href="https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html" target="_blank">https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html</a></p>
<p><strong>&#x6CE8;&#x610F;&#xFF1A;</strong> &#x8BE5; jar &#x5305;&#x5B9E;&#x9645;&#x4E0A;&#x662F; <code>spark.kubernetes.executor.docker.image</code> &#x955C;&#x50CF;&#x4E2D;&#x7684;&#x3002;</p>
<p>&#x8FD9;&#x65F6;&#x5019;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#x8FD0;&#x884C;&#x8FD8;&#x662F;&#x5931;&#x8D25;&#xFF0C;&#x62A5;&#x9519;&#x4FE1;&#x606F;&#x4E2D;&#x53EF;&#x4EE5;&#x770B;&#x5230;&#x4E24;&#x4E2A;&#x95EE;&#x9898;&#xFF1A;</p>
<ul>
<li>Executor &#x65E0;&#x6CD5;&#x627E;&#x5230; driver pod</li>
<li>&#x7528;&#x6237; <code>system:serviceaccount:spark-cluster:defaul</code> &#x6CA1;&#x6709;&#x6743;&#x9650;&#x83B7;&#x53D6; <code>spark-cluster</code> &#x4E2D;&#x7684; pod &#x4FE1;&#x606F;&#x3002;</li>
</ul>
<p>&#x63D0;&#x4E86;&#x4E2A; issue <a href="https://github.com/apache-spark-on-k8s/spark/issues/478" target="_blank">Failed to run the sample spark-pi test using spark-submit on the doc #478</a> </p>
<p>&#x9700;&#x8981;&#x4E3A; spark &#x96C6;&#x7FA4;&#x521B;&#x5EFA;&#x4E00;&#x4E2A; <code>serviceaccount</code> &#x548C; <code>clusterrolebinding</code>&#xFF1A;</p>
<pre><code class="lang-bash">kubectl create serviceaccount spark --namespace spark-cluster
kubectl create rolebinding spark-edit --clusterrole=edit --serviceaccount=spark-cluster:spark --namespace=spark-cluster
</code></pre>
<p>&#x8BE5; Bug &#x5C06;&#x5728;&#x65B0;&#x7248;&#x672C;&#x4E2D;&#x4FEE;&#x590D;&#x3002;</p>
<h2 id="&#x7528;&#x6237;&#x6307;&#x5357;">&#x7528;&#x6237;&#x6307;&#x5357;</h2>
<h3 id="&#x7F16;&#x8BD1;">&#x7F16;&#x8BD1;</h3>
<p>Fork &#x5E76;&#x514B;&#x9686;&#x9879;&#x76EE;&#x5230;&#x672C;&#x5730;&#xFF1A;</p>
<pre><code class="lang-bash">git <span class="hljs-built_in">clone</span> https://github.com/rootsongjc/spark.git
</code></pre>
<p>&#x7F16;&#x8BD1;&#x524D;&#x8BF7;&#x786E;&#x4FDD;&#x4F60;&#x7684;&#x73AF;&#x5883;&#x4E2D;&#x5DF2;&#x7ECF;&#x5B89;&#x88C5; Java8 &#x548C; Maven3&#x3002;</p>
<pre><code class="lang-bash"><span class="hljs-comment">## &#x7B2C;&#x4E00;&#x6B21;&#x7F16;&#x8BD1;&#x524D;&#x9700;&#x8981;&#x5B89;&#x88C5;&#x4F9D;&#x8D56;</span>
build/mvn install -Pkubernetes -pl resource-managers/kubernetes/core -am -DskipTests
<span class="hljs-comment">## &#x7F16;&#x8BD1; spark on kubernetes</span>
build/mvn compile -Pkubernetes -pl resource-managers/kubernetes/core -am -DskipTests
<span class="hljs-comment">## &#x53D1;&#x5E03;</span>
dev/make-distribution.sh --tgz -Phadoop-2.7 -Pkubernetes
</code></pre>
<p>&#x7B2C;&#x4E00;&#x6B21;&#x7F16;&#x8BD1;&#x548C;&#x53D1;&#x5E03;&#x7684;&#x8FC7;&#x7A0B;&#x8017;&#x65F6;&#x53EF;&#x80FD;&#x4F1A;&#x6BD4;&#x8F83;&#x957F;&#xFF0C;&#x8BF7;&#x8010;&#x5FC3;&#x7B49;&#x5F85;&#xFF0C;&#x5982;&#x679C;&#x6709;&#x4F9D;&#x8D56;&#x4E0B;&#x8F7D;&#x4E0D;&#x4E0B;&#x6765;&#xFF0C;&#x8BF7;&#x81EA;&#x5907;&#x68AF;&#x5B50;&#x3002;</p>
<p>&#x8BE6;&#x7EC6;&#x7684;&#x5F00;&#x53D1;&#x6307;&#x5357;&#x8BF7;&#x89C1;&#xFF1A;<a href="https://github.com/apache-spark-on-k8s/spark/blob/branch-2.2-kubernetes/resource-managers/kubernetes/README.md" target="_blank">https://github.com/apache-spark-on-k8s/spark/blob/branch-2.2-kubernetes/resource-managers/kubernetes/README.md</a></p>
<h3 id="&#x6784;&#x5EFA;&#x955C;&#x50CF;">&#x6784;&#x5EFA;&#x955C;&#x50CF;</h3>
<p>&#x4F7F;&#x7528;&#x8BE5;&#x811A;&#x672C;&#x6765;&#x81EA;&#x52A8;&#x6784;&#x5EFA;&#x5BB9;&#x5668;&#x955C;&#x50CF;&#xFF1A;<a href="https://github.com/apache-spark-on-k8s/spark/pull/488" target="_blank">https://github.com/apache-spark-on-k8s/spark/pull/488</a></p>
<p>&#x5C06;&#x8BE5;&#x811A;&#x672C;&#x653E;&#x5728; <code>dist</code> &#x76EE;&#x5F55;&#x4E0B;&#xFF0C;&#x6267;&#x884C;&#xFF1A;</p>
<pre><code class="lang-bash">./build-push-docker-images.sh -r sz-pg-oam-docker-hub-001.tendcloud.com/library -t v2.1.0-kubernetes-0.3.1-1 build
./build-push-docker-images.sh -r sz-pg-oam-docker-hub-001.tendcloud.com/library -t v2.1.0-kubernetes-0.3.1-1 push
</code></pre>
<p><strong>&#x6CE8;&#x610F;&#xFF1A;</strong>&#x5982;&#x679C;&#x4F60;&#x4F7F;&#x7528;&#x7684; MacOS&#xFF0C;bash &#x7684;&#x7248;&#x672C;&#x53EF;&#x80FD;&#x592A;&#x4F4E;&#xFF0C;&#x6267;&#x884C;&#x6539;&#x811A;&#x672C;&#x5C06;&#x51FA;&#x9519;&#xFF0C;&#x8BF7;&#x68C0;&#x67E5;&#x4F60;&#x7684; bash &#x7248;&#x672C;&#xFF1A;</p>
<pre><code class="lang-bash">bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16)
Copyright (C) 2007 Free Software Foundation, Inc.
</code></pre>
<p>&#x4E0A;&#x9762;&#x6211;&#x5728;&#x5347;&#x7EA7; bash &#x4E4B;&#x524D;&#x83B7;&#x53D6;&#x7684;&#x7248;&#x672C;&#x4FE1;&#x606F;&#xFF0C;&#x4F7F;&#x7528;&#x4E0B;&#x9762;&#x7684;&#x547D;&#x4EE4;&#x5347;&#x7EA7; bash&#xFF1A;</p>
<pre><code class="lang-bash">brew install bash
</code></pre>
<p>&#x5347;&#x7EA7;&#x540E;&#x7684; bash &#x7248;&#x672C;&#x4E3A; <code>4.4.12(1)-release (x86_64-apple-darwin16.3.0)</code>&#x3002;</p>
<p>&#x7F16;&#x8BD1;&#x5E76;&#x4E0A;&#x4F20;&#x955C;&#x50CF;&#x5230;&#x6211;&#x7684;&#x79C1;&#x6709;&#x955C;&#x50CF;&#x4ED3;&#x5E93;&#xFF0C;&#x5C06;&#x4F1A;&#x6784;&#x5EFA;&#x51FA;&#x5982;&#x4E0B;&#x51E0;&#x4E2A;&#x955C;&#x50CF;&#xFF1A;</p>
<pre><code class="lang-bash">sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-resource-staging-server:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-init:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-shuffle:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor-py:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver-py:v2.1.0-kubernetes-0.3.1-1
</code></pre>
<h2 id="&#x8FD0;&#x884C;&#x6D4B;&#x8BD5;">&#x8FD0;&#x884C;&#x6D4B;&#x8BD5;</h2>
<p>&#x5728; <code>dist/bin</code> &#x76EE;&#x5F55;&#x4E0B;&#x6267;&#x884C; spark-pi &#x6D4B;&#x8BD5;&#xFF1A;</p>
<pre><code class="lang-bash">./spark-submit \
--deploy-mode cluster \
--class org.apache.spark.examples.SparkPi \
--master k8s://https://172.20.0.113:6443 \
--kubernetes-namespace spark-cluster \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.executor.instances=5 \
--conf spark.app.name=spark-pi \
--conf spark.kubernetes.driver.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.executor.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.initcontainer.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-init:v2.1.0-kubernetes-0.3.1-1 \
<span class="hljs-built_in">local</span>:///opt/spark/examples/jars/spark-examples_2.11-2.2.0-k8s-0.4.0-SNAPSHOT.jar
</code></pre>
<p>&#x8BE6;&#x7EC6;&#x7684;&#x53C2;&#x6570;&#x8BF4;&#x660E;&#x89C1;&#xFF1A;<a href="https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html" target="_blank">https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html</a></p>
<p><strong>&#x6CE8;&#x610F;&#xFF1A;</strong><code>local:///opt/spark/examples/jars/spark-examples_2.11-2.2.0-k8s-0.4.0-SNAPSHOT.jar</code> &#x6587;&#x4EF6;&#x662F;&#x5728; <code>spark-driver</code> &#x548C; <code>spark-executor</code> &#x955C;&#x50CF;&#x91CC;&#x7684;&#xFF0C;&#x5728;&#x4E0A;&#x4E00;&#x6B65;&#x6784;&#x5EFA;&#x955C;&#x50CF;&#x65F6;&#x5DF2;&#x7ECF;&#x6784;&#x5EFA;&#x5E76;&#x4E0A;&#x4F20;&#x5230;&#x4E86;&#x955C;&#x50CF;&#x4ED3;&#x5E93;&#x4E2D;&#x3002;</p>
<p>&#x6267;&#x884C;&#x65E5;&#x5FD7;&#x663E;&#x793A;&#xFF1A;</p>
<pre><code class="lang-bash">2017-09-14 14:59:01 INFO Client:54 - Waiting <span class="hljs-keyword">for</span> application spark-pi to finish...
2017-09-14 14:59:01 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
pod name: spark-pi-1505372339796-driver
namespace: spark-cluster
labels: spark-app-selector -&gt; spark<span class="hljs-_">-f</span>4d3a5d3ad964a05a51feb6191d50357, spark-role -&gt; driver
pod uid: 304cf440-991a-11e7-970c<span class="hljs-_">-f</span>4e9d49f8ed0
creation time: 2017-09-14T06:59:01Z
service account name: spark
volumes: spark-token-zr8wv
node name: N/A
start time: N/A
container images: N/A
phase: Pending
status: []
2017-09-14 14:59:01 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
pod name: spark-pi-1505372339796-driver
namespace: spark-cluster
labels: spark-app-selector -&gt; spark<span class="hljs-_">-f</span>4d3a5d3ad964a05a51feb6191d50357, spark-role -&gt; driver
pod uid: 304cf440-991a-11e7-970c<span class="hljs-_">-f</span>4e9d49f8ed0
creation time: 2017-09-14T06:59:01Z
service account name: spark
volumes: spark-token-zr8wv
node name: 172.20.0.114
start time: N/A
container images: N/A
phase: Pending
status: []
2017-09-14 14:59:01 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
pod name: spark-pi-1505372339796-driver
namespace: spark-cluster
labels: spark-app-selector -&gt; spark<span class="hljs-_">-f</span>4d3a5d3ad964a05a51feb6191d50357, spark-role -&gt; driver
pod uid: 304cf440-991a-11e7-970c<span class="hljs-_">-f</span>4e9d49f8ed0
creation time: 2017-09-14T06:59:01Z
service account name: spark
volumes: spark-token-zr8wv
node name: 172.20.0.114
start time: 2017-09-14T06:59:01Z
container images: sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1
phase: Pending
status: [ContainerStatus(containerID=null, image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1, imageID=, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=<span class="hljs-literal">false</span>, restartCount=0, state=ContainerState(running=null, terminated=null, waiting=ContainerStateWaiting(message=null, reason=ContainerCreating, additionalProperties={}), additionalProperties={}), additionalProperties={})]
2017-09-14 14:59:03 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
pod name: spark-pi-1505372339796-driver
namespace: spark-cluster
labels: spark-app-selector -&gt; spark<span class="hljs-_">-f</span>4d3a5d3ad964a05a51feb6191d50357, spark-role -&gt; driver
pod uid: 304cf440-991a-11e7-970c<span class="hljs-_">-f</span>4e9d49f8ed0
creation time: 2017-09-14T06:59:01Z
service account name: spark
volumes: spark-token-zr8wv
node name: 172.20.0.114
start time: 2017-09-14T06:59:01Z
container images: sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1
phase: Running
status: [ContainerStatus(containerID=docker://5c5c821c482a1e35552adccb567020532b79244392374f25754f0050e6<span class="hljs-built_in">cd</span>4c62, image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1, imageID=docker-pullable://sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver@sha256:beb92a3e3f178e286d9e5baebdead88b5ba76d651f347ad2864bb6f8eda26f94, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=<span class="hljs-literal">true</span>, restartCount=0, state=ContainerState(running=ContainerStateRunning(startedAt=2017-09-14T06:59:02Z, additionalProperties={}), terminated=null, waiting=null, additionalProperties={}), additionalProperties={})]
2017-09-14 14:59:12 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
pod name: spark-pi-1505372339796-driver
namespace: spark-cluster
labels: spark-app-selector -&gt; spark<span class="hljs-_">-f</span>4d3a5d3ad964a05a51feb6191d50357, spark-role -&gt; driver
pod uid: 304cf440-991a-11e7-970c<span class="hljs-_">-f</span>4e9d49f8ed0
creation time: 2017-09-14T06:59:01Z
service account name: spark
volumes: spark-token-zr8wv
node name: 172.20.0.114
start time: 2017-09-14T06:59:01Z
container images: sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1
phase: Succeeded
status: [ContainerStatus(containerID=docker://5c5c821c482a1e35552adccb567020532b79244392374f25754f0050e6<span class="hljs-built_in">cd</span>4c62, image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1, imageID=docker-pullable://sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver@sha256:beb92a3e3f178e286d9e5baebdead88b5ba76d651f347ad2864bb6f8eda26f94, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=<span class="hljs-literal">false</span>, restartCount=0, state=ContainerState(running=null, terminated=ContainerStateTerminated(containerID=docker://5c5c821c482a1e35552adccb567020532b79244392374f25754f0050e6<span class="hljs-built_in">cd</span>4c62, <span class="hljs-built_in">exit</span>Code=0, finishedAt=2017-09-14T06:59:11Z, message=null, reason=Completed, signal=null, startedAt=null, additionalProperties={}), waiting=null, additionalProperties={}), additionalProperties={})]
2017-09-14 14:59:12 INFO LoggingPodStatusWatcherImpl:54 - Container final statuses:
Container name: spark-kubernetes-driver
Container image: sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1
Container state: Terminated
Exit code: 0
2017-09-14 14:59:12 INFO Client:54 - Application spark-pi finished.
</code></pre>
<p>&#x4ECE;&#x65E5;&#x5FD7;&#x4E2D;&#x53EF;&#x4EE5;&#x770B;&#x5230;&#x4EFB;&#x52A1;&#x8FD0;&#x884C;&#x7684;&#x72B6;&#x6001;&#x4FE1;&#x606F;&#x3002;</p>
<p>&#x4F7F;&#x7528;&#x4E0B;&#x9762;&#x7684;&#x547D;&#x4EE4;&#x53EF;&#x4EE5;&#x770B;&#x5230; kubernetes &#x542F;&#x52A8;&#x7684; Pod &#x4FE1;&#x606F;&#xFF1A;</p>
<pre><code class="lang-bash">kubectl --namespace spark-cluster get pods -w
</code></pre>
<p>&#x5C06;&#x4F1A;&#x770B;&#x5230; <code>spark-driver</code> &#x548C; <code>spark-exec</code> &#x7684; Pod &#x4FE1;&#x606F;&#x3002;</p>
<h2 id="&#x4F9D;&#x8D56;&#x7BA1;&#x7406;">&#x4F9D;&#x8D56;&#x7BA1;&#x7406;</h2>
<p>&#x4E0A;&#x6587;&#x4E2D;&#x6211;&#x4EEC;&#x5728;&#x8FD0;&#x884C;&#x6D4B;&#x8BD5;&#x7A0B;&#x5E8F;&#x65F6;&#xFF0C;&#x547D;&#x4EE4;&#x884C;&#x4E2D;&#x6307;&#x5B9A;&#x7684; jar &#x6587;&#x4EF6;&#x5DF2;&#x5305;&#x542B;&#x5728; docker &#x955C;&#x50CF;&#x4E2D;&#xFF0C;&#x662F;&#x4E0D;&#x662F;&#x8BF4;&#x6211;&#x4EEC;&#x6BCF;&#x6B21;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#x90FD;&#x9700;&#x8981;&#x91CD;&#x65B0;&#x521B;&#x5EFA;&#x4E00;&#x4E2A;&#x955C;&#x50CF;&#x5462;&#xFF1F;&#x975E;&#x4E5F;&#xFF01;&#x5982;&#x679C;&#x771F;&#x662F;&#x8FD9;&#x6837;&#x4E5F;&#x592A;&#x9EBB;&#x70E6;&#x4E86;&#x3002;</p>
<h4 id="&#x521B;&#x5EFA;-resource-staging-server">&#x521B;&#x5EFA; resource staging server</h4>
<p>&#x4E3A;&#x4E86;&#x65B9;&#x4FBF;&#x7528;&#x6237;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#xFF0C;&#x4E0D;&#x9700;&#x8981;&#x6BCF;&#x6B21;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#x7684;&#x65F6;&#x5019;&#x90FD;&#x521B;&#x5EFA;&#x4E00;&#x4E2A;&#x955C;&#x50CF;&#xFF0C;&#x6211;&#x4EEC;&#x4F7F;&#x7528;&#x4E86; <strong>resource staging server</strong> &#x3002;</p>
<pre><code>kubectl create -f conf/kubernetes-resource-staging-server.yaml
</code></pre><p>&#x6211;&#x4EEC;&#x540C;&#x6837;&#x5C06;&#x5176;&#x90E8;&#x7F72;&#x5728; <code>spark-cluster</code> namespace &#x4E0B;&#xFF0C;&#x8BE5; yaml &#x6587;&#x4EF6;&#x89C1; <a href="https://github.com/rootsongjc/kubernetes-handbook" target="_blank">kubernetes-handbook</a> &#x7684; <code>manifests/spark-with-kubernetes-native-scheduler</code> &#x76EE;&#x5F55;&#x3002;</p>
<h4 id="&#x4F18;&#x5316;">&#x4F18;&#x5316;</h4>
<p>&#x5176;&#x4E2D;&#x6709;&#x4E00;&#x70B9;&#x9700;&#x8981;&#x4F18;&#x5316;&#xFF0C;&#x5728;&#x4F7F;&#x7528;&#x4E0B;&#x9762;&#x7684;&#x547D;&#x4EE4;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#x65F6;&#xFF0C;&#x4F7F;&#x7528; <code>--conf spark.kubernetes.resourceStagingServer.uri</code> &#x53C2;&#x6570;&#x6307;&#x5B9A; <em>resource staging server</em> &#x5730;&#x5740;&#xFF0C;&#x7528;&#x6237;&#x4E0D;&#x5E94;&#x8BE5;&#x5173;&#x6CE8; <em>resource staging server</em> &#x7A76;&#x7ADF;&#x8FD0;&#x884C;&#x5728;&#x54EA;&#x53F0;&#x5BBF;&#x4E3B;&#x673A;&#x4E0A;&#xFF0C;&#x53EF;&#x4EE5;&#x4F7F;&#x7528;&#x4E0B;&#x9762;&#x4E24;&#x79CD;&#x65B9;&#x5F0F;&#x5B9E;&#x73B0;&#xFF1A;</p>
<ul>
<li>&#x4F7F;&#x7528; <code>nodeSelector</code> &#x5C06; <em>resource staging server</em> &#x56FA;&#x5B9A;&#x8C03;&#x5EA6;&#x5230;&#x67D0;&#x4E00;&#x53F0;&#x673A;&#x5668;&#x4E0A;&#xFF0C;&#x8BE5;&#x5730;&#x5740;&#x4F9D;&#x7136;&#x4F7F;&#x7528;&#x5BBF;&#x4E3B;&#x673A;&#x7684; IP &#x5730;&#x5740;</li>
<li>&#x6539;&#x53D8; <code>spark-resource-staging-service</code> service &#x7684; type &#x4E3A; <strong>ClusterIP</strong>&#xFF0C; &#x7136;&#x540E;&#x4F7F;&#x7528; <strong>Ingress</strong> &#x5C06;&#x5176;&#x66B4;&#x9732;&#x5230;&#x96C6;&#x7FA4;&#x5916;&#x90E8;&#xFF0C;&#x7136;&#x540E;&#x52A0;&#x5165;&#x7684;&#x5185;&#x7F51; DNS &#x91CC;&#xFF0C;&#x7528;&#x6237;&#x4F7F;&#x7528; DNS &#x540D;&#x79F0;&#x6307;&#x5B9A; <em>resource staging server</em> &#x7684;&#x5730;&#x5740;&#x3002;</li>
</ul>
<p>&#x7136;&#x540E;&#x53EF;&#x4EE5;&#x6267;&#x884C;&#x4E0B;&#x9762;&#x7684;&#x547D;&#x4EE4;&#x6765;&#x63D0;&#x4EA4;&#x672C;&#x5730;&#x7684; jar &#x5230; kubernetes &#x4E0A;&#x8FD0;&#x884C;&#x3002;</p>
<pre><code class="lang-bash">./spark-submit \
--deploy-mode cluster \
--class org.apache.spark.examples.SparkPi \
--master k8s://https://172.20.0.113:6443 \
--kubernetes-namespace spark-cluster \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.executor.instances=5 \
--conf spark.app.name=spark-pi \
--conf spark.kubernetes.driver.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.executor.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.initcontainer.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-init:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.resourceStagingServer.uri=http://172.20.0.114:31000 \
../examples/jars/spark-examples_2.11-2.2.0-k8s-0.4.0-SNAPSHOT.jar
</code></pre>
<p>&#x8BE5;&#x547D;&#x4EE4;&#x5C06;&#x63D0;&#x4EA4;&#x672C;&#x5730;&#x7684; <code>../examples/jars/spark-examples_2.11-2.2.0-k8s-0.4.0-SNAPSHOT.jar</code> &#x6587;&#x4EF6;&#x5230; <em>resource staging server</em>&#xFF0C;executor &#x5C06;&#x4ECE;&#x8BE5; server &#x4E0A;&#x83B7;&#x53D6; jar &#x5305;&#x5E76;&#x8FD0;&#x884C;&#xFF0C;&#x8FD9;&#x6837;&#x7528;&#x6237;&#x5C31;&#x4E0D;&#x9700;&#x8981;&#x6BCF;&#x6B21;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#x90FD;&#x7F16;&#x8BD1;&#x4E00;&#x4E2A;&#x955C;&#x50CF;&#x4E86;&#x3002;</p>
<p>&#x8BE6;&#x89C1;&#xFF1A;<a href="https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html#dependency-management" target="_blank">https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html#dependency-management</a></p>
<h4 id="&#x8BBE;&#x7F6E;-hdfs-&#x7528;&#x6237;">&#x8BBE;&#x7F6E; HDFS &#x7528;&#x6237;</h4>
<p>&#x5982;&#x679C; Hadoop &#x96C6;&#x7FA4;&#x6CA1;&#x6709;&#x8BBE;&#x7F6E; kerbros &#x5B89;&#x5168;&#x8BA4;&#x8BC1;&#x7684;&#x8BDD;&#xFF0C;&#x5728;&#x6307;&#x5B9A; <code>spark-submit</code> &#x7684;&#x65F6;&#x5019;&#x53EF;&#x4EE5;&#x901A;&#x8FC7;&#x6307;&#x5B9A;&#x5982;&#x4E0B;&#x56DB;&#x4E2A;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#xFF0C; &#x8BBE;&#x7F6E; Spark &#x4E0E; HDFS &#x901A;&#x4FE1;&#x4F7F;&#x7528;&#x7684;&#x7528;&#x6237;&#xFF1A;</p>
<pre><code class="lang-bash"> --conf spark.kubernetes.driverEnv.SPARK_USER=hadoop
--conf spark.kubernetes.driverEnv.HADOOP_USER_NAME=hadoop
--conf spark.executorEnv.HADOOP_USER_NAME=hadoop
--conf spark.executorEnv.SPARK_USER=hadoop
</code></pre>
<p>&#x4F7F;&#x7528; hadoop &#x7528;&#x6237;&#x63D0;&#x4EA4;&#x672C;&#x5730; jar &#x5305;&#x7684;&#x547D;&#x4EE4;&#x793A;&#x4F8B;&#xFF1A;</p>
<pre><code class="lang-bash">./spark-submit \
--deploy-mode cluster \
--class com.talkingdata.alluxio.hadooptest \
--master k8s://https://172.20.0.113:6443 \
--kubernetes-namespace spark-cluster \
--conf spark.kubernetes.driverEnv.SPARK_USER=hadoop \
--conf spark.kubernetes.driverEnv.HADOOP_USER_NAME=hadoop \
--conf spark.executorEnv.HADOOP_USER_NAME=hadoop \
--conf spark.executorEnv.SPARK_USER=hadoop \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.executor.instances=5 \
--conf spark.app.name=spark-pi \
--conf spark.kubernetes.driver.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.executor.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.initcontainer.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-init:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.resourceStagingServer.uri=http://172.20.0.114:31000 \
~/Downloads/tendcloud_2.10-1.0.jar
</code></pre>
<p>&#x8BE6;&#x89C1;&#xFF1A;<a href="https://github.com/apache-spark-on-k8s/spark/issues/408" target="_blank">https://github.com/apache-spark-on-k8s/spark/issues/408</a></p>
<h4 id="&#x9650;&#x5236;-driver-&#x548C;-executor-&#x7684;&#x8D44;&#x6E90;&#x4F7F;&#x7528;">&#x9650;&#x5236; Driver &#x548C; Executor &#x7684;&#x8D44;&#x6E90;&#x4F7F;&#x7528;</h4>
<p>&#x5728;&#x6267;&#x884C; <code>spark-submit</code> &#x65F6;&#x4F7F;&#x7528;&#x5982;&#x4E0B;&#x53C2;&#x6570;&#x8BBE;&#x7F6E;&#x5185;&#x5B58;&#x548C; CPU &#x8D44;&#x6E90;&#x9650;&#x5236;&#xFF1A;</p>
<pre><code class="lang-bash">--conf spark.driver.memory=3G
--conf spark.executor.memory=3G
--conf spark.driver.cores=2
--conf spark.executor.cores=10
</code></pre>
<p>&#x8FD9;&#x51E0;&#x4E2A;&#x53C2;&#x6570;&#x4E2D;&#x503C;&#x5982;&#x4F55;&#x4F20;&#x9012;&#x5230; Pod &#x7684;&#x8D44;&#x6E90;&#x8BBE;&#x7F6E;&#x4E2D;&#x7684;&#x5462;&#xFF1F;</p>
<p>&#x6BD4;&#x5982;&#x6211;&#x4EEC;&#x8BBE;&#x7F6E;&#x5728;&#x6267;&#x884C; <code>spark-submit</code> &#x7684;&#x65F6;&#x5019;&#x4F20;&#x9012;&#x4E86;&#x8FD9;&#x6837;&#x7684;&#x4E24;&#x4E2A;&#x53C2;&#x6570;&#xFF1A;<code>--conf spark.driver.cores=2</code> &#x548C; <code>--conf spark.driver.memory=100G</code> &#x90A3;&#x4E48;&#x67E5;&#x770B; driver pod &#x7684; yaml &#x8F93;&#x51FA;&#x7ED3;&#x679C;&#x5C06;&#x4F1A;&#x770B;&#x5230;&#x8FD9;&#x6837;&#x7684;&#x8D44;&#x6E90;&#x8BBE;&#x7F6E;&#xFF1A;</p>
<pre><code class="lang-yaml"><span class="hljs-attr"> resources:</span>
<span class="hljs-attr"> limits:</span>
<span class="hljs-attr"> memory:</span> <span class="hljs-number">110</span>Gi
<span class="hljs-attr"> requests:</span>
<span class="hljs-attr"> cpu:</span> <span class="hljs-string">&quot;2&quot;</span>
<span class="hljs-attr"> memory:</span> <span class="hljs-number">100</span>Gi
</code></pre>
<p>&#x4EE5;&#x4E0A;&#x53C2;&#x6570;&#x662F;&#x5BF9; <code>request</code> &#x503C;&#x7684;&#x8BBE;&#x7F6E;&#xFF0C;&#x90A3;&#x4E48; <code>limit</code> &#x7684;&#x8D44;&#x6E90;&#x8BBE;&#x7F6E;&#x7684;&#x503C;&#x53C8;&#x662F;&#x4ECE;&#x4F55;&#x800C;&#x6765;&#xFF1F;</p>
<p>&#x53EF;&#x4EE5;&#x4F7F;&#x7528; <code>spark.kubernetes.driver.limit.cores</code> &#x548C; <code>spark.kubernetes.executor.limit.cores</code> &#x6765;&#x8BBE;&#x7F6E; CPU&#x7684; hard limit&#x3002;</p>
<p>memory limit &#x7684;&#x503C;&#x662F;&#x6839;&#x636E; memory request &#x7684;&#x503C;&#x52A0;&#x4E0A; <code>spark.kubernetes.executor.memoryOverhead</code> &#x7684;&#x503C;&#x8BA1;&#x7B97;&#x800C;&#x6765;&#x7684;&#xFF0C;&#x8BE5;&#x914D;&#x7F6E;&#x9879;&#x7528;&#x4E8E;&#x8BBE;&#x7F6E;&#x5206;&#x914D;&#x7ED9;&#x6BCF;&#x4E2A; executor &#x7684;&#x8D85;&#x8FC7; heap &#x5185;&#x5B58;&#x7684;&#x503C;&#xFF08;&#x53EF;&#x4EE5;&#x4F7F;&#x7528;k&#x3001;m&#x3001;g&#x5355;&#x4F4D;&#xFF09;&#x3002;&#x8BE5;&#x503C;&#x7528;&#x4E8E;&#x865A;&#x62DF;&#x673A;&#x7684;&#x5F00;&#x9500;&#x3001;&#x5176;&#x4ED6;&#x672C;&#x5730;&#x670D;&#x52A1;&#x5F00;&#x9500;&#x3002;&#x6839;&#x636E; executor &#x7684;&#x5927;&#x5C0F;&#x8BBE;&#x7F6E;&#xFF08;&#x901A;&#x5E38;&#x662F; 6%&#x5230;10%&#xFF09;&#x3002;</p>
<p>&#x6211;&#x4EEC;&#x53EF;&#x4EE5;&#x8FD9;&#x6837;&#x6765;&#x63D0;&#x4EA4;&#x4E00;&#x4E2A;&#x4EFB;&#x52A1;&#xFF0C;&#x540C;&#x65F6;&#x8BBE;&#x7F6E; driver &#x548C; executor &#x7684; CPU&#x3001;&#x5185;&#x5B58;&#x7684;&#x8D44;&#x6E90; request &#x548C; limit &#x503C;&#xFF08;driver &#x7684;&#x5185;&#x5B58; limit &#x503C;&#x4E3A; request &#x503C;&#x7684; 110%&#xFF09;&#x3002;</p>
<pre><code class="lang-bash">./spark-submit \
--deploy-mode cluster \
--class org.apache.spark.examples.SparkPi \
--master k8s://https://172.20.0.113:6443 \
--kubernetes-namespace spark-cluster \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.driver.memory=100G \
--conf spark.executor.memory=10G \
--conf spark.driver.cores=30 \
--conf spark.executor.cores=2 \
--conf spark.driver.maxResultSize=10240m \
--conf spark.kubernetes.driver.limit.cores=32 \
--conf spark.kubernetes.executor.limit.cores=3 \
--conf spark.kubernetes.executor.memoryOverhead=2g \
--conf spark.executor.instances=5 \
--conf spark.app.name=spark-pi \
--conf spark.kubernetes.driver.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.executor.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.initcontainer.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-init:v2.1.0-kubernetes-0.3.1-1 \
<span class="hljs-built_in">local</span>:///opt/spark/examples/jars/spark-examples_2.11-2.2.0-k8s-0.4.0-SNAPSHOT.jar 10000000
</code></pre>
<p>&#x8FD9;&#x5C06;&#x542F;&#x52A8;&#x4E00;&#x4E2A;&#x5305;&#x542B;&#x4E00;&#x5343;&#x4E07;&#x4E2A; task &#x7684;&#x8BA1;&#x7B97; pi &#x7684; spark &#x4EFB;&#x52A1;&#xFF0C;&#x4EFB;&#x52A1;&#x8FD0;&#x884C;&#x8FC7;&#x7A0B;&#x4E2D;&#xFF0C;drvier &#x7684; CPU &#x5B9E;&#x9645;&#x6D88;&#x8017;&#x5927;&#x7EA6;&#x4E3A; 3 &#x6838;&#xFF0C;&#x5185;&#x5B58; 40G&#xFF0C;&#x6BCF;&#x4E2A; executor &#x7684; CPU &#x5B9E;&#x9645;&#x6D88;&#x8017;&#x5927;&#x7EA6;&#x4E0D;&#x5230; 1 &#x6838;&#xFF0C;&#x5185;&#x5B58;&#x4E0D;&#x5230; 4G&#xFF0C;&#x6211;&#x4EEC;&#x53EF;&#x4EE5;&#x6839;&#x636E;&#x5B9E;&#x9645;&#x8D44;&#x6E90;&#x6D88;&#x8017;&#x4E0D;&#x65AD;&#x4F18;&#x5316;&#x8D44;&#x6E90;&#x7684; request &#x503C;&#x3002;</p>
<p><code>SPARK_DRIVER_MEMORY</code> &#x548C; <code>SPARK_EXECUTOR_MEMORY</code> &#x548C;&#x5206;&#x522B;&#x4F5C;&#x4E3A; Driver &#x5BB9;&#x5668;&#x548C; Executor &#x5BB9;&#x5668;&#x542F;&#x52A8;&#x7684;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#xFF0C;&#x6BD4;&#x5982;&#x4E0B;&#x9762;&#x8FD9;&#x4E2A; Driver &#x542F;&#x52A8;&#x7684; CMD &#x4E2D;&#xFF1A;</p>
<pre><code class="lang-bash">CMD SPARK_CLASSPATH=<span class="hljs-string">&quot;<span class="hljs-variable">${SPARK_HOME}</span>/jars/*&quot;</span> &amp;&amp; \
env | grep SPARK_JAVA_OPT_ | sed <span class="hljs-string">&apos;s/[^=]*=\(.*\)/\1/g&apos;</span> &gt; /tmp/java_opts.txt &amp;&amp; \
<span class="hljs-built_in">readarray</span> -t SPARK_DRIVER_JAVA_OPTS &lt; /tmp/java_opts.txt &amp;&amp; \
<span class="hljs-keyword">if</span> ! [ -z <span class="hljs-variable">${SPARK_MOUNTED_CLASSPATH+x}</span> ]; <span class="hljs-keyword">then</span> SPARK_CLASSPATH=<span class="hljs-string">&quot;<span class="hljs-variable">$SPARK_MOUNTED_CLASSPATH</span>:<span class="hljs-variable">$SPARK_CLASSPATH</span>&quot;</span>; <span class="hljs-keyword">fi</span> &amp;&amp; \
<span class="hljs-keyword">if</span> ! [ -z <span class="hljs-variable">${SPARK_SUBMIT_EXTRA_CLASSPATH+x}</span> ]; <span class="hljs-keyword">then</span> SPARK_CLASSPATH=<span class="hljs-string">&quot;<span class="hljs-variable">$SPARK_SUBMIT_EXTRA_CLASSPATH</span>:<span class="hljs-variable">$SPARK_CLASSPATH</span>&quot;</span>; <span class="hljs-keyword">fi</span> &amp;&amp; \
<span class="hljs-keyword">if</span> ! [ -z <span class="hljs-variable">${SPARK_EXTRA_CLASSPATH+x}</span> ]; <span class="hljs-keyword">then</span> SPARK_CLASSPATH=<span class="hljs-string">&quot;<span class="hljs-variable">$SPARK_EXTRA_CLASSPATH</span>:<span class="hljs-variable">$SPARK_CLASSPATH</span>&quot;</span>; <span class="hljs-keyword">fi</span> &amp;&amp; \
<span class="hljs-keyword">if</span> ! [ -z <span class="hljs-variable">${SPARK_MOUNTED_FILES_DIR+x}</span> ]; <span class="hljs-keyword">then</span> cp -R <span class="hljs-string">&quot;<span class="hljs-variable">$SPARK_MOUNTED_FILES_DIR</span>/.&quot;</span> .; <span class="hljs-keyword">fi</span> &amp;&amp; \
<span class="hljs-keyword">if</span> ! [ -z <span class="hljs-variable">${SPARK_MOUNTED_FILES_FROM_SECRET_DIR}</span> ]; <span class="hljs-keyword">then</span> cp -R <span class="hljs-string">&quot;<span class="hljs-variable">$SPARK_MOUNTED_FILES_FROM_SECRET_DIR</span>/.&quot;</span> .; <span class="hljs-keyword">fi</span> &amp;&amp; \
<span class="hljs-variable">${JAVA_HOME}</span>/bin/java <span class="hljs-string">&quot;<span class="hljs-variable">${SPARK_DRIVER_JAVA_OPTS[@]}</span>&quot;</span> -cp <span class="hljs-variable">$SPARK_CLASSPATH</span> -Xms<span class="hljs-variable">$SPARK_DRIVER_MEMORY</span> -Xmx<span class="hljs-variable">$SPARK_DRIVER_MEMORY</span> <span class="hljs-variable">$SPARK_DRIVER_CLASS</span> <span class="hljs-variable">$SPARK_DRIVER_ARGS</span>
</code></pre>
<p>&#x6211;&#x4EEC;&#x53EF;&#x4EE5;&#x770B;&#x5230;&#x5BF9; <code>SPARK_DRIVER_MEMORY</code> &#x73AF;&#x5883;&#x53D8;&#x91CF;&#x7684;&#x5F15;&#x7528;&#x3002;Executor &#x7684;&#x8BBE;&#x7F6E;&#x4E0E; driver &#x7C7B;&#x4F3C;&#x3002;</p>
<p>&#x800C;&#x6211;&#x4EEC;&#x53EF;&#x4EE5;&#x4F7F;&#x7528;&#x8FD9;&#x6837;&#x7684;&#x53C2;&#x6570;&#x6765;&#x4F20;&#x9012;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#x7684;&#x503C; <code>spark.executorEnv.[EnvironmentVariableName]</code>&#xFF0C;&#x53EA;&#x8981;&#x5C06; <code>EnvironmentVariableName</code> &#x66FF;&#x6362;&#x4E3A;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#x540D;&#x79F0;&#x5373;&#x53EF;&#x3002;</p>
<h2 id="&#x53C2;&#x8003;">&#x53C2;&#x8003;</h2>
<p><a href="http://lxw1234.com/archives/2015/12/593.htm" target="_blank">Spark&#x52A8;&#x6001;&#x8D44;&#x6E90;&#x5206;&#x914D;-Dynamic Resource Allocation</a></p>
<p><a href="https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html" target="_blank">Running Spark on Kubernetes</a></p>
<p><a href="https://issues.apache.org/jira/browse/SPARK-18278" target="_blank">Apache Spark Jira Issue - 18278 - SPIP: Support native submission of spark jobs to a kubernetes cluster</a></p>
<p><a href="https://github.com/kubernetes/kubernetes/issues/34377" target="_blank">Kubernetes Github Issue - 34377 Support Spark natively in Kubernetes</a></p>
<p><a href="https://github.com/kubernetes/kubernetes/tree/master/examples/spark" target="_blank">Kubernetes example spark</a></p>
<p><a href="https://github.com/rootsongjc/spark-on-kubernetes" target="_blank">https://github.com/rootsongjc/spark-on-kubernetes</a></p>
<p><a href="https://github.com/apache-spark-on-k8s/spark/blob/branch-2.2-kubernetes/resource-managers/kubernetes/architecture-docs/scheduler-backend.md" target="_blank">Scheduler backend</a></p>
<footer class="page-footer"><span class="copyright">Copyright &#xA9; jimmysong.io 2017 all right reserved&#xFF0C;powered by Gitbook</span><span class="footer-modification">Updated:
2017-09-27 21:03:00
</span></footer>
</section>
</div>
<div class="search-results">
<div class="has-results">
<h1 class="search-results-title"><span class='search-results-count'></span> results matching "<span class='search-query'></span>"</h1>
<ul class="search-results-list"></ul>
</div>
<div class="no-results">
<h1 class="search-results-title">No results matching "<span class='search-query'></span>"</h1>
</div>
</div>
</div>
</div>
</div>
</div>
<a href="spark-standalone-on-kubernetes.html" class="navigation navigation-prev " aria-label="Previous page: Spark standalone on Kubernetes">
<i class="fa fa-angle-left"></i>
</a>
<a href="serverless.html" class="navigation navigation-next " aria-label="Next page: Serverless架构">
<i class="fa fa-angle-right"></i>
</a>
</div>
<script>
var gitbook = gitbook || [];
gitbook.push(function() {
gitbook.page.hasChanged({"page":{"title":"运行支持Kubernetes原生调度的Spark程序","level":"6.4.2","depth":2,"next":{"title":"Serverless架构","level":"6.5","depth":1,"path":"usecases/serverless.md","ref":"usecases/serverless.md","articles":[{"title":"理解Serverless","level":"6.5.1","depth":2,"path":"usecases/understanding-serverless.md","ref":"usecases/understanding-serverless.md","articles":[]},{"title":"FaaS-函数即服务","level":"6.5.2","depth":2,"path":"usecases/faas.md","ref":"usecases/faas.md","articles":[{"title":"OpenFaaS快速入门指南","level":"6.5.2.1","depth":3,"path":"usecases/openfaas-quick-start.md","ref":"usecases/openfaas-quick-start.md","articles":[]}]}]},"previous":{"title":"Spark standalone on Kubernetes","level":"6.4.1","depth":2,"path":"usecases/spark-standalone-on-kubernetes.md","ref":"usecases/spark-standalone-on-kubernetes.md","articles":[]},"dir":"ltr"},"config":{"plugins":["github","codesnippet","splitter","page-toc-button","image-captions","editlink","back-to-top-button","-lunr","-search","search-plus","github-buttons@2.1.0","favicon@^0.0.2","tbfed-pagefooter@^0.0.1","3-ba","theme-default"],"styles":{"ebook":"styles/ebook.css","epub":"styles/epub.css","mobi":"styles/mobi.css","pdf":"styles/pdf.css","print":"styles/print.css","website":"styles/website.css"},"pluginsConfig":{"tbfed-pagefooter":{"copyright":"Copyright © jimmysong.io 2017","modify_label":"Updated:","modify_format":"YYYY-MM-DD HH:mm:ss"},"github":{"url":"https://github.com/rootsongjc/kubernetes-handbook"},"editlink":{"label":"编辑本页","multilingual":false,"base":"https://github.com/rootsongjc/kubernetes-handbook/blob/master/"},"splitter":{},"codesnippet":{},"fontsettings":{"theme":"white","family":"sans","size":2},"highlight":{},"favicon":{"shortcut":"favicon.ico","bookmark":"favicon.ico"},"page-toc-button":{},"back-to-top-button":{},"github-buttons":{"repo":"rootsongjc/kubernetes-handbook","types":["star"],"size":"small"},"3-ba":{"configuration":"auto","token":"11f7d254cfa4e0ca44b175c66d379ecc"},"sharing":{"facebook":true,"twitter":true,"google":false,"weibo":false,"instapaper":false,"vk":false,"all":["facebook","google","twitter","weibo","instapaper"]},"theme-default":{"showLevel":true,"styles":{"ebook":"styles/ebook.css","epub":"styles/epub.css","mobi":"styles/mobi.css","pdf":"styles/pdf.css","print":"styles/print.css","website":"styles/website.css"}},"search-plus":{},"image-captions":{"caption":"图片 - _CAPTION_","variable_name":"_pictures"}},"theme":"default","author":"Jimmy Song","pdf":{"pageNumbers":true,"fontSize":12,"fontFamily":"Arial","paperSize":"a4","chapterMark":"pagebreak","pageBreaksBefore":"/","margin":{"right":62,"left":62,"top":56,"bottom":56}},"structure":{"langs":"LANGS.md","readme":"README.md","glossary":"GLOSSARY.md","summary":"SUMMARY.md"},"variables":{"_pictures":[{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.1","level":"2.1","list_caption":"Figure: 云计算演进历程","alt":"云计算演进历程","nro":1,"url":"../images/cloud-computing-evolution-road.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"云计算演进历程","attributes":{},"skip":false,"key":"2.1.1"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.2","level":"2.1","list_caption":"Figure: 来自Twitter @MarcWilczek","alt":"来自Twitter @MarcWilczek","nro":2,"url":"../images/cloud-native-comes-of-age.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"来自Twitter @MarcWilczek","attributes":{},"skip":false,"key":"2.1.2"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.3","level":"2.1","list_caption":"Figure: Cloud native思维导图","alt":"Cloud native思维导图","nro":3,"url":"../images/cloud-native-architecutre-mindnode.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"Cloud native思维导图","attributes":{},"skip":false,"key":"2.1.3"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.4","level":"2.1","list_caption":"Figure: 十二因素应用","alt":"十二因素应用","nro":4,"url":"../images/12-factor-app.png","index":4,"caption_template":"图片 - _CAPTION_","label":"十二因素应用","attributes":{},"skip":false,"key":"2.1.4"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.5","level":"2.1","list_caption":"Figure: 容器生态","alt":"容器生态","nro":5,"url":"../images/container-ecosystem.png","index":5,"caption_template":"图片 - _CAPTION_","label":"容器生态","attributes":{},"skip":false,"key":"2.1.5"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.6","level":"2.1","list_caption":"Figure: 使用Jenkins进行持续集成与发布流程图","alt":"使用Jenkins进行持续集成与发布流程图","nro":6,"url":"../images/kubernetes-jenkins-ci-cd.png","index":6,"caption_template":"图片 - _CAPTION_","label":"使用Jenkins进行持续集成与发布流程图","attributes":{},"skip":false,"key":"2.1.6"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.7","level":"2.1","list_caption":"Figure: filebeat日志收集架构图","alt":"filebeat日志收集架构图","nro":7,"url":"../images/filebeat-log-collector-arch.png","index":7,"caption_template":"图片 - _CAPTION_","label":"filebeat日志收集架构图","attributes":{},"skip":false,"key":"2.1.7"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.8","level":"2.1","list_caption":"Figure: API文档","alt":"API文档","nro":8,"url":"../images/k8s-app-monitor-test-api-doc.jpg","index":8,"caption_template":"图片 - _CAPTION_","label":"API文档","attributes":{},"skip":false,"key":"2.1.8"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.9","level":"2.1","list_caption":"Figure: 迁移步骤示意图","alt":"迁移步骤示意图","nro":9,"url":"../images/migrating-hadoop-yarn-to-kubernetes.png","index":9,"caption_template":"图片 - _CAPTION_","label":"迁移步骤示意图","attributes":{},"skip":false,"key":"2.1.9"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.10","level":"2.1","list_caption":"Figure: service mesh架构图","alt":"service mesh架构图","nro":10,"url":"../images/serivce-mesh-control-plane.png","index":10,"caption_template":"图片 - _CAPTION_","label":"service mesh架构图","attributes":{},"skip":false,"key":"2.1.10"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.11","level":"2.1","list_caption":"Figure: kibana界面","alt":"kibana界面","nro":11,"url":"../images/filebeat-docker-test.jpg","index":11,"caption_template":"图片 - _CAPTION_","label":"kibana界面","attributes":{},"skip":false,"key":"2.1.11"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.12","level":"2.1","list_caption":"Figure: Grafana界面示意图1","alt":"Grafana界面示意图1","nro":12,"url":"../images/kubernetes-devops-example-grafana-1.png","index":12,"caption_template":"图片 - _CAPTION_","label":"Grafana界面示意图1","attributes":{},"skip":false,"key":"2.1.12"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.13","level":"2.1","list_caption":"Figure: Grafana界面示意图2","alt":"Grafana界面示意图2","nro":13,"url":"../images/kubernetes-devops-example-grafana-2.png","index":13,"caption_template":"图片 - _CAPTION_","label":"Grafana界面示意图2","attributes":{},"skip":false,"key":"2.1.13"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.14","level":"2.1","list_caption":"Figure: Grafana界面示意图3","alt":"Grafana界面示意图3","nro":14,"url":"../images/kubernetes-devops-example-grafana-3.png","index":14,"caption_template":"图片 - _CAPTION_","label":"Grafana界面示意图3","attributes":{},"skip":false,"key":"2.1.14"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.15","level":"2.1","list_caption":"Figure: dashboard","alt":"dashboard","nro":15,"url":"../images/spark-job-on-kubernetes-example-1.jpg","index":15,"caption_template":"图片 - _CAPTION_","label":"dashboard","attributes":{},"skip":false,"key":"2.1.15"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig2.1.16","level":"2.1","list_caption":"Figure: Grafana","alt":"Grafana","nro":16,"url":"../images/spark-job-on-kubernetes-example-2.jpg","index":16,"caption_template":"图片 - _CAPTION_","label":"Grafana","attributes":{},"skip":false,"key":"2.1.16"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.1","level":"2.2","list_caption":"Figure: Cloud Native容器实验室","alt":"Cloud Native容器实验室","nro":17,"url":"https://res.cloudinary.com/jimmysong/image/upload/images/cloud-native-container-lab.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Cloud Native容器实验室","attributes":{},"skip":false,"key":"2.2.1"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.2","level":"2.2","list_caption":"Figure: 容器生态图 Container ecosystem","alt":"容器生态图 Container ecosystem","nro":18,"url":"../images/container-ecosystem.png","index":2,"caption_template":"图片 - _CAPTION_","label":"容器生态图 Container ecosystem","attributes":{},"skip":false,"key":"2.2.2"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.3","level":"2.2","list_caption":"Figure: Cloud Native油井","alt":"Cloud Native油井","nro":19,"url":"https://res.cloudinary.com/jimmysong/image/upload/images/cloud-native-oil-well.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"Cloud Native油井","attributes":{},"skip":false,"key":"2.2.3"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.4","level":"2.2","list_caption":"Figure: Kuberentes架构","alt":"Kuberentes架构","nro":20,"url":"../images/kubernetes-high-level-component-archtecture.jpg","index":4,"caption_template":"图片 - _CAPTION_","label":"Kuberentes架构","attributes":{},"skip":false,"key":"2.2.4"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.5","level":"2.2","list_caption":"Figure: Cloud Native Core target","alt":"Cloud Native Core target","nro":21,"url":"../images/cloud-native-core-target.jpg","index":5,"caption_template":"图片 - _CAPTION_","label":"Cloud Native Core target","attributes":{},"skip":false,"key":"2.2.5"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.6","level":"2.2","list_caption":"Figure: FaaS Landscape","alt":"FaaS Landscape","nro":22,"url":"../images/redpoint-faas-landscape.jpg","index":6,"caption_template":"图片 - _CAPTION_","label":"FaaS Landscape","attributes":{},"skip":false,"key":"2.2.6"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.7","level":"2.2","list_caption":"Figure: Workloads running on Kubernetes","alt":"Workloads running on Kubernetes","nro":23,"url":"https://res.cloudinary.com/jimmysong/image/upload/images/workloads-running-on-kubernetes-2017-thenewstack.jpg","index":7,"caption_template":"图片 - _CAPTION_","label":"Workloads running on Kubernetes","attributes":{},"skip":false,"key":"2.2.7"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.8","level":"2.2","list_caption":"Figure: Gartner技术爆发趋势图2017","alt":"Gartner技术爆发趋势图2017","nro":24,"url":"https://res.cloudinary.com/jimmysong/image/upload/images/gartner-hype-cycle-for-emerging-technologies-2017.jpg","index":8,"caption_template":"图片 - _CAPTION_","label":"Gartner技术爆发趋势图2017","attributes":{},"skip":false,"key":"2.2.8"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.9","level":"2.2","list_caption":"Figure: Microservices concerns","alt":"Microservices concerns","nro":25,"url":"../images/microservices-concerns.jpg","index":9,"caption_template":"图片 - _CAPTION_","label":"Microservices concerns","attributes":{},"skip":false,"key":"2.2.9"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.10","level":"2.2","list_caption":"Figure: 两种服务发现方式","alt":"两种服务发现方式","nro":26,"url":"../images/service-discovery-in-microservices.png","index":10,"caption_template":"图片 - _CAPTION_","label":"两种服务发现方式","attributes":{},"skip":false,"key":"2.2.10"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.11","level":"2.2","list_caption":"Figure: Cloud Native Pipeline","alt":"Cloud Native Pipeline","nro":27,"url":"https://res.cloudinary.com/jimmysong/image/upload/images/cloud-natvie-pipeline.jpg","index":11,"caption_template":"图片 - _CAPTION_","label":"Cloud Native Pipeline","attributes":{},"skip":false,"key":"2.2.11"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.12","level":"2.2","list_caption":"Figure: Cloud Native Features","alt":"Cloud Native Features","nro":28,"url":"https://jimmysong.io/kubernetes-handbook/images/cloud-native-architecutre-mindnode.jpg","index":12,"caption_template":"图片 - _CAPTION_","label":"Cloud Native Features","attributes":{},"skip":false,"key":"2.2.12"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.13","level":"2.2","list_caption":"Figure: Cloud Native Landscape v1.0","alt":"Cloud Native Landscape v1.0","nro":29,"url":"https://raw.githubusercontent.com/cncf/landscape/master/landscape/CloudNativeLandscape_v1.0.jpg","index":13,"caption_template":"图片 - _CAPTION_","label":"Cloud Native Landscape v1.0","attributes":{},"skip":false,"key":"2.2.13"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.14","level":"2.2","list_caption":"Figure: Building a Cloud Native Architecture with Kubernetes followed 12 factor app","alt":"Building a Cloud Native Architecture with Kubernetes followed 12 factor app","nro":30,"url":"../images/building-cloud-native-architecture-with-kubernetes.png","index":14,"caption_template":"图片 - _CAPTION_","label":"Building a Cloud Native Architecture with Kubernetes followed 12 factor app","attributes":{},"skip":false,"key":"2.2.14"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.15","level":"2.2","list_caption":"Figure: Creating Kubernetes native app","alt":"Creating Kubernetes native app","nro":31,"url":"../images/creating-kubernetes-native-app.jpg","index":15,"caption_template":"图片 - _CAPTION_","label":"Creating Kubernetes native app","attributes":{},"skip":false,"key":"2.2.15"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.16","level":"2.2","list_caption":"Figure: Service Mesh中国社区slogan","alt":"Service Mesh中国社区slogan","nro":32,"url":"https://res.cloudinary.com/jimmysong/image/upload/images/service-meshes-pro.jpg","index":16,"caption_template":"图片 - _CAPTION_","label":"Service Mesh中国社区slogan","attributes":{},"skip":false,"key":"2.2.16"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.17","level":"2.2","list_caption":"Figure: istio vs linkerd","alt":"istio vs linkerd","nro":33,"url":"../images/istio-vs-linkerd.jpg","index":17,"caption_template":"图片 - _CAPTION_","label":"istio vs linkerd","attributes":{},"skip":false,"key":"2.2.17"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.18","level":"2.2","list_caption":"Figure: Cloud Native factory","alt":"Cloud Native factory","nro":34,"url":"https://res.cloudinary.com/jimmysong/image/upload/images/cloud-native-factory.jpg","index":18,"caption_template":"图片 - _CAPTION_","label":"Cloud Native factory","attributes":{},"skip":false,"key":"2.2.18"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.19","level":"2.2","list_caption":"Figure: Deployment pipeline","alt":"Deployment pipeline","nro":35,"url":"https://res.cloudinary.com/jimmysong/image/upload/images/deployment-pipeline-comic.jpg","index":19,"caption_template":"图片 - _CAPTION_","label":"Deployment pipeline","attributes":{},"skip":false,"key":"2.2.19"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.20","level":"2.2","list_caption":"Figure: Spark on Kubernetes with different schedulers","alt":"Spark on Kubernetes with different schedulers","nro":36,"url":"../images/spark-on-kubernetes-with-different-schedulers.jpg","index":20,"caption_template":"图片 - _CAPTION_","label":"Spark on Kubernetes with different schedulers","attributes":{},"skip":false,"key":"2.2.20"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.21","level":"2.2","list_caption":"Figure: Kubernetes solutions","alt":"Kubernetes solutions","nro":37,"url":"https://res.cloudinary.com/jimmysong/image/upload/images/kubernetes-solutions-choices.jpg","index":21,"caption_template":"图片 - _CAPTION_","label":"Kubernetes solutions","attributes":{},"skip":false,"key":"2.2.21"},{"backlink":"cloud-native/from-kubernetes-to-cloud-native.html#fig2.2.22","level":"2.2","list_caption":"Figure: Kubernetes SIG","alt":"Kubernetes SIG","nro":38,"url":"../images/kubernetes-sigs.jpg","index":22,"caption_template":"图片 - _CAPTION_","label":"Kubernetes SIG","attributes":{},"skip":false,"key":"2.2.22"},{"backlink":"concepts/index.html#fig3.1.1","level":"3.1","list_caption":"Figure: Borg架构","alt":"Borg架构","nro":39,"url":"../images/borg.png","index":1,"caption_template":"图片 - _CAPTION_","label":"Borg架构","attributes":{},"skip":false,"key":"3.1.1"},{"backlink":"concepts/index.html#fig3.1.2","level":"3.1","list_caption":"Figure: Kubernetes架构","alt":"Kubernetes架构","nro":40,"url":"../images/architecture.png","index":2,"caption_template":"图片 - _CAPTION_","label":"Kubernetes架构","attributes":{},"skip":false,"key":"3.1.2"},{"backlink":"concepts/index.html#fig3.1.3","level":"3.1","list_caption":"Figure: Kuberentes架构图片来自于网络","alt":"Kuberentes架构图片来自于网络","nro":41,"url":"../images/kubernetes-high-level-component-archtecture.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"Kuberentes架构图片来自于网络","attributes":{},"skip":false,"key":"3.1.3"},{"backlink":"concepts/index.html#fig3.1.4","level":"3.1","list_caption":"Figure: kubernetes整体架构示意图","alt":"kubernetes整体架构示意图","nro":42,"url":"../images/kubernetes-whole-arch.png","index":4,"caption_template":"图片 - _CAPTION_","label":"kubernetes整体架构示意图","attributes":{},"skip":false,"key":"3.1.4"},{"backlink":"concepts/index.html#fig3.1.5","level":"3.1","list_caption":"Figure: Kubernetes master架构示意图","alt":"Kubernetes master架构示意图","nro":43,"url":"../images/kubernetes-master-arch.png","index":5,"caption_template":"图片 - _CAPTION_","label":"Kubernetes master架构示意图","attributes":{},"skip":false,"key":"3.1.5"},{"backlink":"concepts/index.html#fig3.1.6","level":"3.1","list_caption":"Figure: kubernetes node架构示意图","alt":"kubernetes node架构示意图","nro":44,"url":"../images/kubernetes-node-arch.png","index":6,"caption_template":"图片 - _CAPTION_","label":"kubernetes node架构示意图","attributes":{},"skip":false,"key":"3.1.6"},{"backlink":"concepts/index.html#fig3.1.7","level":"3.1","list_caption":"Figure: Kubernetes分层架构示意图","alt":"Kubernetes分层架构示意图","nro":45,"url":"../images/kubernetes-layers-arch.jpg","index":7,"caption_template":"图片 - _CAPTION_","label":"Kubernetes分层架构示意图","attributes":{},"skip":false,"key":"3.1.7"},{"backlink":"concepts/concepts.html#fig3.2.1","level":"3.2","list_caption":"Figure: 分层架构示意图","alt":"分层架构示意图","nro":46,"url":"../images/kubernetes-layers-arch.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"分层架构示意图","attributes":{},"skip":false,"key":"3.2.1"},{"backlink":"concepts/pod-overview.html#fig3.3.1.1","level":"3.3.1","list_caption":"Figure: pod diagram","alt":"pod diagram","nro":47,"url":"../images/pod-overview.png","index":1,"caption_template":"图片 - _CAPTION_","label":"pod diagram","attributes":{},"skip":false,"key":"3.3.1.1"},{"backlink":"concepts/pod.html#fig3.3.1.1.1","level":"3.3.1.1","list_caption":"Figure: Pod示意图","alt":"Pod示意图","nro":48,"url":"../images/pod-overview.png","index":1,"caption_template":"图片 - _CAPTION_","label":"Pod示意图","attributes":{},"skip":false,"key":"3.3.1.1.1"},{"backlink":"concepts/pod.html#fig3.3.1.1.2","level":"3.3.1.1","list_caption":"Figure: Pod Cheatsheet","alt":"Pod Cheatsheet","nro":49,"url":"../images/kubernetes-pod-cheatsheet.png","index":2,"caption_template":"图片 - _CAPTION_","label":"Pod Cheatsheet","attributes":{},"skip":false,"key":"3.3.1.1.2"},{"backlink":"concepts/service.html#fig3.3.4.1","level":"3.3.4","list_caption":"Figure: userspace代理模式下Service概览图","alt":"userspace代理模式下Service概览图","nro":50,"url":"../images/services-userspace-overview.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"userspace代理模式下Service概览图","attributes":{},"skip":false,"key":"3.3.4.1"},{"backlink":"concepts/service.html#fig3.3.4.2","level":"3.3.4","list_caption":"Figure: iptables代理模式下Service概览图","alt":"iptables代理模式下Service概览图","nro":51,"url":"../images/services-iptables-overview.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"iptables代理模式下Service概览图","attributes":{},"skip":false,"key":"3.3.4.2"},{"backlink":"concepts/deployment.html#fig3.3.6.1","level":"3.3.6","list_caption":"Figure: kubernetes deployment cheatsheet","alt":"kubernetes deployment cheatsheet","nro":52,"url":"../images/deployment-cheatsheet.png","index":1,"caption_template":"图片 - _CAPTION_","label":"kubernetes deployment cheatsheet","attributes":{},"skip":false,"key":"3.3.6.1"},{"backlink":"concepts/horizontal-pod-autoscaling.html#fig3.3.16.1","level":"3.3.16","list_caption":"Figure: horizontal-pod-autoscaler","alt":"horizontal-pod-autoscaler","nro":53,"url":"../images/horizontal-pod-autoscaler.png","index":1,"caption_template":"图片 - _CAPTION_","label":"horizontal-pod-autoscaler","attributes":{},"skip":false,"key":"3.3.16.1"},{"backlink":"concepts/label.html#fig3.3.17.1","level":"3.3.17","list_caption":"Figure: label示意图","alt":"label示意图","nro":54,"url":"../images/labels.png","index":1,"caption_template":"图片 - _CAPTION_","label":"label示意图","attributes":{},"skip":false,"key":"3.3.17.1"},{"backlink":"guide/using-kubectl.html#fig4.3.2.1","level":"4.3.2","list_caption":"Figure: kubectl cheatsheet","alt":"kubectl cheatsheet","nro":55,"url":"../images/kubernetes-kubectl-cheatsheet.png","index":1,"caption_template":"图片 - _CAPTION_","label":"kubectl cheatsheet","attributes":{},"skip":false,"key":"4.3.2.1"},{"backlink":"guide/using-kubectl.html#fig4.3.2.2","level":"4.3.2","list_caption":"Figure: kube-shell页面","alt":"kube-shell页面","nro":56,"url":"../images/kube-shell.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"kube-shell页面","attributes":{},"skip":false,"key":"4.3.2.2"},{"backlink":"guide/ip-masq-agent.html#fig4.4.6.1","level":"4.4.6","list_caption":"Figure: IP伪装代理示意图","alt":"IP伪装代理示意图","nro":57,"url":"../images/ip-masq.png","index":1,"caption_template":"图片 - _CAPTION_","label":"IP伪装代理示意图","attributes":{},"skip":false,"key":"4.4.6.1"},{"backlink":"guide/auth-with-kubeconfig-or-token.html#fig4.4.7.1","level":"4.4.7","list_caption":"Figure: kubeconfig文件","alt":"kubeconfig文件","nro":58,"url":"../images/brand-kubeconfig-yaml.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"kubeconfig文件","attributes":{},"skip":false,"key":"4.4.7.1"},{"backlink":"guide/authentication.html#fig4.4.8.1","level":"4.4.8","list_caption":"Figure: Kubernetes OpenID Connect Flow","alt":"Kubernetes OpenID Connect Flow","nro":59,"url":"../images/kubernetes-oidc-login.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Kubernetes OpenID Connect Flow","attributes":{},"skip":false,"key":"4.4.8.1"},{"backlink":"guide/carbin-mobile-dashboard-for-kubernetes.html#fig4.5.6.1","level":"4.5.6","list_caption":"Figure: App Store","alt":"App Store","nro":60,"url":"../images/carbin-kubernetes-mobile-dashboard-1.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"App Store","attributes":{},"skip":false,"key":"4.5.6.1"},{"backlink":"guide/carbin-mobile-dashboard-for-kubernetes.html#fig4.5.6.2","level":"4.5.6","list_caption":"Figure: 在手机上操作Kubernetes集群","alt":"在手机上操作Kubernetes集群","nro":61,"url":"../images/carbin-kubernetes-mobile-dashboard-4.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"在手机上操作Kubernetes集群","attributes":{},"skip":false,"key":"4.5.6.2"},{"backlink":"guide/kubernetes-desktop-client.html#fig4.5.7.1","level":"4.5.7","list_caption":"Figure: Kubernetic客户端","alt":"Kubernetic客户端","nro":62,"url":"../images/kubernetic-desktop-ui.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Kubernetic客户端","attributes":{},"skip":false,"key":"4.5.7.1"},{"backlink":"guide/kubernator-kubernetes-ui.html#fig4.5.8.1","level":"4.5.8","list_caption":"Figure: Kubernator catalog页面","alt":"Kubernator catalog页面","nro":63,"url":"../images/kubernator-catalog.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Kubernator catalog页面","attributes":{},"skip":false,"key":"4.5.8.1"},{"backlink":"guide/kubernator-kubernetes-ui.html#fig4.5.8.2","level":"4.5.8","list_caption":"Figure: Kubernator rbac页面","alt":"Kubernator rbac页面","nro":64,"url":"../images/kubernator-rbac.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"Kubernator rbac页面","attributes":{},"skip":false,"key":"4.5.8.2"},{"backlink":"guide/deploy-applications-in-kubernetes.html#fig4.6.1.1","level":"4.6.1","list_caption":"Figure: API","alt":"API","nro":65,"url":"../images/k8s-app-monitor-test-api-doc.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"API","attributes":{},"skip":false,"key":"4.6.1.1"},{"backlink":"guide/deploy-applications-in-kubernetes.html#fig4.6.1.2","level":"4.6.1","list_caption":"Figure: wercker","alt":"wercker","nro":66,"url":"../images/k8s-app-monitor-agent-wercker.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"wercker","attributes":{},"skip":false,"key":"4.6.1.2"},{"backlink":"guide/deploy-applications-in-kubernetes.html#fig4.6.1.3","level":"4.6.1","list_caption":"Figure: 图表","alt":"图表","nro":67,"url":"../images/k8s-app-monitor-agent.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"图表","attributes":{},"skip":false,"key":"4.6.1.3"},{"backlink":"guide/migrating-hadoop-yarn-to-kubernetes.html#fig4.6.2.1","level":"4.6.2","list_caption":"Figure: spark on yarn with kubernetes","alt":"spark on yarn with kubernetes","nro":68,"url":"../images/spark-on-yarn-with-kubernetes.png","index":1,"caption_template":"图片 - _CAPTION_","label":"spark on yarn with kubernetes","attributes":{},"skip":false,"key":"4.6.2.1"},{"backlink":"guide/migrating-hadoop-yarn-to-kubernetes.html#fig4.6.2.2","level":"4.6.2","list_caption":"Figure: Terms","alt":"Terms","nro":69,"url":"../images/terms-in-kubernetes-app-deployment.png","index":2,"caption_template":"图片 - _CAPTION_","label":"Terms","attributes":{},"skip":false,"key":"4.6.2.2"},{"backlink":"guide/migrating-hadoop-yarn-to-kubernetes.html#fig4.6.2.3","level":"4.6.2","list_caption":"Figure: 分解步骤解析","alt":"分解步骤解析","nro":70,"url":"../images/migrating-hadoop-yarn-to-kubernetes.png","index":3,"caption_template":"图片 - _CAPTION_","label":"分解步骤解析","attributes":{},"skip":false,"key":"4.6.2.3"},{"backlink":"practice/node-installation.html#fig5.2.7.1","level":"5.2.7","list_caption":"Figure: welcome nginx","alt":"welcome nginx","nro":71,"url":"../images/kubernetes-installation-test-nginx.png","index":1,"caption_template":"图片 - _CAPTION_","label":"welcome nginx","attributes":{},"skip":false,"key":"5.2.7.1"},{"backlink":"practice/dashboard-addon-installation.html#fig5.2.9.1","level":"5.2.9","list_caption":"Figure: kubernetes dashboard","alt":"kubernetes dashboard","nro":72,"url":"../images/kubernetes-dashboard-raw.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"kubernetes dashboard","attributes":{},"skip":false,"key":"5.2.9.1"},{"backlink":"practice/dashboard-addon-installation.html#fig5.2.9.2","level":"5.2.9","list_caption":"Figure: V1.6.3版本的dashboard界面","alt":"V1.6.3版本的dashboard界面","nro":73,"url":"../images/dashboard-v163.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"V1.6.3版本的dashboard界面","attributes":{},"skip":false,"key":"5.2.9.2"},{"backlink":"practice/heapster-addon-installation.html#fig5.2.10.1","level":"5.2.10","list_caption":"Figure: dashboard-heapster","alt":"dashboard-heapster","nro":74,"url":"../images/kubernetes-dashboard-with-heapster.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"dashboard-heapster","attributes":{},"skip":false,"key":"5.2.10.1"},{"backlink":"practice/heapster-addon-installation.html#fig5.2.10.2","level":"5.2.10","list_caption":"Figure: grafana","alt":"grafana","nro":75,"url":"../images/kubernetes-heapster-grafana.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"grafana","attributes":{},"skip":false,"key":"5.2.10.2"},{"backlink":"practice/heapster-addon-installation.html#fig5.2.10.3","level":"5.2.10","list_caption":"Figure: kubernetes-influxdb-heapster","alt":"kubernetes-influxdb-heapster","nro":76,"url":"../images/kubernetes-influxdb-heapster.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"kubernetes-influxdb-heapster","attributes":{},"skip":false,"key":"5.2.10.3"},{"backlink":"practice/heapster-addon-installation.html#fig5.2.10.4","level":"5.2.10","list_caption":"Figure: 修改grafana模板","alt":"修改grafana模板","nro":77,"url":"../images/grafana-dashboard-setting.jpg","index":4,"caption_template":"图片 - _CAPTION_","label":"修改grafana模板","attributes":{},"skip":false,"key":"5.2.10.4"},{"backlink":"practice/efk-addon-installation.html#fig5.2.11.1","level":"5.2.11","list_caption":"Figure: es-setting","alt":"es-setting","nro":78,"url":"../images/es-setting.png","index":1,"caption_template":"图片 - _CAPTION_","label":"es-setting","attributes":{},"skip":false,"key":"5.2.11.1"},{"backlink":"practice/efk-addon-installation.html#fig5.2.11.2","level":"5.2.11","list_caption":"Figure: es-home","alt":"es-home","nro":79,"url":"../images/kubernetes-efk-kibana.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"es-home","attributes":{},"skip":false,"key":"5.2.11.2"},{"backlink":"practice/traefik-ingress-installation.html#fig5.3.1.1","level":"5.3.1","list_caption":"Figure: kubernetes-dashboard","alt":"kubernetes-dashboard","nro":80,"url":"../images/traefik-dashboard.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"kubernetes-dashboard","attributes":{},"skip":false,"key":"5.3.1.1"},{"backlink":"practice/traefik-ingress-installation.html#fig5.3.1.2","level":"5.3.1","list_caption":"Figure: traefik-nginx","alt":"traefik-nginx","nro":81,"url":"../images/traefik-nginx.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"traefik-nginx","attributes":{},"skip":false,"key":"5.3.1.2"},{"backlink":"practice/traefik-ingress-installation.html#fig5.3.1.3","level":"5.3.1","list_caption":"Figure: traefik-guestbook","alt":"traefik-guestbook","nro":82,"url":"../images/traefik-guestbook.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"traefik-guestbook","attributes":{},"skip":false,"key":"5.3.1.3"},{"backlink":"practice/distributed-load-test.html#fig5.3.2.1","level":"5.3.2","list_caption":"Figure: 使用dashboard来扩容","alt":"使用dashboard来扩容","nro":83,"url":"../images/dashbaord-scale.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"使用dashboard来扩容","attributes":{},"skip":false,"key":"5.3.2.1"},{"backlink":"practice/distributed-load-test.html#fig5.3.2.2","level":"5.3.2","list_caption":"Figure: Traefik的UI","alt":"Traefik的UI","nro":84,"url":"../images/traefik-dashboard-locust.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"Traefik的UI","attributes":{},"skip":false,"key":"5.3.2.2"},{"backlink":"practice/distributed-load-test.html#fig5.3.2.3","level":"5.3.2","list_caption":"Figure: Locust启动界面","alt":"Locust启动界面","nro":85,"url":"../images/locust-start-swarming.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"Locust启动界面","attributes":{},"skip":false,"key":"5.3.2.3"},{"backlink":"practice/distributed-load-test.html#fig5.3.2.4","level":"5.3.2","list_caption":"Figure: Dashboard查看页面","alt":"Dashboard查看页面","nro":86,"url":"../images/sample-webapp-rc.jpg","index":4,"caption_template":"图片 - _CAPTION_","label":"Dashboard查看页面","attributes":{},"skip":false,"key":"5.3.2.4"},{"backlink":"practice/distributed-load-test.html#fig5.3.2.5","level":"5.3.2","list_caption":"Figure: Locust测试结果页面","alt":"Locust测试结果页面","nro":87,"url":"../images/locust-dashboard.jpg","index":5,"caption_template":"图片 - _CAPTION_","label":"Locust测试结果页面","attributes":{},"skip":false,"key":"5.3.2.5"},{"backlink":"practice/network-and-cluster-perfermance-test.html#fig5.3.3.1","level":"5.3.3","list_caption":"Figure: kubernetes-dashboard","alt":"kubernetes-dashboard","nro":88,"url":"../images/kubenetes-e2e-test.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"kubernetes-dashboard","attributes":{},"skip":false,"key":"5.3.3.1"},{"backlink":"practice/network-and-cluster-perfermance-test.html#fig5.3.3.2","level":"5.3.3","list_caption":"Figure: locust测试页面","alt":"locust测试页面","nro":89,"url":"../images/kubernetes-locust-test.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"locust测试页面","attributes":{},"skip":false,"key":"5.3.3.2"},{"backlink":"practice/edge-node-configuration.html#fig5.3.4.1","level":"5.3.4","list_caption":"Figure: 边缘节点架构","alt":"边缘节点架构","nro":90,"url":"../images/kubernetes-edge-node-architecture.png","index":1,"caption_template":"图片 - _CAPTION_","label":"边缘节点架构","attributes":{},"skip":false,"key":"5.3.4.1"},{"backlink":"practice/app-log-collection.html#fig5.4.2.1","level":"5.4.2","list_caption":"Figure: filebeat日志收集架构图","alt":"filebeat日志收集架构图","nro":91,"url":"../images/filebeat-log-collector.png","index":1,"caption_template":"图片 - _CAPTION_","label":"filebeat日志收集架构图","attributes":{},"skip":false,"key":"5.4.2.1"},{"backlink":"practice/app-log-collection.html#fig5.4.2.2","level":"5.4.2","list_caption":"Figure: Kibana页面","alt":"Kibana页面","nro":92,"url":"../images/filebeat-docker-test.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"Kibana页面","attributes":{},"skip":false,"key":"5.4.2.2"},{"backlink":"practice/app-log-collection.html#fig5.4.2.3","level":"5.4.2","list_caption":"Figure: filebeat收集的日志详细信息","alt":"filebeat收集的日志详细信息","nro":93,"url":"../images/kubernetes-filebeat-detail.png","index":3,"caption_template":"图片 - _CAPTION_","label":"filebeat收集的日志详细信息","attributes":{},"skip":false,"key":"5.4.2.3"},{"backlink":"practice/monitor.html#fig5.4.4.1","level":"5.4.4","list_caption":"Figure: Kubernetes集群中的监控","alt":"Kubernetes集群中的监控","nro":94,"url":"../images/monitoring-in-kubernetes.png","index":1,"caption_template":"图片 - _CAPTION_","label":"Kubernetes集群中的监控","attributes":{},"skip":false,"key":"5.4.4.1"},{"backlink":"practice/monitor.html#fig5.4.4.2","level":"5.4.4","list_caption":"Figure: kubernetes的容器命名规则示意图","alt":"kubernetes的容器命名规则示意图","nro":95,"url":"../images/kubernetes-container-naming-rule.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"kubernetes的容器命名规则示意图","attributes":{},"skip":false,"key":"5.4.4.2"},{"backlink":"practice/monitor.html#fig5.4.4.3","level":"5.4.4","list_caption":"Figure: Heapster架构图改进版","alt":"Heapster架构图改进版","nro":96,"url":"../images/kubernetes-heapster-monitoring.png","index":3,"caption_template":"图片 - _CAPTION_","label":"Heapster架构图改进版","attributes":{},"skip":false,"key":"5.4.4.3"},{"backlink":"practice/monitor.html#fig5.4.4.4","level":"5.4.4","list_caption":"Figure: 应用监控架构图","alt":"应用监控架构图","nro":97,"url":"../images/kubernetes-app-monitoring.png","index":4,"caption_template":"图片 - _CAPTION_","label":"应用监控架构图","attributes":{},"skip":false,"key":"5.4.4.4"},{"backlink":"practice/monitor.html#fig5.4.4.5","level":"5.4.4","list_caption":"Figure: 应用拓扑图","alt":"应用拓扑图","nro":98,"url":"../images/weave-scope-service-topology.jpg","index":5,"caption_template":"图片 - _CAPTION_","label":"应用拓扑图","attributes":{},"skip":false,"key":"5.4.4.5"},{"backlink":"practice/data-persistence-problem.html#fig5.4.5.1","level":"5.4.5","list_caption":"Figure: 日志持久化收集解决方案示意图","alt":"日志持久化收集解决方案示意图","nro":99,"url":"../images/log-persistence-logstash.png","index":1,"caption_template":"图片 - _CAPTION_","label":"日志持久化收集解决方案示意图","attributes":{},"skip":false,"key":"5.4.5.1"},{"backlink":"practice/storage-for-containers-using-glusterfs-with-openshift.html#fig5.5.1.2.1","level":"5.5.1.2","list_caption":"Figure: 创建存储","alt":"创建存储","nro":100,"url":"../images/create-gluster-storage.png","index":1,"caption_template":"图片 - _CAPTION_","label":"创建存储","attributes":{},"skip":false,"key":"5.5.1.2.1"},{"backlink":"practice/storage-for-containers-using-glusterfs-with-openshift.html#fig5.5.1.2.2","level":"5.5.1.2","list_caption":"Figure: Screen Shot 2017-03-24 at 11.09.34.png","alt":"Screen Shot 2017-03-24 at 11.09.34.png","nro":101,"url":"https://keithtenzer.files.wordpress.com/2017/03/screen-shot-2017-03-24-at-11-09-341.png?w=440","index":2,"caption_template":"图片 - _CAPTION_","label":"Screen Shot 2017-03-24 at 11.09.34.png","attributes":{},"skip":false,"key":"5.5.1.2.2"},{"backlink":"practice/using-heapster-to-get-object-metrics.html#fig5.6.1.1.1","level":"5.6.1.1","list_caption":"Figure: Heapster架构图","alt":"Heapster架构图","nro":102,"url":"../images/heapster-architecture.png","index":1,"caption_template":"图片 - _CAPTION_","label":"Heapster架构图","attributes":{},"skip":false,"key":"5.6.1.1.1"},{"backlink":"practice/prometheus.html#fig5.6.2.1","level":"5.6.2","list_caption":"Figure: Prometheus 架构图","alt":"Prometheus 架构图","nro":103,"url":"../images/prometheus-architecture.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Prometheus 架构图","attributes":{},"skip":false,"key":"5.6.2.1"},{"backlink":"practice/using-prometheus-to-monitor-kuberentes-cluster.html#fig5.6.2.1.1","level":"5.6.2.1","list_caption":"Figure: Grafana页面","alt":"Grafana页面","nro":104,"url":"../images/kubernetes-prometheus-monitoring.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Grafana页面","attributes":{},"skip":false,"key":"5.6.2.1.1"},{"backlink":"practice/helm.html#fig5.7.1.1","level":"5.7.1","list_caption":"Figure: Helm chart源","alt":"Helm chart源","nro":105,"url":"../images/helm-charts-repository.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Helm chart源","attributes":{},"skip":false,"key":"5.7.1.1"},{"backlink":"practice/helm.html#fig5.7.1.2","level":"5.7.1","list_caption":"Figure: TODO应用的Web页面","alt":"TODO应用的Web页面","nro":106,"url":"../images/helm-mean-todo-aholic.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"TODO应用的Web页面","attributes":{},"skip":false,"key":"5.7.1.2"},{"backlink":"practice/create-private-charts-repo.html#fig5.7.2.1","level":"5.7.2","list_caption":"Figure: Helm monocular界面","alt":"Helm monocular界面","nro":107,"url":"../images/helm-monocular-jimmysong.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Helm monocular界面","attributes":{},"skip":false,"key":"5.7.2.1"},{"backlink":"practice/jenkins-ci-cd.html#fig5.8.1.1","level":"5.8.1","list_caption":"Figure: 基于Jenkins的持续集成与发布","alt":"基于Jenkins的持续集成与发布","nro":108,"url":"../images/kubernetes-jenkins-ci-cd.png","index":1,"caption_template":"图片 - _CAPTION_","label":"基于Jenkins的持续集成与发布","attributes":{},"skip":false,"key":"5.8.1.1"},{"backlink":"practice/drone-ci-cd.html#fig5.8.2.1","level":"5.8.2","list_caption":"Figure: OAuth注册","alt":"OAuth注册","nro":109,"url":"../images/github-oauth-register.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"OAuth注册","attributes":{},"skip":false,"key":"5.8.2.1"},{"backlink":"practice/drone-ci-cd.html#fig5.8.2.2","level":"5.8.2","list_caption":"Figure: OAuth key","alt":"OAuth key","nro":110,"url":"../images/github-oauth-drone-key.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"OAuth key","attributes":{},"skip":false,"key":"5.8.2.2"},{"backlink":"practice/drone-ci-cd.html#fig5.8.2.3","level":"5.8.2","list_caption":"Figure: Drone登陆界面","alt":"Drone登陆界面","nro":111,"url":"../images/drone-login-github.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"Drone登陆界面","attributes":{},"skip":false,"key":"5.8.2.3"},{"backlink":"practice/drone-ci-cd.html#fig5.8.2.4","level":"5.8.2","list_caption":"Figure: Github启用repo设置","alt":"Github启用repo设置","nro":112,"url":"../images/drone-github-active.jpg","index":4,"caption_template":"图片 - _CAPTION_","label":"Github启用repo设置","attributes":{},"skip":false,"key":"5.8.2.4"},{"backlink":"practice/drone-ci-cd.html#fig5.8.2.5","level":"5.8.2","list_caption":"Figure: Github单个repo设置","alt":"Github单个repo设置","nro":113,"url":"../images/drone-github-repo-setting.jpg","index":5,"caption_template":"图片 - _CAPTION_","label":"Github单个repo设置","attributes":{},"skip":false,"key":"5.8.2.5"},{"backlink":"practice/manually-upgrade.html#fig5.9.1.1","level":"5.9.1","list_caption":"Figure: Kubernetes零宕机时间升级建议","alt":"Kubernetes零宕机时间升级建议","nro":114,"url":"../images/zero-downtime-kubernetes-upgrade-tips.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Kubernetes零宕机时间升级建议","attributes":{},"skip":false,"key":"5.9.1.1"},{"backlink":"practice/dashboard-upgrade.html#fig5.9.2.1","level":"5.9.2","list_caption":"Figure: 登陆界面","alt":"登陆界面","nro":115,"url":"../images/kubernetes-dashboard-1.7.1-login.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"登陆界面","attributes":{},"skip":false,"key":"5.9.2.1"},{"backlink":"practice/dashboard-upgrade.html#fig5.9.2.2","level":"5.9.2","list_caption":"Figure: 首页","alt":"首页","nro":116,"url":"../images/kubernetes-dashboard-1.7.1-default-page.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"首页","attributes":{},"skip":false,"key":"5.9.2.2"},{"backlink":"practice/dashboard-upgrade.html#fig5.9.2.3","level":"5.9.2","list_caption":"Figure: 用户空间","alt":"用户空间","nro":117,"url":"../images/kubernetes-dashboard-1.7.1-brand.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"用户空间","attributes":{},"skip":false,"key":"5.9.2.3"},{"backlink":"practice/dashboard-upgrade.html#fig5.9.2.4","level":"5.9.2","list_caption":"Figure: kubeconfig文件","alt":"kubeconfig文件","nro":118,"url":"../images/brand-kubeconfig-yaml.jpg","index":4,"caption_template":"图片 - _CAPTION_","label":"kubeconfig文件","attributes":{},"skip":false,"key":"5.9.2.4"},{"backlink":"usecases/microservices.html#fig6.2.1","level":"6.2","list_caption":"Figure: 微服务关注的部分","alt":"微服务关注的部分","nro":119,"url":"../images/microservices-concerns.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"微服务关注的部分","attributes":{},"skip":false,"key":"6.2.1"},{"backlink":"usecases/service-discovery-in-microservices.html#fig6.2.1.1","level":"6.2.1","list_caption":"Figure: 微服务中的服务发现","alt":"微服务中的服务发现","nro":120,"url":"../images/service-discovery-in-microservices.png","index":1,"caption_template":"图片 - _CAPTION_","label":"微服务中的服务发现","attributes":{},"skip":false,"key":"6.2.1.1"},{"backlink":"usecases/service-mesh.html#fig6.3.1","level":"6.3","list_caption":"Figure: 下一代异构微服务架构","alt":"下一代异构微服务架构","nro":121,"url":"../images/polyglot-microservices-serivce-mesh.png","index":1,"caption_template":"图片 - _CAPTION_","label":"下一代异构微服务架构","attributes":{},"skip":false,"key":"6.3.1"},{"backlink":"usecases/service-mesh.html#fig6.3.2","level":"6.3","list_caption":"Figure: Service Mesh 架构图","alt":"Service Mesh 架构图","nro":122,"url":"../images/serivce-mesh-control-plane.png","index":2,"caption_template":"图片 - _CAPTION_","label":"Service Mesh 架构图","attributes":{},"skip":false,"key":"6.3.2"},{"backlink":"usecases/service-mesh.html#fig6.3.3","level":"6.3","list_caption":"Figure: Istio vs linkerd","alt":"Istio vs linkerd","nro":123,"url":"../images/istio-vs-linkerd.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"Istio vs linkerd","attributes":{},"skip":false,"key":"6.3.3"},{"backlink":"usecases/istio.html#fig6.3.1.1","level":"6.3.1","list_caption":"Figure: Istio架构图","alt":"Istio架构图","nro":124,"url":"../images/istio-arch.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Istio架构图","attributes":{},"skip":false,"key":"6.3.1.1"},{"backlink":"usecases/istio-installation.html#fig6.3.1.1.1","level":"6.3.1.1","list_caption":"Figure: BookInfo Sample应用架构图","alt":"BookInfo Sample应用架构图","nro":125,"url":"../images/bookinfo-sample-arch.png","index":1,"caption_template":"图片 - _CAPTION_","label":"BookInfo Sample应用架构图","attributes":{},"skip":false,"key":"6.3.1.1.1"},{"backlink":"usecases/istio-installation.html#fig6.3.1.1.2","level":"6.3.1.1","list_caption":"Figure: BookInfo Sample页面","alt":"BookInfo Sample页面","nro":126,"url":"../images/bookinfo-sample.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"BookInfo Sample页面","attributes":{},"skip":false,"key":"6.3.1.1.2"},{"backlink":"usecases/istio-installation.html#fig6.3.1.1.3","level":"6.3.1.1","list_caption":"Figure: Istio Grafana界面","alt":"Istio Grafana界面","nro":127,"url":"../images/istio-grafana.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"Istio Grafana界面","attributes":{},"skip":false,"key":"6.3.1.1.3"},{"backlink":"usecases/istio-installation.html#fig6.3.1.1.4","level":"6.3.1.1","list_caption":"Figure: Prometheus页面","alt":"Prometheus页面","nro":128,"url":"../images/istio-prometheus.jpg","index":4,"caption_template":"图片 - _CAPTION_","label":"Prometheus页面","attributes":{},"skip":false,"key":"6.3.1.1.4"},{"backlink":"usecases/istio-installation.html#fig6.3.1.1.5","level":"6.3.1.1","list_caption":"Figure: Zipkin页面","alt":"Zipkin页面","nro":129,"url":"../images/istio-zipkin.jpg","index":5,"caption_template":"图片 - _CAPTION_","label":"Zipkin页面","attributes":{},"skip":false,"key":"6.3.1.1.5"},{"backlink":"usecases/istio-installation.html#fig6.3.1.1.6","level":"6.3.1.1","list_caption":"Figure: ServiceGraph页面","alt":"ServiceGraph页面","nro":130,"url":"../images/istio-servicegraph.jpg","index":6,"caption_template":"图片 - _CAPTION_","label":"ServiceGraph页面","attributes":{},"skip":false,"key":"6.3.1.1.6"},{"backlink":"usecases/install-and-expand-istio-mesh.html#fig6.3.1.3.1","level":"6.3.1.3","list_caption":"Figure: BookInfo","alt":"BookInfo","nro":131,"url":"../images/noistio.png","index":1,"caption_template":"图片 - _CAPTION_","label":"BookInfo","attributes":{},"skip":false,"key":"6.3.1.3.1"},{"backlink":"usecases/install-and-expand-istio-mesh.html#fig6.3.1.3.2","level":"6.3.1.3","list_caption":"Figure: BookInfo","alt":"BookInfo","nro":132,"url":"../images/noistio.png","index":2,"caption_template":"图片 - _CAPTION_","label":"BookInfo","attributes":{},"skip":false,"key":"6.3.1.3.2"},{"backlink":"usecases/integrating-vms.html#fig6.3.1.4.1","level":"6.3.1.4","list_caption":"Figure: Bookinfo应用的拓展Mesh","alt":"Bookinfo应用的拓展Mesh","nro":133,"url":"../images/istio-mesh-expansion.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Bookinfo应用的拓展Mesh","attributes":{},"skip":false,"key":"6.3.1.4.1"},{"backlink":"usecases/linkerd.html#fig6.3.2.1","level":"6.3.2","list_caption":"Figure: source https://linkerd.io","alt":"source https://linkerd.io","nro":134,"url":"../images/diagram-individual-instance.png","index":1,"caption_template":"图片 - _CAPTION_","label":"source https://linkerd.io","attributes":{},"skip":false,"key":"6.3.2.1"},{"backlink":"usecases/linkerd-user-guide.html#fig6.3.2.1.1","level":"6.3.2.1","list_caption":"Figure: Jenkins pipeline","alt":"Jenkins pipeline","nro":135,"url":"../images/linkerd-jenkins-pipeline.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Jenkins pipeline","attributes":{},"skip":false,"key":"6.3.2.1.1"},{"backlink":"usecases/linkerd-user-guide.html#fig6.3.2.1.2","level":"6.3.2.1","list_caption":"Figure: Jenkins config","alt":"Jenkins config","nro":136,"url":"../images/linkerd-jenkins.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"Jenkins config","attributes":{},"skip":false,"key":"6.3.2.1.2"},{"backlink":"usecases/linkerd-user-guide.html#fig6.3.2.1.3","level":"6.3.2.1","list_caption":"Figure: namerd","alt":"namerd","nro":137,"url":"../images/namerd-internal.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"namerd","attributes":{},"skip":false,"key":"6.3.2.1.3"},{"backlink":"usecases/linkerd-user-guide.html#fig6.3.2.1.4","level":"6.3.2.1","list_caption":"Figure: linkerd监控","alt":"linkerd监控","nro":138,"url":"../images/linkerd-helloworld-outgoing.jpg","index":4,"caption_template":"图片 - _CAPTION_","label":"linkerd监控","attributes":{},"skip":false,"key":"6.3.2.1.4"},{"backlink":"usecases/linkerd-user-guide.html#fig6.3.2.1.5","level":"6.3.2.1","list_caption":"Figure: linkerd监控","alt":"linkerd监控","nro":139,"url":"../images/linkerd-helloworld-incoming.jpg","index":5,"caption_template":"图片 - _CAPTION_","label":"linkerd监控","attributes":{},"skip":false,"key":"6.3.2.1.5"},{"backlink":"usecases/linkerd-user-guide.html#fig6.3.2.1.6","level":"6.3.2.1","list_caption":"Figure: linkerd性能监控","alt":"linkerd性能监控","nro":140,"url":"../images/linkerd-grafana.png","index":6,"caption_template":"图片 - _CAPTION_","label":"linkerd性能监控","attributes":{},"skip":false,"key":"6.3.2.1.6"},{"backlink":"usecases/linkerd-user-guide.html#fig6.3.2.1.7","level":"6.3.2.1","list_caption":"Figure: Linkerd ingress controller","alt":"Linkerd ingress controller","nro":141,"url":"../images/linkerd-ingress-controller.jpg","index":7,"caption_template":"图片 - _CAPTION_","label":"Linkerd ingress controller","attributes":{},"skip":false,"key":"6.3.2.1.7"},{"backlink":"usecases/conduit-installation.html#fig6.3.3.2.1","level":"6.3.3.2","list_caption":"Figure: Conduit dashboard","alt":"Conduit dashboard","nro":142,"url":"../images/conduit-dashboard.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Conduit dashboard","attributes":{},"skip":false,"key":"6.3.3.2.1"},{"backlink":"usecases/big-data.html#fig6.4.1","level":"6.4","list_caption":"Figure: Spark on yarn with kubernetes","alt":"Spark on yarn with kubernetes","nro":143,"url":"../images/spark-on-yarn-with-kubernetes.png","index":1,"caption_template":"图片 - _CAPTION_","label":"Spark on yarn with kubernetes","attributes":{},"skip":false,"key":"6.4.1"},{"backlink":"usecases/big-data.html#fig6.4.2","level":"6.4","list_caption":"Figure: 在kubernetes上使用多种调度方式","alt":"在kubernetes上使用多种调度方式","nro":144,"url":"../images/spark-on-kubernetes-with-different-schedulers.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"在kubernetes上使用多种调度方式","attributes":{},"skip":false,"key":"6.4.2"},{"backlink":"usecases/spark-standalone-on-kubernetes.html#fig6.4.1.1","level":"6.4.1","list_caption":"Figure: spark master ui","alt":"spark master ui","nro":145,"url":"../images/spark-ui.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"spark master ui","attributes":{},"skip":false,"key":"6.4.1.1"},{"backlink":"usecases/spark-standalone-on-kubernetes.html#fig6.4.1.2","level":"6.4.1","list_caption":"Figure: zeppelin ui","alt":"zeppelin ui","nro":146,"url":"../images/zeppelin-ui.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"zeppelin ui","attributes":{},"skip":false,"key":"6.4.1.2"},{"backlink":"usecases/serverless.html#fig6.5.1","level":"6.5","list_caption":"Figure: 从物理机到函数计算","alt":"从物理机到函数计算","nro":147,"url":"../images/from-bare-metal-to-functions.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"从物理机到函数计算","attributes":{},"skip":false,"key":"6.5.1"},{"backlink":"usecases/serverless.html#fig6.5.2","level":"6.5","list_caption":"Figure: FaaS Landscape","alt":"FaaS Landscape","nro":148,"url":"../images/redpoint-faas-landscape.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"FaaS Landscape","attributes":{},"skip":false,"key":"6.5.2"},{"backlink":"usecases/understanding-serverless.html#fig6.5.1.1","level":"6.5.1","list_caption":"Figure: 服务端软件的运行环境","alt":"服务端软件的运行环境","nro":149,"url":"../images/serverless-server-side-software.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"服务端软件的运行环境","attributes":{},"skip":false,"key":"6.5.1.1"},{"backlink":"usecases/understanding-serverless.html#fig6.5.1.2","level":"6.5.1","list_caption":"Figure: FaaS应用架构","alt":"FaaS应用架构","nro":150,"url":"../images/serverless-faas-platform.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"FaaS应用架构","attributes":{},"skip":false,"key":"6.5.1.2"},{"backlink":"usecases/understanding-serverless.html#fig6.5.1.3","level":"6.5.1","list_caption":"Figure: 传统应用程序架构","alt":"传统应用程序架构","nro":151,"url":"../images/non-serverless-game-arch.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"传统应用程序架构","attributes":{},"skip":false,"key":"6.5.1.3"},{"backlink":"usecases/understanding-serverless.html#fig6.5.1.4","level":"6.5.1","list_caption":"Figure: Serverless架构","alt":"Serverless架构","nro":152,"url":"../images/serverless-game-arch.jpg","index":4,"caption_template":"图片 - _CAPTION_","label":"Serverless架构","attributes":{},"skip":false,"key":"6.5.1.4"},{"backlink":"usecases/openfaas-quick-start.html#fig6.5.2.1.1","level":"6.5.2.1","list_caption":"Figure: OpenFaaS架构","alt":"OpenFaaS架构","nro":153,"url":"../images/openfaas-arch.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"OpenFaaS架构","attributes":{},"skip":false,"key":"6.5.2.1.1"},{"backlink":"usecases/openfaas-quick-start.html#fig6.5.2.1.2","level":"6.5.2.1","list_caption":"Figure: OpenFaaS Prometheus","alt":"OpenFaaS Prometheus","nro":154,"url":"../images/openfaas-prometheus.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"OpenFaaS Prometheus","attributes":{},"skip":false,"key":"6.5.2.1.2"},{"backlink":"usecases/openfaas-quick-start.html#fig6.5.2.1.3","level":"6.5.2.1","list_caption":"Figure: OpenFaas Grafana监控","alt":"OpenFaas Grafana监控","nro":155,"url":"../images/openfaas-grafana.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"OpenFaas Grafana监控","attributes":{},"skip":false,"key":"6.5.2.1.3"},{"backlink":"usecases/openfaas-quick-start.html#fig6.5.2.1.4","level":"6.5.2.1","list_caption":"Figure: OpenFaas Dashboard","alt":"OpenFaas Dashboard","nro":156,"url":"../images/openfaas-deploy-a-function.jpg","index":4,"caption_template":"图片 - _CAPTION_","label":"OpenFaas Dashboard","attributes":{},"skip":false,"key":"6.5.2.1.4"},{"backlink":"usecases/openfaas-quick-start.html#fig6.5.2.1.5","level":"6.5.2.1","list_caption":"Figure: NodeInfo执行结果","alt":"NodeInfo执行结果","nro":157,"url":"../images/openfaas-nodeinfo.jpg","index":5,"caption_template":"图片 - _CAPTION_","label":"NodeInfo执行结果","attributes":{},"skip":false,"key":"6.5.2.1.5"},{"backlink":"develop/sigs-and-working-group.html#fig7.2.1","level":"7.2","list_caption":"Figure: Kubernetes SIG","alt":"Kubernetes SIG","nro":158,"url":"../images/kubernetes-sigs.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"Kubernetes SIG","attributes":{},"skip":false,"key":"7.2.1"},{"backlink":"develop/client-go-sample.html#fig7.5.1","level":"7.5","list_caption":"Figure: 使用kubernetes dashboard进行故障排查","alt":"使用kubernetes dashboard进行故障排查","nro":159,"url":"../images/kubernetes-client-go-sample-update.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"使用kubernetes dashboard进行故障排查","attributes":{},"skip":false,"key":"7.5.1"},{"backlink":"appendix/issues.html#fig8.6.1","level":"8.6","list_caption":"Figure: pvc-storage-limit","alt":"pvc-storage-limit","nro":160,"url":"../images/pvc-storage-limit.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"pvc-storage-limit","attributes":{},"skip":false,"key":"8.6.1"}]},"title":"Kubernetes Handbook - jimmysong.io","language":"zh-hans","links":{"sidebar":{"Jimmy Song":"https://jimmysong.io"}},"gitbook":"*","description":"Kubernetes中文指南/实践手册|kubernetes handbook|jimmysong.io|宋净超"},"file":{"path":"usecases/running-spark-with-kubernetes-native-scheduler.md","mtime":"2017-09-27T13:03:00.000Z","type":"markdown"},"gitbook":{"version":"3.2.2","time":"2017-12-30T02:36:33.675Z"},"basePath":"..","book":{"language":""}});
});
</script>
</div>
<script src="../gitbook/gitbook.js"></script>
<script src="../gitbook/theme.js"></script>
<script src="../gitbook/gitbook-plugin-github/plugin.js"></script>
<script src="../gitbook/gitbook-plugin-splitter/splitter.js"></script>
<script src="../gitbook/gitbook-plugin-page-toc-button/plugin.js"></script>
<script src="../gitbook/gitbook-plugin-editlink/plugin.js"></script>
<script src="../gitbook/gitbook-plugin-back-to-top-button/plugin.js"></script>
<script src="../gitbook/gitbook-plugin-search-plus/jquery.mark.min.js"></script>
<script src="../gitbook/gitbook-plugin-search-plus/search.js"></script>
<script src="../gitbook/gitbook-plugin-github-buttons/plugin.js"></script>
<script src="../gitbook/gitbook-plugin-3-ba/plugin.js"></script>
<script src="../gitbook/gitbook-plugin-sharing/buttons.js"></script>
<script src="../gitbook/gitbook-plugin-fontsettings/fontsettings.js"></script>
</body>
</html>