kubernetes-handbook/usecases/running-spark-with-kubernet...

2868 lines
154 KiB
HTML
Raw Normal View History

2017-09-14 15:57:50 +08:00
<!DOCTYPE HTML>
2017-09-19 22:01:07 +08:00
<html lang="zh-hans" >
2017-09-14 15:57:50 +08:00
<head>
<meta charset="UTF-8">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
2017-11-15 21:18:59 +08:00
<title>运行支持kubernetes原生调度的Spark程序 · Kubernetes Handbook - jimmysong.io</title>
2017-09-14 15:57:50 +08:00
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="description" content="">
<meta name="generator" content="GitBook 3.2.2">
<meta name="author" content="Jimmy Song">
<link rel="stylesheet" href="../gitbook/style.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-splitter/splitter.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-page-toc-button/plugin.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-image-captions/image-captions.css">
2017-10-09 15:28:05 +08:00
<link rel="stylesheet" href="../gitbook/gitbook-plugin-back-to-top-button/plugin.css">
2017-09-14 15:57:50 +08:00
2017-10-09 15:28:05 +08:00
<link rel="stylesheet" href="../gitbook/gitbook-plugin-search-plus/search.css">
2017-09-19 21:38:03 +08:00
2017-10-09 15:28:05 +08:00
<link rel="stylesheet" href="../gitbook/gitbook-plugin-tbfed-pagefooter/footer.css">
2017-09-14 15:57:50 +08:00
<link rel="stylesheet" href="../gitbook/gitbook-plugin-highlight/website.css">
<link rel="stylesheet" href="../gitbook/gitbook-plugin-fontsettings/website.css">
2017-10-09 15:28:05 +08:00
2017-09-14 15:57:50 +08:00
<meta name="HandheldFriendly" content="true"/>
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black">
<link rel="apple-touch-icon-precomposed" sizes="152x152" href="../gitbook/images/apple-touch-icon-precomposed-152.png">
<link rel="shortcut icon" href="../gitbook/images/favicon.ico" type="image/x-icon">
<link rel="next" href="serverless.html" />
<link rel="prev" href="spark-standalone-on-kubernetes.html" />
2017-10-09 15:28:05 +08:00
<link rel="shortcut icon" href='../favicon.ico' type="image/x-icon">
<link rel="bookmark" href='../favicon.ico' type="image/x-icon">
<style>
@media only screen and (max-width: 640px) {
.book-header .hidden-mobile {
display: none;
}
}
</style>
<script>
window["gitbook-plugin-github-buttons"] = {"repo":"rootsongjc/kubernetes-handbook","types":["star"],"size":"small"};
</script>
2017-09-14 15:57:50 +08:00
</head>
<body>
<div class="book">
<div class="book-summary">
<div id="book-search-input" role="search">
2017-09-19 22:01:07 +08:00
<input type="text" placeholder="输入并搜索" />
2017-09-14 15:57:50 +08:00
</div>
<nav role="navigation">
<ul class="summary">
2017-10-09 15:28:05 +08:00
<li>
2017-11-15 21:18:59 +08:00
<a href="https://jimmysong.io" target="_blank" class="custom-link">Jimmy Song</a>
2017-10-09 15:28:05 +08:00
</li>
2017-09-14 15:57:50 +08:00
2017-10-09 15:28:05 +08:00
<li class="divider"></li>
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<li class="header">前言</li>
2017-09-14 15:57:50 +08:00
<li class="chapter " data-level="1.1" data-path="../">
<a href="../">
2017-11-15 21:18:59 +08:00
<b>1.1.</b>
序言
2017-09-14 15:57:50 +08:00
</a>
</li>
<li class="chapter " data-level="1.2" data-path="../cloud-native/kubernetes-and-cloud-native-app-overview.html">
<a href="../cloud-native/kubernetes-and-cloud-native-app-overview.html">
2017-11-15 21:18:59 +08:00
<b>1.2.</b>
Kubernetes与云原生应用概览
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="header">概念与原理</li>
<li class="chapter " data-level="2.1" data-path="../concepts/">
2017-09-14 15:57:50 +08:00
<a href="../concepts/">
2017-11-15 21:18:59 +08:00
<b>2.1.</b>
Kubernetes架构
2017-09-14 15:57:50 +08:00
</a>
2017-11-15 21:18:59 +08:00
</li>
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.2" data-path="../concepts/concepts.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/concepts.html">
2017-11-15 21:18:59 +08:00
<b>2.2.</b>
设计理念
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3" data-path="../concepts/objects.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/objects.html">
2017-11-15 21:18:59 +08:00
<b>2.3.</b>
资源对象与基本概念解析
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.1" data-path="../concepts/pod-overview.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/pod-overview.html">
2017-11-15 21:18:59 +08:00
<b>2.3.1.</b>
Pod
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.1.1" data-path="../concepts/pod.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/pod.html">
2017-11-15 21:18:59 +08:00
<b>2.3.1.1.</b>
Pod解析
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.1.2" data-path="../concepts/init-containers.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/init-containers.html">
2017-11-15 21:18:59 +08:00
<b>2.3.1.2.</b>
Init容器
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.1.3" data-path="../concepts/pod-security-policy.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/pod-security-policy.html">
2017-11-15 21:18:59 +08:00
<b>2.3.1.3.</b>
Pod安全策略
2017-09-14 15:57:50 +08:00
</a>
2017-09-17 15:39:26 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.1.4" data-path="../concepts/pod-lifecycle.html">
2017-09-17 15:39:26 +08:00
<a href="../concepts/pod-lifecycle.html">
2017-11-15 21:18:59 +08:00
<b>2.3.1.4.</b>
Pod的生命周期
2017-09-17 15:39:26 +08:00
</a>
2017-09-14 15:57:50 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.2" data-path="../concepts/node.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/node.html">
2017-11-15 21:18:59 +08:00
<b>2.3.2.</b>
Node
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.3" data-path="../concepts/namespace.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/namespace.html">
2017-11-15 21:18:59 +08:00
<b>2.3.3.</b>
Namespace
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.4" data-path="../concepts/service.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/service.html">
2017-11-15 21:18:59 +08:00
<b>2.3.4.</b>
Service
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.5" data-path="../concepts/volume.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/volume.html">
2017-11-15 21:18:59 +08:00
<b>2.3.5.</b>
Volume和Persistent Volume
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.6" data-path="../concepts/deployment.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/deployment.html">
2017-11-15 21:18:59 +08:00
<b>2.3.6.</b>
Deployment
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.7" data-path="../concepts/secret.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/secret.html">
2017-11-15 21:18:59 +08:00
<b>2.3.7.</b>
Secret
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.8" data-path="../concepts/statefulset.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/statefulset.html">
2017-11-15 21:18:59 +08:00
<b>2.3.8.</b>
StatefulSet
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.9" data-path="../concepts/daemonset.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/daemonset.html">
2017-11-15 21:18:59 +08:00
<b>2.3.9.</b>
DaemonSet
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.10" data-path="../concepts/serviceaccount.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/serviceaccount.html">
2017-11-15 21:18:59 +08:00
<b>2.3.10.</b>
ServiceAccount
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.11" data-path="../concepts/replicaset.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/replicaset.html">
2017-11-15 21:18:59 +08:00
<b>2.3.11.</b>
ReplicationController和ReplicaSet
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.12" data-path="../concepts/job.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/job.html">
2017-11-15 21:18:59 +08:00
<b>2.3.12.</b>
Job
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.13" data-path="../concepts/cronjob.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/cronjob.html">
2017-11-15 21:18:59 +08:00
<b>2.3.13.</b>
CronJob
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.14" data-path="../concepts/ingress.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/ingress.html">
2017-11-15 21:18:59 +08:00
<b>2.3.14.</b>
Ingress
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.15" data-path="../concepts/configmap.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/configmap.html">
2017-11-15 21:18:59 +08:00
<b>2.3.15.</b>
ConfigMap
2017-09-14 15:57:50 +08:00
</a>
2017-11-08 19:28:16 +08:00
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.15.1" data-path="../concepts/configmap-hot-update.html">
2017-11-08 19:28:16 +08:00
<a href="../concepts/configmap-hot-update.html">
2017-11-15 21:18:59 +08:00
<b>2.3.15.1.</b>
ConfigMap的热更新
2017-11-08 19:28:16 +08:00
</a>
</li>
</ul>
2017-09-14 15:57:50 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.16" data-path="../concepts/horizontal-pod-autoscaling.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/horizontal-pod-autoscaling.html">
2017-11-15 21:18:59 +08:00
<b>2.3.16.</b>
Horizontal Pod Autoscaling
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.17" data-path="../concepts/label.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/label.html">
2017-11-15 21:18:59 +08:00
<b>2.3.17.</b>
Label
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.18" data-path="../concepts/garbage-collection.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/garbage-collection.html">
2017-11-15 21:18:59 +08:00
<b>2.3.18.</b>
垃圾收集
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.19" data-path="../concepts/network-policy.html">
2017-09-14 15:57:50 +08:00
<a href="../concepts/network-policy.html">
2017-11-15 21:18:59 +08:00
<b>2.3.19.</b>
NetworkPolicy
2017-09-14 15:57:50 +08:00
</a>
2017-11-09 16:48:38 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="2.3.20" data-path="../concepts/annotation.html">
2017-11-09 16:48:38 +08:00
<a href="../concepts/annotation.html">
2017-11-15 21:18:59 +08:00
<b>2.3.20.</b>
Annotation
2017-11-09 16:48:38 +08:00
</a>
2017-09-14 15:57:50 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="header">用户指南</li>
<li class="chapter " data-level="3.1" data-path="../guide/">
2017-09-14 15:57:50 +08:00
<a href="../guide/">
2017-11-15 21:18:59 +08:00
<b>3.1.</b>
用户指南
2017-09-14 15:57:50 +08:00
</a>
2017-11-15 21:18:59 +08:00
</li>
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.2" data-path="../guide/resource-configuration.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/resource-configuration.html">
2017-11-15 21:18:59 +08:00
<b>3.2.</b>
资源对象配置
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.2.1" data-path="../guide/configure-liveness-readiness-probes.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/configure-liveness-readiness-probes.html">
2017-11-15 21:18:59 +08:00
<b>3.2.1.</b>
配置Pod的liveness和readiness探针
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.2.2" data-path="../guide/configure-pod-service-account.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/configure-pod-service-account.html">
2017-11-15 21:18:59 +08:00
<b>3.2.2.</b>
配置Pod的Service Account
2017-09-14 15:57:50 +08:00
</a>
2017-09-28 21:20:49 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.2.3" data-path="../guide/secret-configuration.html">
2017-09-28 21:20:49 +08:00
<a href="../guide/secret-configuration.html">
2017-11-15 21:18:59 +08:00
<b>3.2.3.</b>
Secret配置
2017-09-28 21:20:49 +08:00
</a>
2017-10-10 14:51:45 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.2.4" data-path="../guide/resource-quota-management.html">
2017-10-10 14:51:45 +08:00
<a href="../guide/resource-quota-management.html">
2017-11-15 21:18:59 +08:00
<b>3.2.4.</b>
管理namespace中的资源配额
2017-10-10 14:51:45 +08:00
</a>
2017-09-14 15:57:50 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.3" data-path="../guide/command-usage.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/command-usage.html">
2017-11-15 21:18:59 +08:00
<b>3.3.</b>
命令使用
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.3.1" data-path="../guide/docker-cli-to-kubectl.html">
2017-11-03 17:48:10 +08:00
<a href="../guide/docker-cli-to-kubectl.html">
2017-11-15 21:18:59 +08:00
<b>3.3.1.</b>
docker用户过度到kubectl命令行指南
2017-11-03 17:48:10 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.3.2" data-path="../guide/using-kubectl.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/using-kubectl.html">
2017-11-15 21:18:59 +08:00
<b>3.3.2.</b>
kubectl命令概览
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.3.3" data-path="../guide/kubectl-cheatsheet.html">
2017-11-03 17:48:10 +08:00
<a href="../guide/kubectl-cheatsheet.html">
2017-11-15 21:18:59 +08:00
<b>3.3.3.</b>
kubectl命令技巧大全
</a>
2017-09-14 15:57:50 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.4" data-path="../guide/cluster-security-management.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/cluster-security-management.html">
2017-11-15 21:18:59 +08:00
<b>3.4.</b>
集群安全性管理
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.4.1" data-path="../guide/managing-tls-in-a-cluster.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/managing-tls-in-a-cluster.html">
2017-11-15 21:18:59 +08:00
<b>3.4.1.</b>
管理集群中的TLS
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.4.2" data-path="../guide/kubelet-authentication-authorization.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/kubelet-authentication-authorization.html">
2017-11-15 21:18:59 +08:00
<b>3.4.2.</b>
kubelet的认证授权
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.4.3" data-path="../guide/tls-bootstrapping.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/tls-bootstrapping.html">
2017-11-15 21:18:59 +08:00
<b>3.4.3.</b>
TLS bootstrap
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.4.4" data-path="../guide/kubectl-user-authentication-authorization.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/kubectl-user-authentication-authorization.html">
2017-11-15 21:18:59 +08:00
<b>3.4.4.</b>
创建用户认证授权的kubeconfig文件
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.4.5" data-path="../guide/rbac.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/rbac.html">
2017-11-15 21:18:59 +08:00
<b>3.4.5.</b>
RBAC——基于角色的访问控制
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.4.6" data-path="../guide/ip-masq-agent.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/ip-masq-agent.html">
2017-11-15 21:18:59 +08:00
<b>3.4.6.</b>
IP伪装代理
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.4.7" data-path="../guide/auth-with-kubeconfig-or-token.html">
<a href="../guide/auth-with-kubeconfig-or-token.html">
2017-11-15 21:18:59 +08:00
<b>3.4.7.</b>
使用kubeconfig或token进行用户身份认证
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.4.8" data-path="../guide/authentication.html">
<a href="../guide/authentication.html">
2017-11-15 21:18:59 +08:00
<b>3.4.8.</b>
kubernetes中的用户与身份认证授权
</a>
2017-09-14 15:57:50 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.5" data-path="../guide/access-kubernetes-cluster.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/access-kubernetes-cluster.html">
2017-11-15 21:18:59 +08:00
<b>3.5.</b>
2017-11-21 20:10:10 +08:00
访问Kubernetes集群
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.5.1" data-path="../guide/access-cluster.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/access-cluster.html">
2017-11-15 21:18:59 +08:00
<b>3.5.1.</b>
访问集群
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.5.2" data-path="../guide/authenticate-across-clusters-kubeconfig.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/authenticate-across-clusters-kubeconfig.html">
2017-11-15 21:18:59 +08:00
<b>3.5.2.</b>
2017-11-21 20:10:10 +08:00
使用kubeconfig文件配置跨集群认证
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.5.3" data-path="../guide/connecting-to-applications-port-forward.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/connecting-to-applications-port-forward.html">
2017-11-15 21:18:59 +08:00
<b>3.5.3.</b>
通过端口转发访问集群中的应用程序
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.5.4" data-path="../guide/service-access-application-cluster.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/service-access-application-cluster.html">
2017-11-15 21:18:59 +08:00
<b>3.5.4.</b>
2017-11-21 20:10:10 +08:00
使用service访问群集中的应用程序
</a>
</li>
<li class="chapter " data-level="3.5.5" data-path="../guide/accessing-kubernetes-pods-from-outside-of-the-cluster.html">
<a href="../guide/accessing-kubernetes-pods-from-outside-of-the-cluster.html">
<b>3.5.5.</b>
从外部访问Kubernetes中的Pod
2017-09-14 15:57:50 +08:00
</a>
2017-11-22 18:43:03 +08:00
</li>
<li class="chapter " data-level="3.5.6" data-path="../guide/carbin-mobile-dashboard-for-kubernetes.html">
<a href="../guide/carbin-mobile-dashboard-for-kubernetes.html">
<b>3.5.6.</b>
Carbin - Kuberentes手机客户端
</a>
2017-09-14 15:57:50 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.6" data-path="../guide/application-development-deployment-flow.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/application-development-deployment-flow.html">
2017-11-15 21:18:59 +08:00
<b>3.6.</b>
在kubernetes中开发部署应用
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.6.1" data-path="../guide/deploy-applications-in-kubernetes.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/deploy-applications-in-kubernetes.html">
2017-11-15 21:18:59 +08:00
<b>3.6.1.</b>
适用于kubernetes的应用开发部署流程
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.6.2" data-path="../guide/migrating-hadoop-yarn-to-kubernetes.html">
2017-09-14 15:57:50 +08:00
<a href="../guide/migrating-hadoop-yarn-to-kubernetes.html">
2017-11-15 21:18:59 +08:00
<b>3.6.2.</b>
迁移传统应用到kubernetes中——以Hadoop YARN为例
2017-09-14 15:57:50 +08:00
</a>
2017-10-23 13:40:58 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="3.6.3" data-path="../guide/using-statefulset.html">
2017-10-23 13:40:58 +08:00
<a href="../guide/using-statefulset.html">
2017-11-15 21:18:59 +08:00
<b>3.6.3.</b>
使用StatefulSet部署用状态应用
2017-10-23 13:40:58 +08:00
</a>
2017-09-14 15:57:50 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="header">最佳实践</li>
<li class="chapter " data-level="4.1" data-path="../practice/">
2017-09-14 15:57:50 +08:00
<a href="../practice/">
2017-11-15 21:18:59 +08:00
<b>4.1.</b>
最佳实践概览
2017-09-14 15:57:50 +08:00
</a>
2017-11-15 21:18:59 +08:00
</li>
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2" data-path="../practice/install-kbernetes1.6-on-centos.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/install-kbernetes1.6-on-centos.html">
2017-11-15 21:18:59 +08:00
<b>4.2.</b>
在CentOS上部署kubernetes1.6集群
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2.1" data-path="../practice/create-tls-and-secret-key.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/create-tls-and-secret-key.html">
2017-11-15 21:18:59 +08:00
<b>4.2.1.</b>
创建TLS证书和秘钥
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2.2" data-path="../practice/create-kubeconfig.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/create-kubeconfig.html">
2017-11-15 21:18:59 +08:00
<b>4.2.2.</b>
创建kubeconfig文件
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2.3" data-path="../practice/etcd-cluster-installation.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/etcd-cluster-installation.html">
2017-11-15 21:18:59 +08:00
<b>4.2.3.</b>
创建高可用etcd集群
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2.4" data-path="../practice/kubectl-installation.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/kubectl-installation.html">
2017-11-15 21:18:59 +08:00
<b>4.2.4.</b>
安装kubectl命令行工具
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2.5" data-path="../practice/master-installation.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/master-installation.html">
2017-11-15 21:18:59 +08:00
<b>4.2.5.</b>
部署master节点
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2.6" data-path="../practice/node-installation.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/node-installation.html">
2017-11-15 21:18:59 +08:00
<b>4.2.6.</b>
部署node节点
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2.7" data-path="../practice/kubedns-addon-installation.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/kubedns-addon-installation.html">
2017-11-15 21:18:59 +08:00
<b>4.2.7.</b>
安装kubedns插件
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2.8" data-path="../practice/dashboard-addon-installation.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/dashboard-addon-installation.html">
2017-11-15 21:18:59 +08:00
<b>4.2.8.</b>
安装dashboard插件
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2.9" data-path="../practice/heapster-addon-installation.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/heapster-addon-installation.html">
2017-11-15 21:18:59 +08:00
<b>4.2.9.</b>
安装heapster插件
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.2.10" data-path="../practice/efk-addon-installation.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/efk-addon-installation.html">
2017-11-15 21:18:59 +08:00
<b>4.2.10.</b>
安装EFK插件
2017-09-14 15:57:50 +08:00
</a>
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.3" data-path="../practice/service-discovery-and-loadbalancing.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/service-discovery-and-loadbalancing.html">
2017-11-15 21:18:59 +08:00
<b>4.3.</b>
服务发现与负载均衡
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.3.1" data-path="../practice/traefik-ingress-installation.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/traefik-ingress-installation.html">
2017-11-15 21:18:59 +08:00
<b>4.3.1.</b>
安装Traefik ingress
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.3.2" data-path="../practice/distributed-load-test.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/distributed-load-test.html">
2017-11-15 21:18:59 +08:00
<b>4.3.2.</b>
分布式负载测试
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.3.3" data-path="../practice/network-and-cluster-perfermance-test.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/network-and-cluster-perfermance-test.html">
2017-11-15 21:18:59 +08:00
<b>4.3.3.</b>
网络和集群性能测试
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.3.4" data-path="../practice/edge-node-configuration.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/edge-node-configuration.html">
2017-11-15 21:18:59 +08:00
<b>4.3.4.</b>
边缘节点配置
2017-09-14 15:57:50 +08:00
</a>
2017-10-27 19:09:01 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.3.5" data-path="../practice/nginx-ingress-installation.html">
2017-10-27 19:09:01 +08:00
<a href="../practice/nginx-ingress-installation.html">
2017-11-15 21:18:59 +08:00
<b>4.3.5.</b>
安装Nginx ingress
2017-10-27 19:09:01 +08:00
</a>
2017-09-14 15:57:50 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.4" data-path="../practice/operation.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/operation.html">
2017-11-15 21:18:59 +08:00
<b>4.4.</b>
运维管理
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.4.1" data-path="../practice/service-rolling-update.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/service-rolling-update.html">
2017-11-15 21:18:59 +08:00
<b>4.4.1.</b>
服务滚动升级
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.4.2" data-path="../practice/app-log-collection.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/app-log-collection.html">
2017-11-15 21:18:59 +08:00
<b>4.4.2.</b>
应用日志收集
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.4.3" data-path="../practice/configuration-best-practice.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/configuration-best-practice.html">
2017-11-15 21:18:59 +08:00
<b>4.4.3.</b>
配置最佳实践
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.4.4" data-path="../practice/monitor.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/monitor.html">
2017-11-15 21:18:59 +08:00
<b>4.4.4.</b>
集群及应用监控
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.4.5" data-path="../practice/data-persistence-problem.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/data-persistence-problem.html">
2017-11-15 21:18:59 +08:00
<b>4.4.5.</b>
数据持久化问题
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.4.6" data-path="../practice/manage-compute-resources-container.html">
2017-09-14 15:57:50 +08:00
<a href="../practice/manage-compute-resources-container.html">
2017-11-15 21:18:59 +08:00
<b>4.4.6.</b>
管理容器的计算资源
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-09 10:55:38 +08:00
</ul>
2017-11-09 10:55:38 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.5" data-path="../practice/storage.html">
2017-11-09 10:55:38 +08:00
<a href="../practice/storage.html">
2017-11-15 21:18:59 +08:00
<b>4.5.</b>
存储管理
2017-11-09 10:55:38 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.5.1" data-path="../practice/glusterfs.html">
2017-11-09 10:55:38 +08:00
<a href="../practice/glusterfs.html">
2017-11-15 21:18:59 +08:00
<b>4.5.1.</b>
GlusterFS
2017-11-09 10:55:38 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.5.1.1" data-path="../practice/using-glusterfs-for-persistent-storage.html">
2017-11-09 10:55:38 +08:00
<a href="../practice/using-glusterfs-for-persistent-storage.html">
2017-11-15 21:18:59 +08:00
<b>4.5.1.1.</b>
使用GlusterFS做持久化存储
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.5.1.2" data-path="../practice/storage-for-containers-using-glusterfs-with-openshift.html">
2017-11-09 10:55:38 +08:00
<a href="../practice/storage-for-containers-using-glusterfs-with-openshift.html">
2017-11-15 21:18:59 +08:00
<b>4.5.1.2.</b>
在OpenShift中使用GlusterFS做持久化存储
</a>
2017-09-14 15:57:50 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.5.2" data-path="../practice/cephfs.html">
2017-09-14 15:57:50 +08:00
2017-11-09 10:55:38 +08:00
<a href="../practice/cephfs.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<b>4.5.2.</b>
CephFS
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.5.2.1" data-path="../practice/using-ceph-for-persistent-storage.html">
2017-09-14 15:57:50 +08:00
2017-11-09 10:55:38 +08:00
<a href="../practice/using-ceph-for-persistent-storage.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<b>4.5.2.1.</b>
使用Ceph做持久化存储
2017-11-09 10:55:38 +08:00
</a>
</li>
</ul>
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.6" data-path="../practice/monitoring.html">
2017-11-09 10:55:38 +08:00
<a href="../practice/monitoring.html">
2017-11-15 21:18:59 +08:00
<b>4.6.</b>
集群与应用监控
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.6.1" data-path="../practice/heapster.html">
2017-09-14 15:57:50 +08:00
2017-11-09 10:55:38 +08:00
<a href="../practice/heapster.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<b>4.6.1.</b>
Heapster
2017-09-14 15:57:50 +08:00
</a>
2017-11-09 10:55:38 +08:00
<ul class="articles">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.6.1.1" data-path="../practice/using-heapster-to-get-object-metrics.html">
2017-09-14 15:57:50 +08:00
2017-11-09 10:55:38 +08:00
<a href="../practice/using-heapster-to-get-object-metrics.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<b>4.6.1.1.</b>
使用Heapster获取集群和对象的metric数据
2017-09-14 15:57:50 +08:00
</a>
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.6.2" data-path="../practice/prometheus.html">
2017-09-14 15:57:50 +08:00
2017-11-09 10:55:38 +08:00
<a href="../practice/prometheus.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<b>4.6.2.</b>
Prometheus
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.6.2.1" data-path="../practice/using-prometheus-to-monitor-kuberentes-cluster.html">
2017-09-14 15:57:50 +08:00
2017-11-09 10:55:38 +08:00
<a href="../practice/using-prometheus-to-monitor-kuberentes-cluster.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<b>4.6.2.1.</b>
使用Prometheus监控kubernetes集群
2017-09-14 15:57:50 +08:00
</a>
</li>
</ul>
</li>
</ul>
2017-10-19 15:31:08 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.7" data-path="../practice/services-management-tool.html">
2017-10-19 15:31:08 +08:00
<a href="../practice/services-management-tool.html">
2017-11-15 21:18:59 +08:00
<b>4.7.</b>
服务编排管理
2017-10-19 15:31:08 +08:00
</a>
2017-10-20 16:22:43 +08:00
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.7.1" data-path="../practice/helm.html">
2017-10-20 16:22:43 +08:00
<a href="../practice/helm.html">
2017-11-15 21:18:59 +08:00
<b>4.7.1.</b>
使用Helm管理kubernetes应用
2017-10-20 16:22:43 +08:00
</a>
2017-10-27 11:59:37 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.7.2" data-path="../practice/create-private-charts-repo.html">
2017-10-27 11:59:37 +08:00
<a href="../practice/create-private-charts-repo.html">
2017-11-15 21:18:59 +08:00
<b>4.7.2.</b>
构建私有Chart仓库
2017-10-27 11:59:37 +08:00
</a>
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.8" data-path="../practice/ci-cd.html">
2017-10-27 11:59:37 +08:00
<a href="../practice/ci-cd.html">
2017-11-15 21:18:59 +08:00
<b>4.8.</b>
持续集成与发布
2017-10-27 11:59:37 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.8.1" data-path="../practice/jenkins-ci-cd.html">
2017-10-27 11:59:37 +08:00
<a href="../practice/jenkins-ci-cd.html">
2017-11-15 21:18:59 +08:00
<b>4.8.1.</b>
使用Jenkins进行持续集成与发布
2017-10-27 11:59:37 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.8.2" data-path="../practice/drone-ci-cd.html">
2017-10-27 11:59:37 +08:00
<a href="../practice/drone-ci-cd.html">
2017-11-15 21:18:59 +08:00
<b>4.8.2.</b>
使用Drone进行持续集成与发布
2017-10-27 11:59:37 +08:00
</a>
2017-11-02 16:33:01 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.9" data-path="../practice/update-and-upgrade.html">
2017-11-02 16:33:01 +08:00
<a href="../practice/update-and-upgrade.html">
2017-11-15 21:18:59 +08:00
<b>4.9.</b>
更新与升级
2017-11-02 16:33:01 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.9.1" data-path="../practice/manually-upgrade.html">
2017-11-02 16:33:01 +08:00
<a href="../practice/manually-upgrade.html">
2017-11-15 21:18:59 +08:00
<b>4.9.1.</b>
手动升级kubernetes集群
2017-11-02 16:33:01 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="4.9.2" data-path="../practice/dashboard-upgrade.html">
2017-11-02 16:33:01 +08:00
<a href="../practice/dashboard-upgrade.html">
2017-11-15 21:18:59 +08:00
<b>4.9.2.</b>
升级dashboard
2017-11-02 16:33:01 +08:00
</a>
2017-10-20 16:22:43 +08:00
</li>
</ul>
2017-09-14 15:57:50 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="header">领域应用</li>
<li class="chapter " data-level="5.1" data-path="./">
2017-09-14 15:57:50 +08:00
<a href="./">
2017-11-15 21:18:59 +08:00
<b>5.1.</b>
领域应用概览
2017-09-14 15:57:50 +08:00
</a>
2017-11-15 21:18:59 +08:00
</li>
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.2" data-path="microservices.html">
2017-09-14 15:57:50 +08:00
<a href="microservices.html">
2017-11-15 21:18:59 +08:00
<b>5.2.</b>
微服务架构
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.2.1" data-path="service-discovery-in-microservices.html">
2017-09-20 21:55:19 +08:00
<a href="service-discovery-in-microservices.html">
2017-11-15 21:18:59 +08:00
<b>5.2.1.</b>
微服务中的服务发现
2017-09-20 21:55:19 +08:00
</a>
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.3" data-path="service-mesh.html">
2017-09-20 21:55:19 +08:00
<a href="service-mesh.html">
2017-11-15 21:18:59 +08:00
<b>5.3.</b>
Service Mesh 服务网格
2017-09-20 21:55:19 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.3.1" data-path="istio.html">
2017-09-14 15:57:50 +08:00
<a href="istio.html">
2017-11-15 21:18:59 +08:00
<b>5.3.1.</b>
Istio
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.3.1.1" data-path="istio-installation.html">
2017-09-14 15:57:50 +08:00
<a href="istio-installation.html">
2017-11-15 21:18:59 +08:00
<b>5.3.1.1.</b>
安装并试用Istio service mesh
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.3.1.2" data-path="configuring-request-routing.html">
2017-09-14 15:57:50 +08:00
<a href="configuring-request-routing.html">
2017-11-15 21:18:59 +08:00
<b>5.3.1.2.</b>
配置请求的路由规则
2017-09-14 15:57:50 +08:00
</a>
2017-11-07 23:57:26 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.3.1.3" data-path="install-and-expand-istio-mesh.html">
2017-11-07 23:57:26 +08:00
<a href="install-and-expand-istio-mesh.html">
2017-11-15 21:18:59 +08:00
<b>5.3.1.3.</b>
安装和拓展Istio service mesh
2017-11-07 23:57:26 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.3.1.4" data-path="integrating-vms.html">
2017-11-07 23:57:26 +08:00
<a href="integrating-vms.html">
2017-11-15 21:18:59 +08:00
<b>5.3.1.4.</b>
集成虚拟机
2017-11-07 23:57:26 +08:00
</a>
2017-09-14 15:57:50 +08:00
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.3.2" data-path="linkerd.html">
2017-09-14 15:57:50 +08:00
<a href="linkerd.html">
2017-11-15 21:18:59 +08:00
<b>5.3.2.</b>
Linkerd
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.3.2.1" data-path="linkerd-user-guide.html">
2017-09-14 15:57:50 +08:00
<a href="linkerd-user-guide.html">
2017-11-15 21:18:59 +08:00
<b>5.3.2.1.</b>
Linkerd 使用指南
2017-09-14 15:57:50 +08:00
</a>
</li>
</ul>
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.4" data-path="big-data.html">
2017-09-14 15:57:50 +08:00
<a href="big-data.html">
2017-11-15 21:18:59 +08:00
<b>5.4.</b>
大数据
2017-09-14 15:57:50 +08:00
</a>
<ul class="articles">
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.4.1" data-path="spark-standalone-on-kubernetes.html">
2017-09-14 15:57:50 +08:00
<a href="spark-standalone-on-kubernetes.html">
2017-11-15 21:18:59 +08:00
<b>5.4.1.</b>
Spark standalone on Kubernetes
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter active" data-level="5.4.2" data-path="running-spark-with-kubernetes-native-scheduler.html">
2017-09-14 15:57:50 +08:00
<a href="running-spark-with-kubernetes-native-scheduler.html">
2017-11-15 21:18:59 +08:00
<b>5.4.2.</b>
运行支持kubernetes原生调度的Spark程序
2017-09-14 15:57:50 +08:00
</a>
</li>
</ul>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.5" data-path="serverless.html">
2017-09-14 15:57:50 +08:00
<a href="serverless.html">
2017-11-15 21:18:59 +08:00
<b>5.5.</b>
Serverless架构
2017-09-14 15:57:50 +08:00
</a>
2017-10-27 19:09:01 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="5.6" data-path="edge-computing.html">
2017-10-27 19:09:01 +08:00
<a href="edge-computing.html">
2017-11-15 21:18:59 +08:00
<b>5.6.</b>
边缘计算
2017-10-27 19:09:01 +08:00
</a>
2017-09-14 15:57:50 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="header">开发指南</li>
<li class="chapter " data-level="6.1" data-path="../develop/">
2017-09-14 15:57:50 +08:00
<a href="../develop/">
2017-11-15 21:18:59 +08:00
<b>6.1.</b>
开发指南概览
2017-09-14 15:57:50 +08:00
</a>
2017-11-15 21:18:59 +08:00
</li>
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="6.2" data-path="../develop/developing-environment.html">
2017-09-14 15:57:50 +08:00
<a href="../develop/developing-environment.html">
2017-11-15 21:18:59 +08:00
<b>6.2.</b>
开发环境搭建
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="6.3" data-path="../develop/testing.html">
2017-09-14 15:57:50 +08:00
<a href="../develop/testing.html">
2017-11-15 21:18:59 +08:00
<b>6.3.</b>
单元测试和集成测试
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="6.4" data-path="../develop/client-go-sample.html">
2017-09-14 15:57:50 +08:00
<a href="../develop/client-go-sample.html">
2017-11-15 21:18:59 +08:00
<b>6.4.</b>
client-go示例
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="6.5" data-path="../develop/contribute.html">
2017-09-14 15:57:50 +08:00
<a href="../develop/contribute.html">
2017-11-15 21:18:59 +08:00
<b>6.5.</b>
社区贡献
2017-09-14 15:57:50 +08:00
</a>
2017-10-23 19:09:41 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="6.6" data-path="../develop/minikube.html">
2017-10-23 19:09:41 +08:00
<a href="../develop/minikube.html">
2017-11-15 21:18:59 +08:00
<b>6.6.</b>
Minikube
2017-10-23 19:09:41 +08:00
</a>
2017-09-14 15:57:50 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="header">附录</li>
<li class="chapter " data-level="7.1" data-path="../appendix/">
2017-09-14 15:57:50 +08:00
<a href="../appendix/">
2017-11-15 21:18:59 +08:00
<b>7.1.</b>
附录说明
2017-09-14 15:57:50 +08:00
</a>
2017-11-15 21:18:59 +08:00
</li>
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="7.2" data-path="../appendix/debug-kubernetes-services.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<a href="../appendix/debug-kubernetes-services.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<b>7.2.</b>
Kubernetes中的应用故障排查
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="7.3" data-path="../appendix/material-share.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<a href="../appendix/material-share.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<b>7.3.</b>
Kubernetes相关资讯和情报链接
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="7.4" data-path="../appendix/docker-best-practice.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<a href="../appendix/docker-best-practice.html">
2017-09-14 15:57:50 +08:00
2017-11-15 21:18:59 +08:00
<b>7.4.</b>
Docker最佳实践
2017-09-14 15:57:50 +08:00
</a>
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="7.5" data-path="../appendix/tricks.html">
2017-11-15 21:18:59 +08:00
<a href="../appendix/tricks.html">
2017-11-15 21:18:59 +08:00
<b>7.5.</b>
使用技巧
</a>
2017-10-18 14:16:55 +08:00
</li>
2017-11-15 21:18:59 +08:00
<li class="chapter " data-level="7.6" data-path="../appendix/issues.html">
2017-10-18 14:16:55 +08:00
2017-11-15 21:18:59 +08:00
<a href="../appendix/issues.html">
2017-10-18 14:16:55 +08:00
2017-11-15 21:18:59 +08:00
<b>7.6.</b>
问题记录
2017-10-18 14:16:55 +08:00
</a>
2017-09-14 15:57:50 +08:00
</li>
<li class="divider"></li>
<li>
<a href="https://www.gitbook.com" target="blank" class="gitbook-link">
2017-09-19 22:01:07 +08:00
本书使用 GitBook 发布
2017-09-14 15:57:50 +08:00
</a>
</li>
</ul>
</nav>
</div>
<div class="book-body">
<div class="body-inner">
<div class="book-header" role="navigation">
<!-- Title -->
<h1>
<i class="fa fa-circle-o-notch fa-spin"></i>
2017-11-15 21:18:59 +08:00
<a href=".." >运行支持kubernetes原生调度的Spark程序</a>
2017-09-14 15:57:50 +08:00
</h1>
</div>
<div class="page-wrapper" tabindex="-1" role="main">
<div class="page-inner">
<div class="search-plus" id="book-search-results">
<div class="search-noresults">
<section class="normal markdown-section">
<h1 id="&#x8FD0;&#x884C;&#x652F;&#x6301;kubernetes&#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684;spark&#x7A0B;&#x5E8F;">&#x8FD0;&#x884C;&#x652F;&#x6301;kubernetes&#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684;Spark&#x7A0B;&#x5E8F;</h1>
<p>TL;DR &#x8FD9;&#x4E2A;&#x4E3B;&#x9898;&#x6BD4;&#x8F83;&#x5927;&#xFF0C;&#x8BE5;&#x5F00;&#x6E90;&#x9879;&#x76EE;&#x4E5F;&#x8FD8;&#x5728;&#x4E0D;&#x65AD;&#x8FDB;&#x884C;&#x4E2D;&#xFF0C;&#x6211;&#x5355;&#x72EC;&#x505A;&#x4E86;&#x4E00;&#x4E2A; web &#x7528;&#x6765;&#x8BB0;&#x5F55; spark on kubernetes &#x7684;&#x7814;&#x7A76;&#x548C;&#x6700;&#x65B0;&#x8FDB;&#x5C55;&#x89C1;: <a href="https://jimmysong.io/spark-on-k8s" target="_blank">https://jimmysong.io/spark-on-k8s</a></p>
2017-09-14 15:57:50 +08:00
<p>&#x6211;&#x4EEC;&#x4E4B;&#x524D;&#x5C31;&#x5728; kubernetes &#x4E2D;&#x8FD0;&#x884C;&#x8FC7; standalone &#x65B9;&#x5F0F;&#x7684; spark &#x96C6;&#x7FA4;&#xFF0C;&#x89C1; <a href="spark-standalone-on-kubernetes.html">Spark standalone on kubernetes</a>&#x3002;</p>
2017-09-14 16:06:45 +08:00
<p>&#x76EE;&#x524D;&#x8FD0;&#x884C;&#x652F;&#x6301; kubernetes &#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684; spark &#x7A0B;&#x5E8F;&#x7531; Google &#x4E3B;&#x5BFC;&#xFF0C;&#x76EE;&#x524D;&#x8FD0;&#x884C;&#x652F;&#x6301; kubernetes &#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684; spark &#x7A0B;&#x5E8F;&#x7531; Google &#x4E3B;&#x5BFC;&#xFF0C;fork &#x81EA; spark &#x7684;&#x5B98;&#x65B9;&#x4EE3;&#x7801;&#x5E93;&#xFF0C;&#x89C1;<a href="https://github.com/apache-spark-on-k8s/spark/" target="_blank">https://github.com/apache-spark-on-k8s/spark/</a> &#xFF0C;&#x5C5E;&#x4E8E;Big Data SIG&#x3002;</p>
<p>&#x53C2;&#x4E0E;&#x5230;&#x8BE5;&#x9879;&#x76EE;&#x7684;&#x516C;&#x53F8;&#x6709;&#xFF1A;</p>
<ul>
<li>Bloomberg</li>
<li>Google</li>
<li>Haiwen</li>
<li>Hyperpilot</li>
<li>Intel</li>
<li>Palantir</li>
<li>Pepperdata</li>
<li>Red Hat</li>
</ul>
2017-09-27 21:04:05 +08:00
<h2 id="&#x4E3A;&#x4F55;&#x4F7F;&#x7528;-spark-on-kubernetes">&#x4E3A;&#x4F55;&#x4F7F;&#x7528; spark on kubernetes</h2>
<p>&#x4F7F;&#x7528;kubernetes&#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684;spark on kubernetes&#x662F;&#x5BF9;&#x73B0;&#x6709;&#x7684;spark on yarn/mesos&#x7684;&#x8D44;&#x6E90;&#x4F7F;&#x7528;&#x65B9;&#x5F0F;&#x7684;&#x9769;&#x547D;&#x6027;&#x7684;&#x6539;&#x8FDB;&#xFF0C;&#x4E3B;&#x8981;&#x8868;&#x73B0;&#x5728;&#x4EE5;&#x4E0B;&#x51E0;&#x70B9;&#xFF1A;</p>
<ol>
<li>Kubernetes&#x539F;&#x751F;&#x8C03;&#x5EA6;&#xFF1A;&#x4E0D;&#x518D;&#x9700;&#x8981;&#x4E8C;&#x5C42;&#x8C03;&#x5EA6;&#xFF0C;&#x76F4;&#x63A5;&#x4F7F;&#x7528;kubernetes&#x7684;&#x8D44;&#x6E90;&#x8C03;&#x5EA6;&#x529F;&#x80FD;&#xFF0C;&#x8DDF;&#x5176;&#x4ED6;&#x5E94;&#x7528;&#x5171;&#x7528;&#x6574;&#x4E2A;kubernetes&#x7BA1;&#x7406;&#x7684;&#x8D44;&#x6E90;&#x6C60;&#xFF1B;</li>
<li>&#x8D44;&#x6E90;&#x9694;&#x79BB;&#xFF0C;&#x7C92;&#x5EA6;&#x66F4;&#x7EC6;&#xFF1A;&#x539F;&#x5148;yarn&#x4E2D;&#x7684;queue&#x5728;spark on kubernetes&#x4E2D;&#x5DF2;&#x4E0D;&#x5B58;&#x5728;&#xFF0C;&#x53D6;&#x800C;&#x4EE3;&#x4E4B;&#x7684;&#x662F;kubernetes&#x4E2D;&#x539F;&#x751F;&#x7684;namespace&#xFF0C;&#x53EF;&#x4EE5;&#x4E3A;&#x6BCF;&#x4E2A;&#x7528;&#x6237;&#x5206;&#x522B;&#x6307;&#x5B9A;&#x4E00;&#x4E2A;namespace&#xFF0C;&#x9650;&#x5236;&#x7528;&#x6237;&#x7684;&#x8D44;&#x6E90;quota&#xFF1B;</li>
<li>&#x7EC6;&#x7C92;&#x5EA6;&#x7684;&#x8D44;&#x6E90;&#x5206;&#x914D;&#xFF1A;&#x53EF;&#x4EE5;&#x7ED9;&#x6BCF;&#x4E2A;spark&#x4EFB;&#x52A1;&#x6307;&#x5B9A;&#x8D44;&#x6E90;&#x9650;&#x5236;&#xFF0C;&#x5B9E;&#x9645;&#x6307;&#x5B9A;&#x591A;&#x5C11;&#x8D44;&#x6E90;&#x5C31;&#x4F7F;&#x7528;&#x591A;&#x5C11;&#x8D44;&#x6E90;&#xFF0C;&#x56E0;&#x4E3A;&#x6CA1;&#x6709;&#x4E86;&#x50CF;yarn&#x90A3;&#x6837;&#x7684;&#x4E8C;&#x5C42;&#x8C03;&#x5EA6;&#xFF08;&#x5708;&#x5730;&#x5F0F;&#x7684;&#xFF09;&#xFF0C;&#x6240;&#x4EE5;&#x53EF;&#x4EE5;&#x66F4;&#x9AD8;&#x6548;&#x548C;&#x7EC6;&#x7C92;&#x5EA6;&#x7684;&#x4F7F;&#x7528;&#x8D44;&#x6E90;&#xFF1B;</li>
<li>&#x76D1;&#x63A7;&#x7684;&#x53D8;&#x9769;&#xFF1A;&#x56E0;&#x4E3A;&#x505A;&#x5230;&#x4E86;&#x7EC6;&#x7C92;&#x5EA6;&#x7684;&#x8D44;&#x6E90;&#x5206;&#x914D;&#xFF0C;&#x6240;&#x4EE5;&#x53EF;&#x4EE5;&#x5BF9;&#x7528;&#x6237;&#x63D0;&#x4EA4;&#x7684;&#x6BCF;&#x4E00;&#x4E2A;&#x4EFB;&#x52A1;&#x505A;&#x5230;&#x8D44;&#x6E90;&#x4F7F;&#x7528;&#x7684;&#x76D1;&#x63A7;&#xFF0C;&#x4ECE;&#x800C;&#x5224;&#x65AD;&#x7528;&#x6237;&#x7684;&#x8D44;&#x6E90;&#x4F7F;&#x7528;&#x60C5;&#x51B5;&#xFF0C;&#x6240;&#x6709;&#x7684;metric&#x90FD;&#x8BB0;&#x5F55;&#x5728;&#x6570;&#x636E;&#x5E93;&#x4E2D;&#xFF0C;&#x751A;&#x81F3;&#x53EF;&#x4EE5;&#x4E3A;&#x6BCF;&#x4E2A;&#x7528;&#x6237;&#x7684;&#x6BCF;&#x6B21;&#x4EFB;&#x52A1;&#x63D0;&#x4EA4;&#x8BA1;&#x91CF;&#xFF1B;</li>
<li>&#x65E5;&#x5FD7;&#x7684;&#x53D8;&#x9769;&#xFF1A;&#x7528;&#x6237;&#x4E0D;&#x518D;&#x901A;&#x8FC7;yarn&#x7684;web&#x9875;&#x9762;&#x6765;&#x67E5;&#x770B;&#x4EFB;&#x52A1;&#x72B6;&#x6001;&#xFF0C;&#x800C;&#x662F;&#x901A;&#x8FC7;pod&#x7684;log&#x6765;&#x67E5;&#x770B;&#xFF0C;&#x53EF;&#x5C06;&#x6240;&#x6709;&#x7684;kuberentes&#x4E2D;&#x7684;&#x5E94;&#x7528;&#x7684;&#x65E5;&#x5FD7;&#x7B49;&#x540C;&#x770B;&#x5F85;&#x6536;&#x96C6;&#x8D77;&#x6765;&#xFF0C;&#x7136;&#x540E;&#x53EF;&#x4EE5;&#x6839;&#x636E;&#x6807;&#x7B7E;&#x67E5;&#x770B;&#x5BF9;&#x5E94;&#x5E94;&#x7528;&#x7684;&#x65E5;&#x5FD7;&#xFF1B;</li>
</ol>
<p>&#x6240;&#x6709;&#x8FD9;&#x4E9B;&#x53D8;&#x9769;&#x90FD;&#x53EF;&#x4EE5;&#x8BA9;&#x6211;&#x4EEC;&#x66F4;&#x9AD8;&#x6548;&#x7684;&#x83B7;&#x53D6;&#x8D44;&#x6E90;&#x3001;&#x66F4;&#x6709;&#x6548;&#x7387;&#x7684;&#x83B7;&#x53D6;&#x8D44;&#x6E90;&#xFF01;</p>
2017-09-14 15:57:50 +08:00
<h2 id="spark-&#x6982;&#x5FF5;&#x8BF4;&#x660E;">Spark &#x6982;&#x5FF5;&#x8BF4;&#x660E;</h2>
<p><a href="http://spark.apache.org" target="_blank">Apache Spark</a> &#x662F;&#x4E00;&#x4E2A;&#x56F4;&#x7ED5;&#x901F;&#x5EA6;&#x3001;&#x6613;&#x7528;&#x6027;&#x548C;&#x590D;&#x6742;&#x5206;&#x6790;&#x6784;&#x5EFA;&#x7684;&#x5927;&#x6570;&#x636E;&#x5904;&#x7406;&#x6846;&#x67B6;&#x3002;&#x6700;&#x521D;&#x5728;2009&#x5E74;&#x7531;&#x52A0;&#x5DDE;&#x5927;&#x5B66;&#x4F2F;&#x514B;&#x5229;&#x5206;&#x6821;&#x7684;AMPLab&#x5F00;&#x53D1;&#xFF0C;&#x5E76;&#x4E8E;2010&#x5E74;&#x6210;&#x4E3A;Apache&#x7684;&#x5F00;&#x6E90;&#x9879;&#x76EE;&#x4E4B;&#x4E00;&#x3002;</p>
<p>&#x5728; Spark &#x4E2D;&#x5305;&#x62EC;&#x5982;&#x4E0B;&#x7EC4;&#x4EF6;&#x6216;&#x6982;&#x5FF5;&#xFF1A;</p>
<ul>
<li><strong>Application</strong>&#xFF1A;Spark Application &#x7684;&#x6982;&#x5FF5;&#x548C; Hadoop &#x4E2D;&#x7684; MapReduce &#x7C7B;&#x4F3C;&#xFF0C;&#x6307;&#x7684;&#x662F;&#x7528;&#x6237;&#x7F16;&#x5199;&#x7684; Spark &#x5E94;&#x7528;&#x7A0B;&#x5E8F;&#xFF0C;&#x5305;&#x542B;&#x4E86;&#x4E00;&#x4E2A; Driver &#x529F;&#x80FD;&#x7684;&#x4EE3;&#x7801;&#x548C;&#x5206;&#x5E03;&#x5728;&#x96C6;&#x7FA4;&#x4E2D;&#x591A;&#x4E2A;&#x8282;&#x70B9;&#x4E0A;&#x8FD0;&#x884C;&#x7684; Executor &#x4EE3;&#x7801;&#xFF1B;</li>
<li><strong>Driver</strong>&#xFF1A;Spark &#x4E2D;&#x7684; Driver &#x5373;&#x8FD0;&#x884C;&#x4E0A;&#x8FF0; Application &#x7684; main() &#x51FD;&#x6570;&#x5E76;&#x4E14;&#x521B;&#x5EFA; SparkContext&#xFF0C;&#x5176;&#x4E2D;&#x521B;&#x5EFA; SparkContext &#x7684;&#x76EE;&#x7684;&#x662F;&#x4E3A;&#x4E86;&#x51C6;&#x5907;Spark&#x5E94;&#x7528;&#x7A0B;&#x5E8F;&#x7684;&#x8FD0;&#x884C;&#x73AF;&#x5883;&#x3002;&#x5728; Spark &#x4E2D;&#x7531; SparkContext &#x8D1F;&#x8D23;&#x548C; ClusterManager &#x901A;&#x4FE1;&#xFF0C;&#x8FDB;&#x884C;&#x8D44;&#x6E90;&#x7684;&#x7533;&#x8BF7;&#x3001;&#x4EFB;&#x52A1;&#x7684;&#x5206;&#x914D;&#x548C;&#x76D1;&#x63A7;&#x7B49;&#xFF1B;&#x5F53; Executor &#x90E8;&#x5206;&#x8FD0;&#x884C;&#x5B8C;&#x6BD5;&#x540E;&#xFF0C;Driver&#x8D1F;&#x8D23;&#x5C06;SparkContext &#x5173;&#x95ED;&#x3002;&#x901A;&#x5E38;&#x7528; SparkContext &#x4EE3;&#x8868; Driver&#xFF1B;</li>
<li><strong>Executor</strong>&#xFF1A;Application&#x8FD0;&#x884C;&#x5728;Worker &#x8282;&#x70B9;&#x4E0A;&#x7684;&#x4E00;&#x4E2A;&#x8FDB;&#x7A0B;&#xFF0C;&#x8BE5;&#x8FDB;&#x7A0B;&#x8D1F;&#x8D23;&#x8FD0;&#x884C;Task&#xFF0C;&#x5E76;&#x4E14;&#x8D1F;&#x8D23;&#x5C06;&#x6570;&#x636E;&#x5B58;&#x5728;&#x5185;&#x5B58;&#x6216;&#x8005;&#x78C1;&#x76D8;&#x4E0A;&#xFF0C;&#x6BCF;&#x4E2A;Application&#x90FD;&#x6709;&#x5404;&#x81EA;&#x72EC;&#x7ACB;&#x7684;&#x4E00;&#x6279;Executor&#x3002;&#x5728;Spark on Yarn&#x6A21;&#x5F0F;&#x4E0B;&#xFF0C;&#x5176;&#x8FDB;&#x7A0B;&#x540D;&#x79F0;&#x4E3A;<code>CoarseGrainedExecutorBackend</code>&#xFF0C;&#x7C7B;&#x4F3C;&#x4E8E; Hadoop MapReduce &#x4E2D;&#x7684; YarnChild&#x3002;&#x4E00;&#x4E2A; <code>CoarseGrainedExecutorBackend</code> &#x8FDB;&#x7A0B;&#x6709;&#x4E14;&#x4EC5;&#x6709;&#x4E00;&#x4E2A; executor &#x5BF9;&#x8C61;&#xFF0C;&#x5B83;&#x8D1F;&#x8D23;&#x5C06; Task &#x5305;&#x88C5;&#x6210; taskRunner&#xFF0C;&#x5E76;&#x4ECE;&#x7EBF;&#x7A0B;&#x6C60;&#x4E2D;&#x62BD;&#x53D6;&#x51FA;&#x4E00;&#x4E2A;&#x7A7A;&#x95F2;&#x7EBF;&#x7A0B;&#x8FD0;&#x884C; Task&#x3002;&#x6BCF;&#x4E2A; <code>CoarseGrainedExecutorBackend</code> &#x80FD;&#x5E76;&#x884C;&#x8FD0;&#x884C; Task &#x7684;&#x6570;&#x91CF;&#x5C31;&#x53D6;&#x51B3;&#x4E8E;&#x5206;&#x914D;&#x7ED9;&#x5B83;&#x7684; CPU &#x7684;&#x4E2A;&#x6570;&#x4E86;&#xFF1B;</li>
<li><strong>Cluster Manager</strong>&#xFF1A;&#x6307;&#x7684;&#x662F;&#x5728;&#x96C6;&#x7FA4;&#x4E0A;&#x83B7;&#x53D6;&#x8D44;&#x6E90;&#x7684;&#x5916;&#x90E8;&#x670D;&#x52A1;&#xFF0C;&#x76EE;&#x524D;&#x6709;&#xFF1A;<ul>
<li>Standalone&#xFF1A;Spark&#x539F;&#x751F;&#x7684;&#x8D44;&#x6E90;&#x7BA1;&#x7406;&#xFF0C;&#x7531;Master&#x8D1F;&#x8D23;&#x8D44;&#x6E90;&#x7684;&#x5206;&#x914D;&#xFF1B;</li>
<li>Hadoop Yarn&#xFF1A;&#x7531;YARN&#x4E2D;&#x7684;ResourceManager&#x8D1F;&#x8D23;&#x8D44;&#x6E90;&#x7684;&#x5206;&#x914D;&#xFF1B;</li>
</ul>
</li>
<li><strong>Worker</strong>&#xFF1A;&#x96C6;&#x7FA4;&#x4E2D;&#x4EFB;&#x4F55;&#x53EF;&#x4EE5;&#x8FD0;&#x884C;Application&#x4EE3;&#x7801;&#x7684;&#x8282;&#x70B9;&#xFF0C;&#x7C7B;&#x4F3C;&#x4E8E;YARN&#x4E2D;&#x7684;NodeManager&#x8282;&#x70B9;&#x3002;&#x5728;Standalone&#x6A21;&#x5F0F;&#x4E2D;&#x6307;&#x7684;&#x5C31;&#x662F;&#x901A;&#x8FC7;Slave&#x6587;&#x4EF6;&#x914D;&#x7F6E;&#x7684;Worker&#x8282;&#x70B9;&#xFF0C;&#x5728;Spark on Yarn&#x6A21;&#x5F0F;&#x4E2D;&#x6307;&#x7684;&#x5C31;&#x662F;NodeManager&#x8282;&#x70B9;&#xFF1B;</li>
<li><strong>&#x4F5C;&#x4E1A;&#xFF08;Job&#xFF09;</strong>&#xFF1A;&#x5305;&#x542B;&#x591A;&#x4E2A;Task&#x7EC4;&#x6210;&#x7684;&#x5E76;&#x884C;&#x8BA1;&#x7B97;&#xFF0C;&#x5F80;&#x5F80;&#x7531;Spark Action&#x50AC;&#x751F;&#xFF0C;&#x4E00;&#x4E2A;JOB&#x5305;&#x542B;&#x591A;&#x4E2A;RDD&#x53CA;&#x4F5C;&#x7528;&#x4E8E;&#x76F8;&#x5E94;RDD&#x4E0A;&#x7684;&#x5404;&#x79CD;Operation&#xFF1B;</li>
<li><strong>&#x9636;&#x6BB5;&#xFF08;Stage&#xFF09;</strong>&#xFF1A;&#x6BCF;&#x4E2A;Job&#x4F1A;&#x88AB;&#x62C6;&#x5206;&#x5F88;&#x591A;&#x7EC4; Task&#xFF0C;&#x6BCF;&#x7EC4;&#x4EFB;&#x52A1;&#x88AB;&#x79F0;&#x4E3A;Stage&#xFF0C;&#x4E5F;&#x53EF;&#x79F0;TaskSet&#xFF0C;&#x4E00;&#x4E2A;&#x4F5C;&#x4E1A;&#x5206;&#x4E3A;&#x591A;&#x4E2A;&#x9636;&#x6BB5;&#xFF0C;&#x6BCF;&#x4E00;&#x4E2A;stage&#x7684;&#x5206;&#x5272;&#x70B9;&#x662F;action&#x3002;&#x6BD4;&#x5982;&#x4E00;&#x4E2A;job&#x662F;&#xFF1A;&#xFF08;transformation1 -&gt; transformation1 -&gt; action1 -&gt; transformation3 -&gt; action2&#xFF09;&#xFF0C;&#x8FD9;&#x4E2A;job&#x5C31;&#x4F1A;&#x88AB;&#x5206;&#x4E3A;&#x4E24;&#x4E2A;stage&#xFF0C;&#x5206;&#x5272;&#x70B9;&#x662F;action1&#x548C;action2&#x3002;</li>
<li><p><strong>&#x4EFB;&#x52A1;&#xFF08;Task&#xFF09;</strong>&#xFF1A; &#x88AB;&#x9001;&#x5230;&#x67D0;&#x4E2A;Executor&#x4E0A;&#x7684;&#x5DE5;&#x4F5C;&#x4EFB;&#x52A1;&#xFF1B;</p>
</li>
<li><p><strong>Context</strong>&#xFF1A;&#x542F;&#x52A8;spark application&#x7684;&#x65F6;&#x5019;&#x521B;&#x5EFA;&#xFF0C;&#x4F5C;&#x4E3A;Spark &#x8FD0;&#x884C;&#x65F6;&#x73AF;&#x5883;&#x3002;</p>
</li>
<li><strong>Dynamic Allocation&#xFF08;&#x52A8;&#x6001;&#x8D44;&#x6E90;&#x5206;&#x914D;&#xFF09;</strong>&#xFF1A;&#x4E00;&#x4E2A;&#x914D;&#x7F6E;&#x9009;&#x9879;&#xFF0C;&#x53EF;&#x4EE5;&#x5C06;&#x5176;&#x6253;&#x5F00;&#x3002;&#x4ECE;Spark1.2&#x4E4B;&#x540E;&#xFF0C;&#x5BF9;&#x4E8E;On Yarn&#x6A21;&#x5F0F;&#xFF0C;&#x5DF2;&#x7ECF;&#x652F;&#x6301;&#x52A8;&#x6001;&#x8D44;&#x6E90;&#x5206;&#x914D;&#xFF08;Dynamic Resource Allocation&#xFF09;&#xFF0C;&#x8FD9;&#x6837;&#xFF0C;&#x5C31;&#x53EF;&#x4EE5;&#x6839;&#x636E;Application&#x7684;&#x8D1F;&#x8F7D;&#xFF08;Task&#x60C5;&#x51B5;&#xFF09;&#xFF0C;&#x52A8;&#x6001;&#x7684;&#x589E;&#x52A0;&#x548C;&#x51CF;&#x5C11;executors&#xFF0C;&#x8FD9;&#x79CD;&#x7B56;&#x7565;&#x975E;&#x5E38;&#x9002;&#x5408;&#x5728;YARN&#x4E0A;&#x4F7F;&#x7528;spark-sql&#x505A;&#x6570;&#x636E;&#x5F00;&#x53D1;&#x548C;&#x5206;&#x6790;&#xFF0C;&#x4EE5;&#x53CA;&#x5C06;spark-sql&#x4F5C;&#x4E3A;&#x957F;&#x670D;&#x52A1;&#x6765;&#x4F7F;&#x7528;&#x7684;&#x573A;&#x666F;&#x3002;Executor &#x7684;&#x52A8;&#x6001;&#x5206;&#x914D;&#x9700;&#x8981;&#x5728; cluster mode &#x4E0B;&#x542F;&#x7528; &quot;external shuffle service&quot;&#x3002;</li>
<li><strong>&#x52A8;&#x6001;&#x8D44;&#x6E90;&#x5206;&#x914D;&#x7B56;&#x7565;</strong>&#xFF1A;&#x5F00;&#x542F;&#x52A8;&#x6001;&#x5206;&#x914D;&#x7B56;&#x7565;&#x540E;&#xFF0C;application&#x4F1A;&#x5728;task&#x56E0;&#x6CA1;&#x6709;&#x8DB3;&#x591F;&#x8D44;&#x6E90;&#x88AB;&#x6302;&#x8D77;&#x7684;&#x65F6;&#x5019;&#x53BB;&#x52A8;&#x6001;&#x7533;&#x8BF7;&#x8D44;&#x6E90;&#xFF0C;&#x8FD9;&#x610F;&#x5473;&#x7740;&#x8BE5;application&#x73B0;&#x6709;&#x7684;executor&#x65E0;&#x6CD5;&#x6EE1;&#x8DB3;&#x6240;&#x6709;task&#x5E76;&#x884C;&#x8FD0;&#x884C;&#x3002;spark&#x4E00;&#x8F6E;&#x4E00;&#x8F6E;&#x7684;&#x7533;&#x8BF7;&#x8D44;&#x6E90;&#xFF0C;&#x5F53;&#x6709;task&#x6302;&#x8D77;&#x6216;&#x7B49;&#x5F85; <code>spark.dynamicAllocation.schedulerBacklogTimeout</code> (&#x9ED8;&#x8BA4;1s)&#x65F6;&#x95F4;&#x7684;&#x65F6;&#x5019;&#xFF0C;&#x4F1A;&#x5F00;&#x59CB;&#x52A8;&#x6001;&#x8D44;&#x6E90;&#x5206;&#x914D;&#xFF1B;&#x4E4B;&#x540E;&#x4F1A;&#x6BCF;&#x9694; <code>spark.dynamicAllocation.sustainedSchedulerBacklogTimeout</code> (&#x9ED8;&#x8BA4;1s)&#x65F6;&#x95F4;&#x7533;&#x8BF7;&#x4E00;&#x6B21;&#xFF0C;&#x76F4;&#x5230;&#x7533;&#x8BF7;&#x5230;&#x8DB3;&#x591F;&#x7684;&#x8D44;&#x6E90;&#x3002;&#x6BCF;&#x6B21;&#x7533;&#x8BF7;&#x7684;&#x8D44;&#x6E90;&#x91CF;&#x662F;&#x6307;&#x6570;&#x589E;&#x957F;&#x7684;&#xFF0C;&#x5373;1,2,4,8&#x7B49;&#x3002;&#x4E4B;&#x6240;&#x4EE5;&#x91C7;&#x7528;&#x6307;&#x6570;&#x589E;&#x957F;&#xFF0C;&#x51FA;&#x4E8E;&#x4E24;&#x65B9;&#x9762;&#x8003;&#x8651;&#xFF1A;&#x5176;&#x4E00;&#xFF0C;&#x5F00;&#x59CB;&#x7533;&#x8BF7;&#x7684;&#x5C11;&#x662F;&#x8003;&#x8651;&#x5230;&#x53EF;&#x80FD;application&#x4F1A;&#x9A6C;&#x4E0A;&#x5F97;&#x5230;&#x6EE1;&#x8DB3;&#xFF1B;&#x5176;&#x6B21;&#x8981;&#x6210;&#x500D;&#x589E;&#x52A0;&#xFF0C;&#x662F;&#x4E3A;&#x4E86;&#x9632;&#x6B62;application&#x9700;&#x8981;&#x5F88;&#x591A;&#x8D44;&#x6E90;&#xFF0C;&#x800C;&#x8BE5;&#x65B9;&#x5F0F;&#x53EF;&#x4EE5;&#x5728;&#x5F88;&#x5C11;&#x6B21;&#x6570;&#x7684;&#x7533;&#x8BF7;&#x4E4B;&#x540E;&#x5F97;&#x5230;&#x6EE1;&#x8DB3;&#x3002;</li>
</ul>
<h2 id="&#x67B6;&#x6784;&#x8BBE;&#x8BA1;">&#x67B6;&#x6784;&#x8BBE;&#x8BA1;</h2>
<p>&#x5173;&#x4E8E; spark standalone &#x7684;&#x5C40;&#x9650;&#x6027;&#x4E0E; kubernetes native spark &#x67B6;&#x6784;&#x4E4B;&#x95F4;&#x7684;&#x533A;&#x522B;&#x8BF7;&#x53C2;&#x8003; Anirudh Ramanathan &#x5728; 2016&#x5E74;10&#x6708;8&#x65E5;&#x63D0;&#x4EA4;&#x7684; issue <a href="https://github.com/kubernetes/kubernetes/issues/34377" target="_blank">Support Spark natively in Kubernetes #34377</a>&#x3002;</p>
<p>&#x7B80;&#x800C;&#x8A00;&#x4E4B;&#xFF0C;spark standalone on kubernetes &#x6709;&#x5982;&#x4E0B;&#x51E0;&#x4E2A;&#x7F3A;&#x70B9;&#xFF1A;</p>
<ul>
<li>&#x65E0;&#x6CD5;&#x5BF9;&#x4E8E;&#x591A;&#x79DF;&#x6237;&#x505A;&#x9694;&#x79BB;&#xFF0C;&#x6BCF;&#x4E2A;&#x7528;&#x6237;&#x90FD;&#x60F3;&#x7ED9; pod &#x7533;&#x8BF7; node &#x8282;&#x70B9;&#x53EF;&#x7528;&#x7684;&#x6700;&#x5927;&#x7684;&#x8D44;&#x6E90;&#x3002;</li>
<li>Spark &#x7684; master&#xFF0F;worker &#x672C;&#x6765;&#x4E0D;&#x662F;&#x8BBE;&#x8BA1;&#x6210;&#x4F7F;&#x7528; kubernetes &#x7684;&#x8D44;&#x6E90;&#x8C03;&#x5EA6;&#xFF0C;&#x8FD9;&#x6837;&#x4F1A;&#x5B58;&#x5728;&#x4E24;&#x5C42;&#x7684;&#x8D44;&#x6E90;&#x8C03;&#x5EA6;&#x95EE;&#x9898;&#xFF0C;&#x4E0D;&#x5229;&#x4E8E;&#x4E0E; kuberentes &#x96C6;&#x6210;&#x3002;</li>
</ul>
<p>&#x800C; kubernetes native spark &#x96C6;&#x7FA4;&#x4E2D;&#xFF0C;spark &#x53EF;&#x4EE5;&#x8C03;&#x7528; kubernetes API &#x83B7;&#x53D6;&#x96C6;&#x7FA4;&#x8D44;&#x6E90;&#x548C;&#x8C03;&#x5EA6;&#x3002;&#x8981;&#x5B9E;&#x73B0; kubernetes native spark &#x9700;&#x8981;&#x4E3A; spark &#x63D0;&#x4F9B;&#x4E00;&#x4E2A;&#x96C6;&#x7FA4;&#x5916;&#x90E8;&#x7684; manager &#x53EF;&#x4EE5;&#x7528;&#x6765;&#x8DDF; kubernetes API &#x4EA4;&#x4E92;&#x3002;</p>
2017-09-19 19:26:13 +08:00
<h3 id="&#x8C03;&#x5EA6;&#x5668;&#x540E;&#x53F0;">&#x8C03;&#x5EA6;&#x5668;&#x540E;&#x53F0;</h3>
<p>&#x4F7F;&#x7528; kubernetes &#x539F;&#x751F;&#x8C03;&#x5EA6;&#x7684; spark &#x7684;&#x57FA;&#x672C;&#x8BBE;&#x8BA1;&#x601D;&#x8DEF;&#x662F;&#x5C06; spark &#x7684; driver &#x548C; executor &#x90FD;&#x653E;&#x5728; kubernetes &#x7684; pod &#x4E2D;&#x8FD0;&#x884C;&#xFF0C;&#x53E6;&#x5916;&#x8FD8;&#x6709;&#x4E24;&#x4E2A;&#x9644;&#x52A0;&#x7684;&#x7EC4;&#x4EF6;&#xFF1A;<code>ResourceStagingServer</code> &#x548C; <code>KubernetesExternalShuffleService</code>&#x3002;</p>
<p>Spark driver &#x5176;&#x5B9E;&#x53EF;&#x4EE5;&#x8FD0;&#x884C;&#x5728; kubernetes &#x96C6;&#x7FA4;&#x5185;&#x90E8;&#xFF08;cluster mode&#xFF09;&#x53EF;&#x4EE5;&#x8FD0;&#x884C;&#x5728;&#x5916;&#x90E8;&#xFF08;client mode&#xFF09;&#xFF0C;executor &#x53EA;&#x80FD;&#x8FD0;&#x884C;&#x5728;&#x96C6;&#x7FA4;&#x5185;&#x90E8;&#xFF0C;&#x5F53;&#x6709; spark &#x4F5C;&#x4E1A;&#x63D0;&#x4EA4;&#x5230; kubernetes &#x96C6;&#x7FA4;&#x4E0A;&#x65F6;&#xFF0C;&#x8C03;&#x5EA6;&#x5668;&#x540E;&#x53F0;&#x5C06;&#x4F1A;&#x4E3A; executor pod &#x8BBE;&#x7F6E;&#x5982;&#x4E0B;&#x5C5E;&#x6027;&#xFF1A;</p>
<ul>
<li>&#x4F7F;&#x7528;&#x6211;&#x4EEC;&#x9884;&#x5148;&#x7F16;&#x8BD1;&#x597D;&#x7684;&#x5305;&#x542B; kubernetes &#x652F;&#x6301;&#x7684; spark &#x955C;&#x50CF;&#xFF0C;&#x7136;&#x540E;&#x8C03;&#x7528; <code>CoarseGrainedExecutorBackend</code> main class &#x542F;&#x52A8; JVM&#x3002;</li>
<li>&#x8C03;&#x5EA6;&#x5668;&#x540E;&#x53F0;&#x4E3A; executor pod &#x7684;&#x8FD0;&#x884C;&#x65F6;&#x6CE8;&#x5165;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#xFF0C;&#x4F8B;&#x5982;&#x5404;&#x79CD; JVM &#x53C2;&#x6570;&#xFF0C;&#x5305;&#x62EC;&#x7528;&#x6237;&#x5728; <code>spark-submit</code> &#x65F6;&#x6307;&#x5B9A;&#x7684;&#x90A3;&#x4E9B;&#x53C2;&#x6570;&#x3002;</li>
<li>Executor &#x7684; CPU&#x3001;&#x5185;&#x5B58;&#x9650;&#x5236;&#x6839;&#x636E;&#x8FD9;&#x4E9B;&#x6CE8;&#x5165;&#x7684;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#x4FDD;&#x5B58;&#x5230;&#x5E94;&#x7528;&#x7A0B;&#x5E8F;&#x7684; <code>SparkConf</code> &#x4E2D;&#x3002;</li>
<li>&#x53EF;&#x4EE5;&#x5728;&#x914D;&#x7F6E;&#x4E2D;&#x6307;&#x5B9A; spark &#x8FD0;&#x884C;&#x5728;&#x6307;&#x5B9A;&#x7684; namespace &#x4E2D;&#x3002;</li>
</ul>
<p>&#x53C2;&#x8003;&#xFF1A;<a href="https://github.com/apache-spark-on-k8s/spark/blob/branch-2.2-kubernetes/resource-managers/kubernetes/architecture-docs/scheduler-backend.md" target="_blank">Scheduler backend &#x6587;&#x6863;</a></p>
2017-09-14 15:57:50 +08:00
<h2 id="&#x5B89;&#x88C5;&#x6307;&#x5357;">&#x5B89;&#x88C5;&#x6307;&#x5357;</h2>
<p>&#x6211;&#x4EEC;&#x53EF;&#x4EE5;&#x76F4;&#x63A5;&#x4F7F;&#x7528;&#x5B98;&#x65B9;&#x5DF2;&#x7F16;&#x8BD1;&#x597D;&#x7684; docker &#x955C;&#x50CF;&#x6765;&#x90E8;&#x7F72;&#xFF0C;&#x4E0B;&#x9762;&#x662F;&#x5B98;&#x65B9;&#x53D1;&#x5E03;&#x7684;&#x955C;&#x50CF;&#xFF1A;</p>
<table>
<thead>
<tr>
<th>&#x7EC4;&#x4EF6;</th>
<th>&#x955C;&#x50CF;</th>
</tr>
</thead>
<tbody>
<tr>
<td>Spark Driver Image</td>
<td><code>kubespark/spark-driver:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
<tr>
<td>Spark Executor Image</td>
<td><code>kubespark/spark-executor:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
<tr>
<td>Spark Initialization Image</td>
<td><code>kubespark/spark-init:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
<tr>
<td>Spark Staging Server Image</td>
<td><code>kubespark/spark-resource-staging-server:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
<tr>
<td>PySpark Driver Image</td>
<td><code>kubespark/driver-py:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
<tr>
<td>PySpark Executor Image</td>
<td><code>kubespark/executor-py:v2.1.0-kubernetes-0.3.1</code></td>
</tr>
</tbody>
</table>
<p>&#x6211;&#x5C06;&#x8FD9;&#x4E9B;&#x955C;&#x50CF;&#x653E;&#x5230;&#x4E86;&#x6211;&#x7684;&#x79C1;&#x6709;&#x955C;&#x50CF;&#x4ED3;&#x5E93;&#x4E2D;&#x4E86;&#x3002;</p>
<p>&#x8FD8;&#x9700;&#x8981;&#x5B89;&#x88C5;&#x652F;&#x6301; kubernetes &#x7684; spark &#x5BA2;&#x6237;&#x7AEF;&#xFF0C;&#x5728;&#x8FD9;&#x91CC;&#x4E0B;&#x8F7D;&#xFF1A;<a href="https://github.com/apache-spark-on-k8s/spark/releases" target="_blank">https://github.com/apache-spark-on-k8s/spark/releases</a></p>
<p>&#x6839;&#x636E;&#x4F7F;&#x7528;&#x7684;&#x955C;&#x50CF;&#x7248;&#x672C;&#xFF0C;&#x6211;&#x4E0B;&#x8F7D;&#x7684;&#x662F; <a href="https://github.com/apache-spark-on-k8s/spark/releases/tag/v2.1.0-kubernetes-0.3.1" target="_blank">v2.1.0-kubernetes-0.3.1</a> </p>
<p><strong>&#x8FD0;&#x884C; SparkPi &#x6D4B;&#x8BD5;</strong></p>
2017-09-15 19:36:28 +08:00
<p>&#x6211;&#x4EEC;&#x5C06;&#x4EFB;&#x52A1;&#x8FD0;&#x884C;&#x5728; <code>spark-cluster</code> &#x7684; namespace &#x4E2D;&#xFF0C;&#x542F;&#x52A8; 5 &#x4E2A; executor &#x5B9E;&#x4F8B;&#x3002;</p>
2017-09-14 15:57:50 +08:00
<pre><code class="lang-bash">./bin/spark-submit \
--deploy-mode cluster \
--class org.apache.spark.examples.SparkPi \
--master k8s://https://172.20.0.113:6443 \
--kubernetes-namespace spark-cluster \
--conf spark.executor.instances=5 \
--conf spark.app.name=spark-pi \
--conf spark.kubernetes.driver.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver:v2.1.0-kubernetes-0.3.1 \
--conf spark.kubernetes.executor.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-executor:v2.1.0-kubernetes-0.3.1 \
--conf spark.kubernetes.initcontainer.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-init:v2.1.0-kubernetes-0.3.1 \
<span class="hljs-built_in">local</span>:///opt/spark/examples/jars/spark-examples_2.11-2.1.0-k8s-0.3.1-SNAPSHOT.jar
</code></pre>
<p>&#x5173;&#x4E8E;&#x8BE5;&#x547D;&#x4EE4;&#x53C2;&#x6570;&#x7684;&#x4ECB;&#x7ECD;&#x8BF7;&#x53C2;&#x8003;&#xFF1A;<a href="https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html" target="_blank">https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html</a></p>
<p><strong>&#x6CE8;&#x610F;&#xFF1A;</strong> &#x8BE5; jar &#x5305;&#x5B9E;&#x9645;&#x4E0A;&#x662F; <code>spark.kubernetes.executor.docker.image</code> &#x955C;&#x50CF;&#x4E2D;&#x7684;&#x3002;</p>
<p>&#x8FD9;&#x65F6;&#x5019;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#x8FD0;&#x884C;&#x8FD8;&#x662F;&#x5931;&#x8D25;&#xFF0C;&#x62A5;&#x9519;&#x4FE1;&#x606F;&#x4E2D;&#x53EF;&#x4EE5;&#x770B;&#x5230;&#x4E24;&#x4E2A;&#x95EE;&#x9898;&#xFF1A;</p>
<ul>
<li>Executor &#x65E0;&#x6CD5;&#x627E;&#x5230; driver pod</li>
<li>&#x7528;&#x6237; <code>system:serviceaccount:spark-cluster:defaul</code> &#x6CA1;&#x6709;&#x6743;&#x9650;&#x83B7;&#x53D6; <code>spark-cluster</code> &#x4E2D;&#x7684; pod &#x4FE1;&#x606F;&#x3002;</li>
</ul>
<p>&#x63D0;&#x4E86;&#x4E2A; issue <a href="https://github.com/apache-spark-on-k8s/spark/issues/478" target="_blank">Failed to run the sample spark-pi test using spark-submit on the doc #478</a> </p>
<p>&#x9700;&#x8981;&#x4E3A; spark &#x96C6;&#x7FA4;&#x521B;&#x5EFA;&#x4E00;&#x4E2A; <code>serviceaccount</code> &#x548C; <code>clusterrolebinding</code>&#xFF1A;</p>
<pre><code class="lang-bash">kubectl create serviceaccount spark --namespace spark-cluster
kubectl create rolebinding spark-edit --clusterrole=edit --serviceaccount=spark-cluster:spark --namespace=spark-cluster
</code></pre>
<p>&#x8BE5; Bug &#x5C06;&#x5728;&#x65B0;&#x7248;&#x672C;&#x4E2D;&#x4FEE;&#x590D;&#x3002;</p>
2017-09-19 19:26:13 +08:00
<h2 id="&#x7528;&#x6237;&#x6307;&#x5357;">&#x7528;&#x6237;&#x6307;&#x5357;</h2>
2017-09-14 15:57:50 +08:00
<h3 id="&#x7F16;&#x8BD1;">&#x7F16;&#x8BD1;</h3>
<p>Fork &#x5E76;&#x514B;&#x9686;&#x9879;&#x76EE;&#x5230;&#x672C;&#x5730;&#xFF1A;</p>
<pre><code class="lang-bash">git <span class="hljs-built_in">clone</span> https://github.com/rootsongjc/spark.git
</code></pre>
<p>&#x7F16;&#x8BD1;&#x524D;&#x8BF7;&#x786E;&#x4FDD;&#x4F60;&#x7684;&#x73AF;&#x5883;&#x4E2D;&#x5DF2;&#x7ECF;&#x5B89;&#x88C5; Java8 &#x548C; Maven3&#x3002;</p>
<pre><code class="lang-bash"><span class="hljs-comment">## &#x7B2C;&#x4E00;&#x6B21;&#x7F16;&#x8BD1;&#x524D;&#x9700;&#x8981;&#x5B89;&#x88C5;&#x4F9D;&#x8D56;</span>
build/mvn install -Pkubernetes -pl resource-managers/kubernetes/core -am -DskipTests
<span class="hljs-comment">## &#x7F16;&#x8BD1; spark on kubernetes</span>
build/mvn compile -Pkubernetes -pl resource-managers/kubernetes/core -am -DskipTests
<span class="hljs-comment">## &#x53D1;&#x5E03;</span>
dev/make-distribution.sh --tgz -Phadoop-2.7 -Pkubernetes
</code></pre>
<p>&#x7B2C;&#x4E00;&#x6B21;&#x7F16;&#x8BD1;&#x548C;&#x53D1;&#x5E03;&#x7684;&#x8FC7;&#x7A0B;&#x8017;&#x65F6;&#x53EF;&#x80FD;&#x4F1A;&#x6BD4;&#x8F83;&#x957F;&#xFF0C;&#x8BF7;&#x8010;&#x5FC3;&#x7B49;&#x5F85;&#xFF0C;&#x5982;&#x679C;&#x6709;&#x4F9D;&#x8D56;&#x4E0B;&#x8F7D;&#x4E0D;&#x4E0B;&#x6765;&#xFF0C;&#x8BF7;&#x81EA;&#x5907;&#x68AF;&#x5B50;&#x3002;</p>
<p>&#x8BE6;&#x7EC6;&#x7684;&#x5F00;&#x53D1;&#x6307;&#x5357;&#x8BF7;&#x89C1;&#xFF1A;<a href="https://github.com/apache-spark-on-k8s/spark/blob/branch-2.2-kubernetes/resource-managers/kubernetes/README.md" target="_blank">https://github.com/apache-spark-on-k8s/spark/blob/branch-2.2-kubernetes/resource-managers/kubernetes/README.md</a></p>
<h3 id="&#x6784;&#x5EFA;&#x955C;&#x50CF;">&#x6784;&#x5EFA;&#x955C;&#x50CF;</h3>
<p>&#x4F7F;&#x7528;&#x8BE5;&#x811A;&#x672C;&#x6765;&#x81EA;&#x52A8;&#x6784;&#x5EFA;&#x5BB9;&#x5668;&#x955C;&#x50CF;&#xFF1A;<a href="https://github.com/apache-spark-on-k8s/spark/pull/488" target="_blank">https://github.com/apache-spark-on-k8s/spark/pull/488</a></p>
<p>&#x5C06;&#x8BE5;&#x811A;&#x672C;&#x653E;&#x5728; <code>dist</code> &#x76EE;&#x5F55;&#x4E0B;&#xFF0C;&#x6267;&#x884C;&#xFF1A;</p>
<pre><code class="lang-bash">./build-push-docker-images.sh -r sz-pg-oam-docker-hub-001.tendcloud.com/library -t v2.1.0-kubernetes-0.3.1-1 build
./build-push-docker-images.sh -r sz-pg-oam-docker-hub-001.tendcloud.com/library -t v2.1.0-kubernetes-0.3.1-1 push
</code></pre>
<p><strong>&#x6CE8;&#x610F;&#xFF1A;</strong>&#x5982;&#x679C;&#x4F60;&#x4F7F;&#x7528;&#x7684; MacOS&#xFF0C;bash &#x7684;&#x7248;&#x672C;&#x53EF;&#x80FD;&#x592A;&#x4F4E;&#xFF0C;&#x6267;&#x884C;&#x6539;&#x811A;&#x672C;&#x5C06;&#x51FA;&#x9519;&#xFF0C;&#x8BF7;&#x68C0;&#x67E5;&#x4F60;&#x7684; bash &#x7248;&#x672C;&#xFF1A;</p>
<pre><code class="lang-bash">bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16)
Copyright (C) 2007 Free Software Foundation, Inc.
</code></pre>
<p>&#x4E0A;&#x9762;&#x6211;&#x5728;&#x5347;&#x7EA7; bash &#x4E4B;&#x524D;&#x83B7;&#x53D6;&#x7684;&#x7248;&#x672C;&#x4FE1;&#x606F;&#xFF0C;&#x4F7F;&#x7528;&#x4E0B;&#x9762;&#x7684;&#x547D;&#x4EE4;&#x5347;&#x7EA7; bash&#xFF1A;</p>
<pre><code class="lang-bash">brew install bash
</code></pre>
<p>&#x5347;&#x7EA7;&#x540E;&#x7684; bash &#x7248;&#x672C;&#x4E3A; <code>4.4.12(1)-release (x86_64-apple-darwin16.3.0)</code>&#x3002;</p>
<p>&#x7F16;&#x8BD1;&#x5E76;&#x4E0A;&#x4F20;&#x955C;&#x50CF;&#x5230;&#x6211;&#x7684;&#x79C1;&#x6709;&#x955C;&#x50CF;&#x4ED3;&#x5E93;&#xFF0C;&#x5C06;&#x4F1A;&#x6784;&#x5EFA;&#x51FA;&#x5982;&#x4E0B;&#x51E0;&#x4E2A;&#x955C;&#x50CF;&#xFF1A;</p>
<pre><code class="lang-bash">sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-resource-staging-server:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-init:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-shuffle:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor-py:v2.1.0-kubernetes-0.3.1-1
sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver-py:v2.1.0-kubernetes-0.3.1-1
</code></pre>
<h2 id="&#x8FD0;&#x884C;&#x6D4B;&#x8BD5;">&#x8FD0;&#x884C;&#x6D4B;&#x8BD5;</h2>
<p>&#x5728; <code>dist/bin</code> &#x76EE;&#x5F55;&#x4E0B;&#x6267;&#x884C; spark-pi &#x6D4B;&#x8BD5;&#xFF1A;</p>
<pre><code class="lang-bash">./spark-submit \
--deploy-mode cluster \
--class org.apache.spark.examples.SparkPi \
--master k8s://https://172.20.0.113:6443 \
--kubernetes-namespace spark-cluster \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.executor.instances=5 \
--conf spark.app.name=spark-pi \
--conf spark.kubernetes.driver.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.executor.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.initcontainer.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-init:v2.1.0-kubernetes-0.3.1-1 \
<span class="hljs-built_in">local</span>:///opt/spark/examples/jars/spark-examples_2.11-2.2.0-k8s-0.4.0-SNAPSHOT.jar
</code></pre>
<p>&#x8BE6;&#x7EC6;&#x7684;&#x53C2;&#x6570;&#x8BF4;&#x660E;&#x89C1;&#xFF1A;<a href="https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html" target="_blank">https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html</a></p>
<p><strong>&#x6CE8;&#x610F;&#xFF1A;</strong><code>local:///opt/spark/examples/jars/spark-examples_2.11-2.2.0-k8s-0.4.0-SNAPSHOT.jar</code> &#x6587;&#x4EF6;&#x662F;&#x5728; <code>spark-driver</code> &#x548C; <code>spark-executor</code> &#x955C;&#x50CF;&#x91CC;&#x7684;&#xFF0C;&#x5728;&#x4E0A;&#x4E00;&#x6B65;&#x6784;&#x5EFA;&#x955C;&#x50CF;&#x65F6;&#x5DF2;&#x7ECF;&#x6784;&#x5EFA;&#x5E76;&#x4E0A;&#x4F20;&#x5230;&#x4E86;&#x955C;&#x50CF;&#x4ED3;&#x5E93;&#x4E2D;&#x3002;</p>
<p>&#x6267;&#x884C;&#x65E5;&#x5FD7;&#x663E;&#x793A;&#xFF1A;</p>
<pre><code class="lang-bash">2017-09-14 14:59:01 INFO Client:54 - Waiting <span class="hljs-keyword">for</span> application spark-pi to finish...
2017-09-14 14:59:01 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
pod name: spark-pi-1505372339796-driver
namespace: spark-cluster
labels: spark-app-selector -&gt; spark<span class="hljs-_">-f</span>4d3a5d3ad964a05a51feb6191d50357, spark-role -&gt; driver
pod uid: 304cf440-991a-11e7-970c<span class="hljs-_">-f</span>4e9d49f8ed0
creation time: 2017-09-14T06:59:01Z
service account name: spark
volumes: spark-token-zr8wv
node name: N/A
start time: N/A
container images: N/A
phase: Pending
status: []
2017-09-14 14:59:01 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
pod name: spark-pi-1505372339796-driver
namespace: spark-cluster
labels: spark-app-selector -&gt; spark<span class="hljs-_">-f</span>4d3a5d3ad964a05a51feb6191d50357, spark-role -&gt; driver
pod uid: 304cf440-991a-11e7-970c<span class="hljs-_">-f</span>4e9d49f8ed0
creation time: 2017-09-14T06:59:01Z
service account name: spark
volumes: spark-token-zr8wv
node name: 172.20.0.114
start time: N/A
container images: N/A
phase: Pending
status: []
2017-09-14 14:59:01 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
pod name: spark-pi-1505372339796-driver
namespace: spark-cluster
labels: spark-app-selector -&gt; spark<span class="hljs-_">-f</span>4d3a5d3ad964a05a51feb6191d50357, spark-role -&gt; driver
pod uid: 304cf440-991a-11e7-970c<span class="hljs-_">-f</span>4e9d49f8ed0
creation time: 2017-09-14T06:59:01Z
service account name: spark
volumes: spark-token-zr8wv
node name: 172.20.0.114
start time: 2017-09-14T06:59:01Z
container images: sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1
phase: Pending
status: [ContainerStatus(containerID=null, image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1, imageID=, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=<span class="hljs-literal">false</span>, restartCount=0, state=ContainerState(running=null, terminated=null, waiting=ContainerStateWaiting(message=null, reason=ContainerCreating, additionalProperties={}), additionalProperties={}), additionalProperties={})]
2017-09-14 14:59:03 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
pod name: spark-pi-1505372339796-driver
namespace: spark-cluster
labels: spark-app-selector -&gt; spark<span class="hljs-_">-f</span>4d3a5d3ad964a05a51feb6191d50357, spark-role -&gt; driver
pod uid: 304cf440-991a-11e7-970c<span class="hljs-_">-f</span>4e9d49f8ed0
creation time: 2017-09-14T06:59:01Z
service account name: spark
volumes: spark-token-zr8wv
node name: 172.20.0.114
start time: 2017-09-14T06:59:01Z
container images: sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1
phase: Running
status: [ContainerStatus(containerID=docker://5c5c821c482a1e35552adccb567020532b79244392374f25754f0050e6<span class="hljs-built_in">cd</span>4c62, image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1, imageID=docker-pullable://sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver@sha256:beb92a3e3f178e286d9e5baebdead88b5ba76d651f347ad2864bb6f8eda26f94, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=<span class="hljs-literal">true</span>, restartCount=0, state=ContainerState(running=ContainerStateRunning(startedAt=2017-09-14T06:59:02Z, additionalProperties={}), terminated=null, waiting=null, additionalProperties={}), additionalProperties={})]
2017-09-14 14:59:12 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
pod name: spark-pi-1505372339796-driver
namespace: spark-cluster
labels: spark-app-selector -&gt; spark<span class="hljs-_">-f</span>4d3a5d3ad964a05a51feb6191d50357, spark-role -&gt; driver
pod uid: 304cf440-991a-11e7-970c<span class="hljs-_">-f</span>4e9d49f8ed0
creation time: 2017-09-14T06:59:01Z
service account name: spark
volumes: spark-token-zr8wv
node name: 172.20.0.114
start time: 2017-09-14T06:59:01Z
container images: sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1
phase: Succeeded
status: [ContainerStatus(containerID=docker://5c5c821c482a1e35552adccb567020532b79244392374f25754f0050e6<span class="hljs-built_in">cd</span>4c62, image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1, imageID=docker-pullable://sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver@sha256:beb92a3e3f178e286d9e5baebdead88b5ba76d651f347ad2864bb6f8eda26f94, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=<span class="hljs-literal">false</span>, restartCount=0, state=ContainerState(running=null, terminated=ContainerStateTerminated(containerID=docker://5c5c821c482a1e35552adccb567020532b79244392374f25754f0050e6<span class="hljs-built_in">cd</span>4c62, <span class="hljs-built_in">exit</span>Code=0, finishedAt=2017-09-14T06:59:11Z, message=null, reason=Completed, signal=null, startedAt=null, additionalProperties={}), waiting=null, additionalProperties={}), additionalProperties={})]
2017-09-14 14:59:12 INFO LoggingPodStatusWatcherImpl:54 - Container final statuses:
Container name: spark-kubernetes-driver
Container image: sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1
Container state: Terminated
Exit code: 0
2017-09-14 14:59:12 INFO Client:54 - Application spark-pi finished.
</code></pre>
<p>&#x4ECE;&#x65E5;&#x5FD7;&#x4E2D;&#x53EF;&#x4EE5;&#x770B;&#x5230;&#x4EFB;&#x52A1;&#x8FD0;&#x884C;&#x7684;&#x72B6;&#x6001;&#x4FE1;&#x606F;&#x3002;</p>
<p>&#x4F7F;&#x7528;&#x4E0B;&#x9762;&#x7684;&#x547D;&#x4EE4;&#x53EF;&#x4EE5;&#x770B;&#x5230; kubernetes &#x542F;&#x52A8;&#x7684; Pod &#x4FE1;&#x606F;&#xFF1A;</p>
<pre><code class="lang-bash">kubectl --namespace spark-cluster get pods -w
</code></pre>
<p>&#x5C06;&#x4F1A;&#x770B;&#x5230; <code>spark-driver</code> &#x548C; <code>spark-exec</code> &#x7684; Pod &#x4FE1;&#x606F;&#x3002;</p>
2017-09-15 19:36:28 +08:00
<h2 id="&#x4F9D;&#x8D56;&#x7BA1;&#x7406;">&#x4F9D;&#x8D56;&#x7BA1;&#x7406;</h2>
<p>&#x4E0A;&#x6587;&#x4E2D;&#x6211;&#x4EEC;&#x5728;&#x8FD0;&#x884C;&#x6D4B;&#x8BD5;&#x7A0B;&#x5E8F;&#x65F6;&#xFF0C;&#x547D;&#x4EE4;&#x884C;&#x4E2D;&#x6307;&#x5B9A;&#x7684; jar &#x6587;&#x4EF6;&#x5DF2;&#x5305;&#x542B;&#x5728; docker &#x955C;&#x50CF;&#x4E2D;&#xFF0C;&#x662F;&#x4E0D;&#x662F;&#x8BF4;&#x6211;&#x4EEC;&#x6BCF;&#x6B21;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#x90FD;&#x9700;&#x8981;&#x91CD;&#x65B0;&#x521B;&#x5EFA;&#x4E00;&#x4E2A;&#x955C;&#x50CF;&#x5462;&#xFF1F;&#x975E;&#x4E5F;&#xFF01;&#x5982;&#x679C;&#x771F;&#x662F;&#x8FD9;&#x6837;&#x4E5F;&#x592A;&#x9EBB;&#x70E6;&#x4E86;&#x3002;</p>
<h4 id="&#x521B;&#x5EFA;-resource-staging-server">&#x521B;&#x5EFA; resource staging server</h4>
<p>&#x4E3A;&#x4E86;&#x65B9;&#x4FBF;&#x7528;&#x6237;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#xFF0C;&#x4E0D;&#x9700;&#x8981;&#x6BCF;&#x6B21;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#x7684;&#x65F6;&#x5019;&#x90FD;&#x521B;&#x5EFA;&#x4E00;&#x4E2A;&#x955C;&#x50CF;&#xFF0C;&#x6211;&#x4EEC;&#x4F7F;&#x7528;&#x4E86; <strong>resource staging server</strong> &#x3002;</p>
<pre><code>kubectl create -f conf/kubernetes-resource-staging-server.yaml
</code></pre><p>&#x6211;&#x4EEC;&#x540C;&#x6837;&#x5C06;&#x5176;&#x90E8;&#x7F72;&#x5728; <code>spark-cluster</code> namespace &#x4E0B;&#xFF0C;&#x8BE5; yaml &#x6587;&#x4EF6;&#x89C1; <a href="https://github.com/rootsongjc/kubernetes-handbook" target="_blank">kubernetes-handbook</a> &#x7684; <code>manifests/spark-with-kubernetes-native-scheduler</code> &#x76EE;&#x5F55;&#x3002;</p>
<h4 id="&#x4F18;&#x5316;">&#x4F18;&#x5316;</h4>
<p>&#x5176;&#x4E2D;&#x6709;&#x4E00;&#x70B9;&#x9700;&#x8981;&#x4F18;&#x5316;&#xFF0C;&#x5728;&#x4F7F;&#x7528;&#x4E0B;&#x9762;&#x7684;&#x547D;&#x4EE4;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#x65F6;&#xFF0C;&#x4F7F;&#x7528; <code>--conf spark.kubernetes.resourceStagingServer.uri</code> &#x53C2;&#x6570;&#x6307;&#x5B9A; <em>resource staging server</em> &#x5730;&#x5740;&#xFF0C;&#x7528;&#x6237;&#x4E0D;&#x5E94;&#x8BE5;&#x5173;&#x6CE8; <em>resource staging server</em> &#x7A76;&#x7ADF;&#x8FD0;&#x884C;&#x5728;&#x54EA;&#x53F0;&#x5BBF;&#x4E3B;&#x673A;&#x4E0A;&#xFF0C;&#x53EF;&#x4EE5;&#x4F7F;&#x7528;&#x4E0B;&#x9762;&#x4E24;&#x79CD;&#x65B9;&#x5F0F;&#x5B9E;&#x73B0;&#xFF1A;</p>
<ul>
<li>&#x4F7F;&#x7528; <code>nodeSelector</code> &#x5C06; <em>resource staging server</em> &#x56FA;&#x5B9A;&#x8C03;&#x5EA6;&#x5230;&#x67D0;&#x4E00;&#x53F0;&#x673A;&#x5668;&#x4E0A;&#xFF0C;&#x8BE5;&#x5730;&#x5740;&#x4F9D;&#x7136;&#x4F7F;&#x7528;&#x5BBF;&#x4E3B;&#x673A;&#x7684; IP &#x5730;&#x5740;</li>
<li>&#x6539;&#x53D8; <code>spark-resource-staging-service</code> service &#x7684; type &#x4E3A; <strong>ClusterIP</strong>&#xFF0C; &#x7136;&#x540E;&#x4F7F;&#x7528; <strong>Ingress</strong> &#x5C06;&#x5176;&#x66B4;&#x9732;&#x5230;&#x96C6;&#x7FA4;&#x5916;&#x90E8;&#xFF0C;&#x7136;&#x540E;&#x52A0;&#x5165;&#x7684;&#x5185;&#x7F51; DNS &#x91CC;&#xFF0C;&#x7528;&#x6237;&#x4F7F;&#x7528; DNS &#x540D;&#x79F0;&#x6307;&#x5B9A; <em>resource staging server</em> &#x7684;&#x5730;&#x5740;&#x3002;</li>
</ul>
<p>&#x7136;&#x540E;&#x53EF;&#x4EE5;&#x6267;&#x884C;&#x4E0B;&#x9762;&#x7684;&#x547D;&#x4EE4;&#x6765;&#x63D0;&#x4EA4;&#x672C;&#x5730;&#x7684; jar &#x5230; kubernetes &#x4E0A;&#x8FD0;&#x884C;&#x3002;</p>
<pre><code class="lang-bash">./spark-submit \
--deploy-mode cluster \
--class org.apache.spark.examples.SparkPi \
--master k8s://https://172.20.0.113:6443 \
--kubernetes-namespace spark-cluster \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.executor.instances=5 \
--conf spark.app.name=spark-pi \
--conf spark.kubernetes.driver.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.executor.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.initcontainer.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-init:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.resourceStagingServer.uri=http://172.20.0.114:31000 \
../examples/jars/spark-examples_2.11-2.2.0-k8s-0.4.0-SNAPSHOT.jar
</code></pre>
<p>&#x8BE5;&#x547D;&#x4EE4;&#x5C06;&#x63D0;&#x4EA4;&#x672C;&#x5730;&#x7684; <code>../examples/jars/spark-examples_2.11-2.2.0-k8s-0.4.0-SNAPSHOT.jar</code> &#x6587;&#x4EF6;&#x5230; <em>resource staging server</em>&#xFF0C;executor &#x5C06;&#x4ECE;&#x8BE5; server &#x4E0A;&#x83B7;&#x53D6; jar &#x5305;&#x5E76;&#x8FD0;&#x884C;&#xFF0C;&#x8FD9;&#x6837;&#x7528;&#x6237;&#x5C31;&#x4E0D;&#x9700;&#x8981;&#x6BCF;&#x6B21;&#x63D0;&#x4EA4;&#x4EFB;&#x52A1;&#x90FD;&#x7F16;&#x8BD1;&#x4E00;&#x4E2A;&#x955C;&#x50CF;&#x4E86;&#x3002;</p>
<p>&#x8BE6;&#x89C1;&#xFF1A;<a href="https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html#dependency-management" target="_blank">https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html#dependency-management</a></p>
2017-09-19 19:26:13 +08:00
<h4 id="&#x8BBE;&#x7F6E;-hdfs-&#x7528;&#x6237;">&#x8BBE;&#x7F6E; HDFS &#x7528;&#x6237;</h4>
<p>&#x5982;&#x679C; Hadoop &#x96C6;&#x7FA4;&#x6CA1;&#x6709;&#x8BBE;&#x7F6E; kerbros &#x5B89;&#x5168;&#x8BA4;&#x8BC1;&#x7684;&#x8BDD;&#xFF0C;&#x5728;&#x6307;&#x5B9A; <code>spark-submit</code> &#x7684;&#x65F6;&#x5019;&#x53EF;&#x4EE5;&#x901A;&#x8FC7;&#x6307;&#x5B9A;&#x5982;&#x4E0B;&#x56DB;&#x4E2A;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#xFF0C; &#x8BBE;&#x7F6E; Spark &#x4E0E; HDFS &#x901A;&#x4FE1;&#x4F7F;&#x7528;&#x7684;&#x7528;&#x6237;&#xFF1A;</p>
<pre><code class="lang-bash"> --conf spark.kubernetes.driverEnv.SPARK_USER=hadoop
--conf spark.kubernetes.driverEnv.HADOOP_USER_NAME=hadoop
--conf spark.executorEnv.HADOOP_USER_NAME=hadoop
--conf spark.executorEnv.SPARK_USER=hadoop
</code></pre>
2017-09-19 19:34:05 +08:00
<p>&#x4F7F;&#x7528; hadoop &#x7528;&#x6237;&#x63D0;&#x4EA4;&#x672C;&#x5730; jar &#x5305;&#x7684;&#x547D;&#x4EE4;&#x793A;&#x4F8B;&#xFF1A;</p>
<pre><code class="lang-bash">./spark-submit \
--deploy-mode cluster \
--class com.talkingdata.alluxio.hadooptest \
--master k8s://https://172.20.0.113:6443 \
--kubernetes-namespace spark-cluster \
--conf spark.kubernetes.driverEnv.SPARK_USER=hadoop \
--conf spark.kubernetes.driverEnv.HADOOP_USER_NAME=hadoop \
--conf spark.executorEnv.HADOOP_USER_NAME=hadoop \
--conf spark.executorEnv.SPARK_USER=hadoop \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.executor.instances=5 \
--conf spark.app.name=spark-pi \
--conf spark.kubernetes.driver.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.executor.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.initcontainer.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-init:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.resourceStagingServer.uri=http://172.20.0.114:31000 \
~/Downloads/tendcloud_2.10-1.0.jar
</code></pre>
2017-09-19 19:26:13 +08:00
<p>&#x8BE6;&#x89C1;&#xFF1A;<a href="https://github.com/apache-spark-on-k8s/spark/issues/408" target="_blank">https://github.com/apache-spark-on-k8s/spark/issues/408</a></p>
<h4 id="&#x9650;&#x5236;-driver-&#x548C;-executor-&#x7684;&#x8D44;&#x6E90;&#x4F7F;&#x7528;">&#x9650;&#x5236; Driver &#x548C; Executor &#x7684;&#x8D44;&#x6E90;&#x4F7F;&#x7528;</h4>
<p>&#x5728;&#x6267;&#x884C; <code>spark-submit</code> &#x65F6;&#x4F7F;&#x7528;&#x5982;&#x4E0B;&#x53C2;&#x6570;&#x8BBE;&#x7F6E;&#x5185;&#x5B58;&#x548C; CPU &#x8D44;&#x6E90;&#x9650;&#x5236;&#xFF1A;</p>
<pre><code class="lang-bash">--conf spark.driver.memory=3G
--conf spark.executor.memory=3G
--conf spark.driver.cores=2
--conf spark.executor.cores=10
</code></pre>
<p>&#x8FD9;&#x51E0;&#x4E2A;&#x53C2;&#x6570;&#x4E2D;&#x503C;&#x5982;&#x4F55;&#x4F20;&#x9012;&#x5230; Pod &#x7684;&#x8D44;&#x6E90;&#x8BBE;&#x7F6E;&#x4E2D;&#x7684;&#x5462;&#xFF1F;</p>
<p>&#x6BD4;&#x5982;&#x6211;&#x4EEC;&#x8BBE;&#x7F6E;&#x5728;&#x6267;&#x884C; <code>spark-submit</code> &#x7684;&#x65F6;&#x5019;&#x4F20;&#x9012;&#x4E86;&#x8FD9;&#x6837;&#x7684;&#x4E24;&#x4E2A;&#x53C2;&#x6570;&#xFF1A;<code>--conf spark.driver.cores=2</code> &#x548C; <code>--conf spark.driver.memory=100G</code> &#x90A3;&#x4E48;&#x67E5;&#x770B; driver pod &#x7684; yaml &#x8F93;&#x51FA;&#x7ED3;&#x679C;&#x5C06;&#x4F1A;&#x770B;&#x5230;&#x8FD9;&#x6837;&#x7684;&#x8D44;&#x6E90;&#x8BBE;&#x7F6E;&#xFF1A;</p>
<pre><code class="lang-yaml"><span class="hljs-attr"> resources:</span>
<span class="hljs-attr"> limits:</span>
<span class="hljs-attr"> memory:</span> <span class="hljs-number">110</span>Gi
<span class="hljs-attr"> requests:</span>
<span class="hljs-attr"> cpu:</span> <span class="hljs-string">&quot;2&quot;</span>
<span class="hljs-attr"> memory:</span> <span class="hljs-number">100</span>Gi
</code></pre>
<p>&#x4EE5;&#x4E0A;&#x53C2;&#x6570;&#x662F;&#x5BF9; <code>request</code> &#x503C;&#x7684;&#x8BBE;&#x7F6E;&#xFF0C;&#x90A3;&#x4E48; <code>limit</code> &#x7684;&#x8D44;&#x6E90;&#x8BBE;&#x7F6E;&#x7684;&#x503C;&#x53C8;&#x662F;&#x4ECE;&#x4F55;&#x800C;&#x6765;&#xFF1F;</p>
<p>&#x53EF;&#x4EE5;&#x4F7F;&#x7528; <code>spark.kubernetes.driver.limit.cores</code> &#x548C; <code>spark.kubernetes.executor.limit.cores</code> &#x6765;&#x8BBE;&#x7F6E; CPU&#x7684; hard limit&#x3002;</p>
<p>memory limit &#x7684;&#x503C;&#x662F;&#x6839;&#x636E; memory request &#x7684;&#x503C;&#x52A0;&#x4E0A; <code>spark.kubernetes.executor.memoryOverhead</code> &#x7684;&#x503C;&#x8BA1;&#x7B97;&#x800C;&#x6765;&#x7684;&#xFF0C;&#x8BE5;&#x914D;&#x7F6E;&#x9879;&#x7528;&#x4E8E;&#x8BBE;&#x7F6E;&#x5206;&#x914D;&#x7ED9;&#x6BCF;&#x4E2A; executor &#x7684;&#x8D85;&#x8FC7; heap &#x5185;&#x5B58;&#x7684;&#x503C;&#xFF08;&#x53EF;&#x4EE5;&#x4F7F;&#x7528;k&#x3001;m&#x3001;g&#x5355;&#x4F4D;&#xFF09;&#x3002;&#x8BE5;&#x503C;&#x7528;&#x4E8E;&#x865A;&#x62DF;&#x673A;&#x7684;&#x5F00;&#x9500;&#x3001;&#x5176;&#x4ED6;&#x672C;&#x5730;&#x670D;&#x52A1;&#x5F00;&#x9500;&#x3002;&#x6839;&#x636E; executor &#x7684;&#x5927;&#x5C0F;&#x8BBE;&#x7F6E;&#xFF08;&#x901A;&#x5E38;&#x662F; 6%&#x5230;10%&#xFF09;&#x3002;</p>
<p>&#x6211;&#x4EEC;&#x53EF;&#x4EE5;&#x8FD9;&#x6837;&#x6765;&#x63D0;&#x4EA4;&#x4E00;&#x4E2A;&#x4EFB;&#x52A1;&#xFF0C;&#x540C;&#x65F6;&#x8BBE;&#x7F6E; driver &#x548C; executor &#x7684; CPU&#x3001;&#x5185;&#x5B58;&#x7684;&#x8D44;&#x6E90; request &#x548C; limit &#x503C;&#xFF08;driver &#x7684;&#x5185;&#x5B58; limit &#x503C;&#x4E3A; request &#x503C;&#x7684; 110%&#xFF09;&#x3002;</p>
<pre><code class="lang-bash">./spark-submit \
--deploy-mode cluster \
--class org.apache.spark.examples.SparkPi \
--master k8s://https://172.20.0.113:6443 \
--kubernetes-namespace spark-cluster \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--conf spark.driver.memory=100G \
--conf spark.executor.memory=10G \
--conf spark.driver.cores=30 \
--conf spark.executor.cores=2 \
--conf spark.driver.maxResultSize=10240m \
--conf spark.kubernetes.driver.limit.cores=32 \
--conf spark.kubernetes.executor.limit.cores=3 \
--conf spark.kubernetes.executor.memoryOverhead=2g \
--conf spark.executor.instances=5 \
--conf spark.app.name=spark-pi \
--conf spark.kubernetes.driver.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-driver:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.executor.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-executor:v2.1.0-kubernetes-0.3.1-1 \
--conf spark.kubernetes.initcontainer.docker.image=sz-pg-oam-docker-hub-001.tendcloud.com/library/spark-init:v2.1.0-kubernetes-0.3.1-1 \
<span class="hljs-built_in">local</span>:///opt/spark/examples/jars/spark-examples_2.11-2.2.0-k8s-0.4.0-SNAPSHOT.jar 10000000
</code></pre>
<p>&#x8FD9;&#x5C06;&#x542F;&#x52A8;&#x4E00;&#x4E2A;&#x5305;&#x542B;&#x4E00;&#x5343;&#x4E07;&#x4E2A; task &#x7684;&#x8BA1;&#x7B97; pi &#x7684; spark &#x4EFB;&#x52A1;&#xFF0C;&#x4EFB;&#x52A1;&#x8FD0;&#x884C;&#x8FC7;&#x7A0B;&#x4E2D;&#xFF0C;drvier &#x7684; CPU &#x5B9E;&#x9645;&#x6D88;&#x8017;&#x5927;&#x7EA6;&#x4E3A; 3 &#x6838;&#xFF0C;&#x5185;&#x5B58; 40G&#xFF0C;&#x6BCF;&#x4E2A; executor &#x7684; CPU &#x5B9E;&#x9645;&#x6D88;&#x8017;&#x5927;&#x7EA6;&#x4E0D;&#x5230; 1 &#x6838;&#xFF0C;&#x5185;&#x5B58;&#x4E0D;&#x5230; 4G&#xFF0C;&#x6211;&#x4EEC;&#x53EF;&#x4EE5;&#x6839;&#x636E;&#x5B9E;&#x9645;&#x8D44;&#x6E90;&#x6D88;&#x8017;&#x4E0D;&#x65AD;&#x4F18;&#x5316;&#x8D44;&#x6E90;&#x7684; request &#x503C;&#x3002;</p>
<p><code>SPARK_DRIVER_MEMORY</code> &#x548C; <code>SPARK_EXECUTOR_MEMORY</code> &#x548C;&#x5206;&#x522B;&#x4F5C;&#x4E3A; Driver &#x5BB9;&#x5668;&#x548C; Executor &#x5BB9;&#x5668;&#x542F;&#x52A8;&#x7684;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#xFF0C;&#x6BD4;&#x5982;&#x4E0B;&#x9762;&#x8FD9;&#x4E2A; Driver &#x542F;&#x52A8;&#x7684; CMD &#x4E2D;&#xFF1A;</p>
<pre><code class="lang-bash">CMD SPARK_CLASSPATH=<span class="hljs-string">&quot;<span class="hljs-variable">${SPARK_HOME}</span>/jars/*&quot;</span> &amp;&amp; \
env | grep SPARK_JAVA_OPT_ | sed <span class="hljs-string">&apos;s/[^=]*=\(.*\)/\1/g&apos;</span> &gt; /tmp/java_opts.txt &amp;&amp; \
<span class="hljs-built_in">readarray</span> -t SPARK_DRIVER_JAVA_OPTS &lt; /tmp/java_opts.txt &amp;&amp; \
<span class="hljs-keyword">if</span> ! [ -z <span class="hljs-variable">${SPARK_MOUNTED_CLASSPATH+x}</span> ]; <span class="hljs-keyword">then</span> SPARK_CLASSPATH=<span class="hljs-string">&quot;<span class="hljs-variable">$SPARK_MOUNTED_CLASSPATH</span>:<span class="hljs-variable">$SPARK_CLASSPATH</span>&quot;</span>; <span class="hljs-keyword">fi</span> &amp;&amp; \
<span class="hljs-keyword">if</span> ! [ -z <span class="hljs-variable">${SPARK_SUBMIT_EXTRA_CLASSPATH+x}</span> ]; <span class="hljs-keyword">then</span> SPARK_CLASSPATH=<span class="hljs-string">&quot;<span class="hljs-variable">$SPARK_SUBMIT_EXTRA_CLASSPATH</span>:<span class="hljs-variable">$SPARK_CLASSPATH</span>&quot;</span>; <span class="hljs-keyword">fi</span> &amp;&amp; \
<span class="hljs-keyword">if</span> ! [ -z <span class="hljs-variable">${SPARK_EXTRA_CLASSPATH+x}</span> ]; <span class="hljs-keyword">then</span> SPARK_CLASSPATH=<span class="hljs-string">&quot;<span class="hljs-variable">$SPARK_EXTRA_CLASSPATH</span>:<span class="hljs-variable">$SPARK_CLASSPATH</span>&quot;</span>; <span class="hljs-keyword">fi</span> &amp;&amp; \
<span class="hljs-keyword">if</span> ! [ -z <span class="hljs-variable">${SPARK_MOUNTED_FILES_DIR+x}</span> ]; <span class="hljs-keyword">then</span> cp -R <span class="hljs-string">&quot;<span class="hljs-variable">$SPARK_MOUNTED_FILES_DIR</span>/.&quot;</span> .; <span class="hljs-keyword">fi</span> &amp;&amp; \
<span class="hljs-keyword">if</span> ! [ -z <span class="hljs-variable">${SPARK_MOUNTED_FILES_FROM_SECRET_DIR}</span> ]; <span class="hljs-keyword">then</span> cp -R <span class="hljs-string">&quot;<span class="hljs-variable">$SPARK_MOUNTED_FILES_FROM_SECRET_DIR</span>/.&quot;</span> .; <span class="hljs-keyword">fi</span> &amp;&amp; \
<span class="hljs-variable">${JAVA_HOME}</span>/bin/java <span class="hljs-string">&quot;<span class="hljs-variable">${SPARK_DRIVER_JAVA_OPTS[@]}</span>&quot;</span> -cp <span class="hljs-variable">$SPARK_CLASSPATH</span> -Xms<span class="hljs-variable">$SPARK_DRIVER_MEMORY</span> -Xmx<span class="hljs-variable">$SPARK_DRIVER_MEMORY</span> <span class="hljs-variable">$SPARK_DRIVER_CLASS</span> <span class="hljs-variable">$SPARK_DRIVER_ARGS</span>
</code></pre>
<p>&#x6211;&#x4EEC;&#x53EF;&#x4EE5;&#x770B;&#x5230;&#x5BF9; <code>SPARK_DRIVER_MEMORY</code> &#x73AF;&#x5883;&#x53D8;&#x91CF;&#x7684;&#x5F15;&#x7528;&#x3002;Executor &#x7684;&#x8BBE;&#x7F6E;&#x4E0E; driver &#x7C7B;&#x4F3C;&#x3002;</p>
<p>&#x800C;&#x6211;&#x4EEC;&#x53EF;&#x4EE5;&#x4F7F;&#x7528;&#x8FD9;&#x6837;&#x7684;&#x53C2;&#x6570;&#x6765;&#x4F20;&#x9012;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#x7684;&#x503C; <code>spark.executorEnv.[EnvironmentVariableName]</code>&#xFF0C;&#x53EA;&#x8981;&#x5C06; <code>EnvironmentVariableName</code> &#x66FF;&#x6362;&#x4E3A;&#x73AF;&#x5883;&#x53D8;&#x91CF;&#x540D;&#x79F0;&#x5373;&#x53EF;&#x3002;</p>
2017-09-14 15:57:50 +08:00
<h2 id="&#x53C2;&#x8003;">&#x53C2;&#x8003;</h2>
<p><a href="http://lxw1234.com/archives/2015/12/593.htm" target="_blank">Spark&#x52A8;&#x6001;&#x8D44;&#x6E90;&#x5206;&#x914D;-Dynamic Resource Allocation</a></p>
<p><a href="https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html" target="_blank">Running Spark on Kubernetes</a></p>
<p><a href="https://issues.apache.org/jira/browse/SPARK-18278" target="_blank">Apache Spark Jira Issue - 18278 - SPIP: Support native submission of spark jobs to a kubernetes cluster</a></p>
<p><a href="https://github.com/kubernetes/kubernetes/issues/34377" target="_blank">Kubernetes Github Issue - 34377 Support Spark natively in Kubernetes</a></p>
<p><a href="https://github.com/kubernetes/kubernetes/tree/master/examples/spark" target="_blank">Kubernetes example spark</a></p>
2017-09-19 19:26:13 +08:00
<p><a href="https://github.com/rootsongjc/spark-on-kubernetes" target="_blank">https://github.com/rootsongjc/spark-on-kubernetes</a></p>
<p><a href="https://github.com/apache-spark-on-k8s/spark/blob/branch-2.2-kubernetes/resource-managers/kubernetes/architecture-docs/scheduler-backend.md" target="_blank">Scheduler backend</a></p>
2017-10-09 15:28:05 +08:00
<footer class="page-footer"><span class="copyright">Copyright &#xA9; jimmysong.io 2017 all right reserved&#xFF0C;powered by Gitbook</span><span class="footer-modification">Updated:
2017-09-27 21:03:00
</span></footer>
2017-09-14 15:57:50 +08:00
</section>
</div>
<div class="search-results">
<div class="has-results">
<h1 class="search-results-title"><span class='search-results-count'></span> results matching "<span class='search-query'></span>"</h1>
<ul class="search-results-list"></ul>
</div>
<div class="no-results">
<h1 class="search-results-title">No results matching "<span class='search-query'></span>"</h1>
</div>
</div>
</div>
</div>
</div>
</div>
2017-11-15 21:18:59 +08:00
<a href="spark-standalone-on-kubernetes.html" class="navigation navigation-prev " aria-label="Previous page: Spark standalone on Kubernetes">
2017-09-14 15:57:50 +08:00
<i class="fa fa-angle-left"></i>
</a>
2017-11-15 21:18:59 +08:00
<a href="serverless.html" class="navigation navigation-next " aria-label="Next page: Serverless架构">
2017-09-14 15:57:50 +08:00
<i class="fa fa-angle-right"></i>
</a>
</div>
<script>
var gitbook = gitbook || [];
gitbook.push(function() {
2017-11-22 18:43:03 +08:00
gitbook.page.hasChanged({"page":{"title":"运行支持kubernetes原生调度的Spark程序","level":"5.4.2","depth":2,"next":{"title":"Serverless架构","level":"5.5","depth":1,"path":"usecases/serverless.md","ref":"usecases/serverless.md","articles":[]},"previous":{"title":"Spark standalone on Kubernetes","level":"5.4.1","depth":2,"path":"usecases/spark-standalone-on-kubernetes.md","ref":"usecases/spark-standalone-on-kubernetes.md","articles":[]},"dir":"ltr"},"config":{"plugins":["github","codesnippet","splitter","page-toc-button","image-captions","editlink","back-to-top-button","-lunr","-search","search-plus","github-buttons@2.1.0","favicon@^0.0.2","tbfed-pagefooter@^0.0.1","3-ba","theme-default"],"styles":{"website":"styles/website.css","pdf":"styles/pdf.css","epub":"styles/epub.css","mobi":"styles/mobi.css","ebook":"styles/ebook.css","print":"styles/print.css"},"pluginsConfig":{"tbfed-pagefooter":{"copyright":"Copyright © jimmysong.io 2017","modify_label":"Updated:","modify_format":"YYYY-MM-DD HH:mm:ss"},"github":{"url":"https://github.com/rootsongjc/kubernetes-handbook"},"editlink":{"label":"编辑本页","multilingual":false,"base":"https://github.com/rootsongjc/kubernetes-handbook/blob/master/"},"splitter":{},"codesnippet":{},"fontsettings":{"theme":"white","family":"sans","size":2},"highlight":{},"favicon":{"shortcut":"favicon.ico","bookmark":"favicon.ico"},"page-toc-button":{},"back-to-top-button":{},"github-buttons":{"repo":"rootsongjc/kubernetes-handbook","types":["star"],"size":"small"},"3-ba":{"configuration":"auto","token":"11f7d254cfa4e0ca44b175c66d379ecc"},"sharing":{"facebook":true,"twitter":true,"google":false,"weibo":false,"instapaper":false,"vk":false,"all":["facebook","google","twitter","weibo","instapaper"]},"theme-default":{"styles":{"website":"styles/website.css","pdf":"styles/pdf.css","epub":"styles/epub.css","mobi":"styles/mobi.css","ebook":"styles/ebook.css","print":"styles/print.css"},"showLevel":true},"search-plus":{},"image-captions":{"caption":"图片 - _CAPTION_","variable_name":"_pictures"}},"theme":"default","author":"Jimmy Song","pdf":{"pageNumbers":true,"fontSize":12,"fontFamily":"Arial","paperSize":"a4","chapterMark":"pagebreak","pageBreaksBefore":"/","margin":{"right":62,"left":62,"top":56,"bottom":56}},"structure":{"langs":"LANGS.md","readme":"README.md","glossary":"GLOSSARY.md","summary":"SUMMARY.md"},"variables":{"_pictures":[{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig1.2.1","level":"1.2","list_caption":"Figure: 云计算演进历程","alt":"云计算演进历程","nro":1,"url":"../images/cloud-computing-evolution-road.jpg","index":1,"caption_template":"图片 - _CAPTION_","label":"云计算演进历程","attributes":{},"skip":false,"key":"1.2.1"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig1.2.2","level":"1.2","list_caption":"Figure: 来自Twitter @MarcWilczek","alt":"来自Twitter @MarcWilczek","nro":2,"url":"../images/cloud-native-comes-of-age.jpg","index":2,"caption_template":"图片 - _CAPTION_","label":"来自Twitter @MarcWilczek","attributes":{},"skip":false,"key":"1.2.2"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig1.2.3","level":"1.2","list_caption":"Figure: Cloud native思维导图","alt":"Cloud native思维导图","nro":3,"url":"../images/cloud-native-architecutre-mindnode.jpg","index":3,"caption_template":"图片 - _CAPTION_","label":"Cloud native思维导图","attributes":{},"skip":false,"key":"1.2.3"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig1.2.4","level":"1.2","list_caption":"Figure: 十二因素应用","alt":"十二因素应用","nro":4,"url":"../images/12-factor-app.png","index":4,"caption_template":"图片 - _CAPTION_","label":"十二因素应用","attributes":{},"skip":false,"key":"1.2.4"},{"backlink":"cloud-native/kubernetes-and-cloud-native-app-overview.html#fig1.2.5","level":"1.2","list_caption":"Figure: 容器生态","alt":"容器生态","nro":5,"url":"../images/container-ecosystem.png","index":5,"caption_template":"<22>
2017-09-14 15:57:50 +08:00
});
</script>
</div>
<script src="../gitbook/gitbook.js"></script>
<script src="../gitbook/theme.js"></script>
<script src="../gitbook/gitbook-plugin-github/plugin.js"></script>
<script src="../gitbook/gitbook-plugin-splitter/splitter.js"></script>
<script src="../gitbook/gitbook-plugin-page-toc-button/plugin.js"></script>
<script src="../gitbook/gitbook-plugin-editlink/plugin.js"></script>
2017-09-19 21:38:03 +08:00
<script src="../gitbook/gitbook-plugin-back-to-top-button/plugin.js"></script>
2017-09-14 15:57:50 +08:00
<script src="../gitbook/gitbook-plugin-search-plus/jquery.mark.min.js"></script>
<script src="../gitbook/gitbook-plugin-search-plus/search.js"></script>
2017-10-09 15:28:05 +08:00
<script src="../gitbook/gitbook-plugin-github-buttons/plugin.js"></script>
2017-10-11 18:13:13 +08:00
<script src="../gitbook/gitbook-plugin-3-ba/plugin.js"></script>
2017-09-14 15:57:50 +08:00
<script src="../gitbook/gitbook-plugin-sharing/buttons.js"></script>
<script src="../gitbook/gitbook-plugin-fontsettings/fontsettings.js"></script>
</body>
</html>