关于该命令参数的介绍请参考：https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html
-注意： 该 jar 包实际上是
-运行失败，报错信息：
-2017-09-05 14:45:52 INFO Client:54 - Waiting for application spark-pi to finish...
-2017-09-05 14:45:52 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
- pod name: spark-pi-1504593950039-driver
- namespace: spark-cluster
- labels: spark-app-selector -> spark-81cd1d33adbd4f728f7c609356b54c43, spark-role -> driver
- pod uid: dbf66ecf-9205-11e7-970c-f4e9d49f8ed0
- creation time: 2017-09-05T06:45:52Z
- service account name: default
- volumes: default-token-klxp8
- node name: N/A
- start time: N/A
- container images: N/A
- phase: Pending
- status: []
-2017-09-05 14:45:52 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
- pod name: spark-pi-1504593950039-driver
- namespace: spark-cluster
- labels: spark-app-selector -> spark-81cd1d33adbd4f728f7c609356b54c43, spark-role -> driver
- pod uid: dbf66ecf-9205-11e7-970c-f4e9d49f8ed0
- creation time: 2017-09-05T06:45:52Z
- service account name: default
- volumes: default-token-klxp8
- node name: 172.20.0.115
- start time: N/A
- container images: N/A
- phase: Pending
- status: []
-2017-09-05 14:45:52 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
- pod name: spark-pi-1504593950039-driver
- namespace: spark-cluster
- labels: spark-app-selector -> spark-81cd1d33adbd4f728f7c609356b54c43, spark-role -> driver
- pod uid: dbf66ecf-9205-11e7-970c-f4e9d49f8ed0
- creation time: 2017-09-05T06:45:52Z
- service account name: default
- volumes: default-token-klxp8
- node name: 172.20.0.115
- start time: 2017-09-05T06:45:52Z
- container images: sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver:v2.1.0-kubernetes-0.3.1
- phase: Pending
- status: [ContainerStatus(containerID=null, image=sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver:v2.1.0-kubernetes-0.3.1, imageID=, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=false, restartCount=0, state=ContainerState(running=null, terminated=null, waiting=ContainerStateWaiting(message=null, reason=ContainerCreating, additionalProperties={}), additionalProperties={}), additionalProperties={})]
-2017-09-05 14:45:53 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
- pod name: spark-pi-1504593950039-driver
- namespace: spark-cluster
- labels: spark-app-selector -> spark-81cd1d33adbd4f728f7c609356b54c43, spark-role -> driver
- pod uid: dbf66ecf-9205-11e7-970c-f4e9d49f8ed0
- creation time: 2017-09-05T06:45:52Z
- service account name: default
- volumes: default-token-klxp8
- node name: 172.20.0.115
- start time: 2017-09-05T06:45:52Z
- container images: sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver:v2.1.0-kubernetes-0.3.1
- phase: Running
- status: [ContainerStatus(containerID=docker://53de39eb83435a344ef780aae83139229d4d6d78fa4e1655f9f81da95d89f439, image=sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver:v2.1.0-kubernetes-0.3.1, imageID=docker-pullable://sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver@sha256:19c3b76a34fee02104de0d859a60d79608ebd0b7ebae33ec3b86a71af777c833, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=true, restartCount=0, state=ContainerState(running=ContainerStateRunning(startedAt=2017-09-05T06:45:53Z, additionalProperties={}), terminated=null, waiting=null, additionalProperties={}), additionalProperties={})]
-2017-09-05 14:45:56 INFO LoggingPodStatusWatcherImpl:54 - State changed, new state:
- pod name: spark-pi-1504593950039-driver
- namespace: spark-cluster
- labels: spark-app-selector -> spark-81cd1d33adbd4f728f7c609356b54c43, spark-role -> driver
- pod uid: dbf66ecf-9205-11e7-970c-f4e9d49f8ed0
- creation time: 2017-09-05T06:45:52Z
- service account name: default
- volumes: default-token-klxp8
- node name: 172.20.0.115
- start time: 2017-09-05T06:45:52Z
- container images: sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver:v2.1.0-kubernetes-0.3.1
- phase: Failed
- status: [ContainerStatus(containerID=docker://53de39eb83435a344ef780aae83139229d4d6d78fa4e1655f9f81da95d89f439, image=sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver:v2.1.0-kubernetes-0.3.1, imageID=docker-pullable://sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver@sha256:19c3b76a34fee02104de0d859a60d79608ebd0b7ebae33ec3b86a71af777c833, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=false, restartCount=0, state=ContainerState(running=null, terminated=ContainerStateTerminated(containerID=docker://53de39eb83435a344ef780aae83139229d4d6d78fa4e1655f9f81da95d89f439, exitCode=1, finishedAt=2017-09-05T06:45:55Z, message=null, reason=Error, signal=null, startedAt=null, additionalProperties={}), waiting=null, additionalProperties={}), additionalProperties={})]
-2017-09-05 14:45:56 INFO LoggingPodStatusWatcherImpl:54 - Container final statuses:
-
-
- Container name: spark-kubernetes-driver
- Container image: sz-pg-oam-docker-hub-001.tendcloud.com/library/kubespark-spark-driver:v2.1.0-kubernetes-0.3.1
- Container state: Terminated
- Exit code: 1
-2017-09-05 14:45:56 INFO Client:54 - Application spark-pi finished.
-注意： 该 jar 包实际上是 spark.kubernetes.executor.docker.image 镜像中的。
这时候提交任务运行还是失败，报错信息中可以看到两个问题：
+system:serviceaccount:spark-cluster:defaul 没有权限获取 spark-cluster 中的 pod 信息。提了个 issue Failed to run the sample spark-pi test using spark-submit on the doc #478
+需要为 spark 集群创建一个 serviceaccount 和 clusterrolebinding：
kubectl create serviceaccount spark --namespace spark-cluster
+kubectl create rolebinding spark-edit --clusterrole=edit --serviceaccount=spark-cluster:spark --namespace=spark-cluster
+该 Bug 将在新版本中修复。