using vistio to visualize your istio mesh
rootsongjc
parent
360525600c
commit
ee55791594
|
|
@ -165,6 +165,7 @@
|
|||
- [使用Heapster获取集群和对象的metric数据](practice/using-heapster-to-get-object-metrics.md)
|
||||
- [Prometheus](practice/prometheus.md)
|
||||
- [使用Prometheus监控kubernetes集群](practice/using-prometheus-to-monitor-kuberentes-cluster.md)
|
||||
- [使用Vistio监控Istio服务网格中的流量](practice/vistio-visualize-your-istio-mesh.md)
|
||||
- [服务编排管理](practice/services-management-tool.md)
|
||||
- [使用Helm管理kubernetes应用](practice/helm.md)
|
||||
- [构建私有Chart仓库](practice/create-private-charts-repo.md)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,208 @@
|
|||
---
|
||||
# Source: vistio/templates/configmap.yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: vistio-api-config
|
||||
labels:
|
||||
heritage: Tiller
|
||||
chart: "vistio-0.1.0"
|
||||
app: vistio
|
||||
release: vistio
|
||||
data:
|
||||
vistio.yaml: |-
|
||||
clusterLevel:
|
||||
- cluster: istio-mesh
|
||||
maxVolume: 100
|
||||
serviceConnections:
|
||||
- notices:
|
||||
- name: HighErrorRate
|
||||
severityThreshold:
|
||||
error: 0.05
|
||||
warning: 0.01
|
||||
statusType: danger
|
||||
title: '[{{ .value }}] High Error Rate'
|
||||
prometheusURL: http://prometheus.istio-system:9090
|
||||
query: sum(rate(istio_request_count[1m])) by (source_service,destination_service,response_code)
|
||||
source:
|
||||
label: source_service
|
||||
status:
|
||||
dangerRegex: ^5..$
|
||||
label: response_code
|
||||
warningRegex: ^4..$
|
||||
target:
|
||||
label: destination_service
|
||||
globalLevel:
|
||||
clusterConnections:
|
||||
- prometheusURL: http://prometheus.istio-system:9090
|
||||
query: sum(rate(istio_request_count[1m])) by (response_code)
|
||||
source:
|
||||
replacement: Total Mesh Requests
|
||||
status:
|
||||
dangerRegex: ^5..$
|
||||
label: response_code
|
||||
warningRegex: ^4..$
|
||||
target:
|
||||
replacement: istio-mesh
|
||||
maxVolume: 100
|
||||
graphName: Vistio
|
||||
|
||||
---
|
||||
# Source: vistio/templates/service.yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: vistio-api
|
||||
labels:
|
||||
heritage: Tiller
|
||||
chart: "vistio-0.1.0"
|
||||
app: vistio-api
|
||||
release: vistio
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app: vistio-api
|
||||
release: vistio
|
||||
ports:
|
||||
- name: http
|
||||
port: 9091
|
||||
targetPort: 9091
|
||||
protocol: TCP
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: vistio-web
|
||||
labels:
|
||||
heritage: Tiller
|
||||
chart: "vistio-0.1.0"
|
||||
app: vistio-web
|
||||
release: vistio
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
app: vistio-web
|
||||
release: vistio
|
||||
ports:
|
||||
- name: http
|
||||
port: 8080
|
||||
targetPort: 8080
|
||||
protocol: TCP
|
||||
|
||||
---
|
||||
# Source: vistio/templates/deployment.yaml
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: vistio-web
|
||||
labels:
|
||||
heritage: Tiller
|
||||
chart: "vistio-0.1.0"
|
||||
app: vistio-web
|
||||
release: vistio
|
||||
spec:
|
||||
replicas: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: vistio-web
|
||||
release: vistio
|
||||
spec:
|
||||
containers:
|
||||
- name: vistio
|
||||
image: "nmnellis/vistio-web:v0.1.0"
|
||||
imagePullPolicy: Always
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
env:
|
||||
- name: UPDATE_URL
|
||||
value: http://localhost:9091/graph
|
||||
- name: INTERVAL
|
||||
value: "1000"
|
||||
- name: MAX_REPLAY_OFFSET
|
||||
value: "43200"
|
||||
resources:
|
||||
{}
|
||||
|
||||
|
||||
---
|
||||
# Source: vistio/templates/statefulset.yaml
|
||||
apiVersion: apps/v1beta1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: vistio-api
|
||||
labels:
|
||||
heritage: Tiller
|
||||
chart: "vistio-0.1.0"
|
||||
app: vistio-api
|
||||
release: vistio
|
||||
spec:
|
||||
replicas: 1
|
||||
serviceName: vistio
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: vistio-api
|
||||
release: vistio
|
||||
spec:
|
||||
containers:
|
||||
- name: vistio-api
|
||||
image: nmnellis/vistio-api:v0.1.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
args:
|
||||
- --config.file=/etc/vistio/vistio.yaml
|
||||
- --log.level=info
|
||||
- --storage.path=/var/vistio/data
|
||||
- --storage.retention=24h
|
||||
- --cache.size=100
|
||||
- --retrieval.scrape-interval=10s
|
||||
- --retrieval.scrape-timeout=8s
|
||||
- --api.port=9091
|
||||
ports:
|
||||
- containerPort: 9091
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 9091
|
||||
initialDelaySeconds: 30
|
||||
timeoutSeconds: 30
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: 9091
|
||||
initialDelaySeconds: 30
|
||||
timeoutSeconds: 30
|
||||
resources:
|
||||
{}
|
||||
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/vistio
|
||||
- name: vistio-db
|
||||
mountPath: /var/vistio/data
|
||||
subPath: vistio-db
|
||||
- name: reloader
|
||||
image: nghialv2607/k8s-config-reloader:v0.1.0
|
||||
imagePullPolicy: IfNotPresent
|
||||
args:
|
||||
- --config.promviz-reload-url=http://localhost:9091/reload
|
||||
- --config.promviz-config-dir=/etc/vistio
|
||||
- --config.log-level=info
|
||||
resources:
|
||||
limits:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
requests:
|
||||
cpu: 20m
|
||||
memory: 32Mi
|
||||
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/vistio
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: vistio-api-config
|
||||
- name: vistio-db
|
||||
emptyDir: {}
|
||||
|
||||
|
|
@ -0,0 +1,137 @@
|
|||
# 使用Vistio监控Istio服务网格中的流量
|
||||
|
||||
Vistio GitHub地址：<https://github.com/nmnellis/vistio>
|
||||
|
||||
[Vizceral](https://github.com/Netflix/vizceral)是Netflix发布的一个开源项目，用于近乎实时地监控应用程序和集群之间的网络流量。Vistio是使用Vizceral对Istio和网格监控的改进。它利用Istio Mixer生成的指标，然后将其输入Prometheus。Vistio查询Prometheus并将数据存储在本地以允许重播流量。
|
||||
|
||||
Vizceral有两个可视化级别，全局可视化和集群级别可视化。在全局范围内（如上所示），您可以通过Istio Ingress Gateway等入口点将从Internet到Istio服务网格网络的网络流量可视化，或者您可以在Istio服务网格网络中显示总网络流量。
|
||||
|
||||
在集群级别（如下所示），您可以可视化内部网格的流量。通过设置警告和错误级别警报，当应用程序出现问题时可以被快速检测出来。
|
||||
|
||||
|
||||
|
||||
### 在Istio服务网格中安装Vistio
|
||||
|
||||
**依赖**
|
||||
|
||||
- Prometheus
|
||||
- Istio 0.7或更高版本
|
||||
|
||||
**假设**
|
||||
|
||||
以下Demo使得这些假设更容易部署。如果您的环境设置不同，则可能需要将代码下载到本地并编辑一些文件。
|
||||
|
||||
- Prometheus部署在`istio-system` namespace下，可以通过[http://prometheus.istio-system:9090](http://prometheus.istio-system:9090/)地址访问
|
||||
- Istio mixer启用了`istio_request_count` metric
|
||||
- Kubernetes集群包含有`standard` StorageClass
|
||||
- 为了便于部署已安装了Helm（可选）
|
||||
|
||||
**前言**
|
||||
|
||||
如果您还尚未部署服务网格，可以按照此[Istio Bookinfo Demo](https://istio.io/docs/guides/bookinfo/)中的说明部署Istio及其示例应用程序。您需要能够在应用程序之间生成流量。要测试指标是否从Mixer正确发送到Prometheus，您可以运行以下Prometheus查询`istio_request_count`，应该会看到多个条目。
|
||||
|
||||
|
||||
|
||||
### 部署Vistio
|
||||
|
||||
我们直接使用kubectl命令来部署。
|
||||
|
||||
**使用kubectl部署**
|
||||
|
||||
```bash
|
||||
kubectl apply -f https://raw.githubusercontent.com/rootsongjc/kubernetes-handbook/master/manifests/vistio/vistio-mesh-only.yaml -n default
|
||||
```
|
||||
|
||||
### 验证和暴露Vistio Web/API
|
||||
|
||||
验证应用程序已经启动并在运行。使用`kubectl port-forward`命令暴露应用程序。
|
||||
|
||||
**验证vistio-api**
|
||||
|
||||
```bash
|
||||
kubectl describe statefulset vistio-api -n default
|
||||
```
|
||||
|
||||
**日志检查（可选的）**
|
||||
|
||||
您应该能够从vistio-api的日志中查看是否存在与Prometheus的连接/查询相关的错误。
|
||||
|
||||
```bash
|
||||
kubectl logs -n default -c vistio-api $(kubectl -n default get pod -l app=vistio-api -o jsonpath='{.items[0].metadata.name}')
|
||||
```
|
||||
|
||||
**验证vistio-web**
|
||||
|
||||
```bash
|
||||
kubectl describe deployment vistio-web -n default
|
||||
```
|
||||
|
||||
**暴露vistio-api**
|
||||
|
||||
我们使用`kubectl port-forward`将vistio-api暴露到[http://localhost:9191](http://localhost:9191/)。
|
||||
|
||||
```bash
|
||||
kubectl -n default port-forward $(kubectl -n default get pod -l app=vistio-api -o jsonpath='{.items[0].metadata.name}') 9091:9091 &
|
||||
```
|
||||
|
||||
**验证visito-api**
|
||||
|
||||
vistio-web调用vistio-api来渲染服务网格。访问<http://localhost:9091/graph>您应该会看到类似下列的输出。
|
||||
|
||||
|
||||
|
||||
**暴露Vistio**
|
||||
|
||||
在另一个命令行终端中，暴露Vizcera UI到[http://localhost:8080](http://localhost:8080/)。
|
||||
|
||||
```bash
|
||||
kubectl -n default port-forward $(kubectl -n default get pod -l app=vistio-web -o jsonpath='{.items[0].metadata.name}') 8080:8080 &
|
||||
```
|
||||
|
||||
**访问Vistio**
|
||||
|
||||
如果一切都已经启动并准备就绪，您就可以访问Vistio UI，开始探索服务网格网络，访问[http://localhost:8080](http://localhost:8080/)您将会看到类似下图的输出。
|
||||
|
||||
|
||||
|
||||
### 探索
|
||||
|
||||
在全局范围内，您将看到Istio网格内所有请求的总和。如果您部署了istio-ingressgateway，则可以选择显示通过其他配置从网格外部接收的流量，参考[使用Ingress Gateway部署Vistio](https://github.com/nmnellis/vistio#deploy-vistio-with-istio-ingress-gateway-helm)。
|
||||
|
||||
如果您点击istio-mesh气泡，您将能够查看您的网状网络。
|
||||
|
||||
|
||||
|
||||
在您的Istio网格中，您可以使用许多可视化工具来帮助您查明故障的应用程序。
|
||||
|
||||
|
||||
|
||||
使用屏幕右上方的过滤器可以快速过滤出错误率较高的应用程序。通过高级配置，当错误率超过特定值时，也可以触发警报。警报将显示给定应用程序的当前错误率趋势。
|
||||
|
||||
### 问题排查
|
||||
|
||||
访问<http://localhost:9091/graph>，如果您从vistio-api中看到以下输出，表示某些功能无法正常工作。正确的输出显示在教程上面。
|
||||
|
||||
|
||||
|
||||
**1.** 检查vistio-api日志中是否有错误——在大多数情况下，vistio-api将记录与Prometheus通信时遇到的任何问题。
|
||||
|
||||
```bash
|
||||
kubectl logs -n default -c vistio-api $(kubectl -n default get pod -l app=vistio-api -o jsonpath='{.items[0].metadata.name}')
|
||||
```
|
||||
|
||||
**2.** 验证Prometheus查询——vistio-api使用以下查询检索其数据。您应该确保Prometheus内部的数据都存在。
|
||||
|
||||
```bash
|
||||
# Global Level Query
|
||||
sum(rate(istio_request_count[1m])) by (response_code)
|
||||
# Cluster Level Query
|
||||
sum(rate(istio_request_count[1m])) by (source_service,destination_service,response_code)
|
||||
```
|
||||
|
||||
**3.** 提交Issue——如果遇到问题无法解决请提交Issue：<https://github.com/nmnellis/vistio/issues>
|
||||
|
||||
## 参考
|
||||
|
||||
- https://github.com/nmnellis/vistio
|
||||
- [Vistio—使用Netflix的Vizceral可视化Istio service mesh](https://servicemesher.github.io/blog/vistio-visualize-your-istio-mesh-using-netflixs-vizceral/)
|
||||
Loading…
Reference in New Issue