remove mesos

Signed-off-by: Kang Huaishuai <khs1994@khs1994.com>
pull/481/head
Kang Huaishuai 2020-12-20 22:58:09 +08:00
parent fb162df033
commit 2cd0fac44b
No known key found for this signature in database
GPG Key ID: 5E515022F565DA09
14 changed files with 0 additions and 883 deletions

View File

@ -87,7 +87,6 @@ jobs:
cp -r install/_images .vuepress/dist/install
cp -r introduction/_images .vuepress/dist/introduction
cp -r kubernetes/_images .vuepress/dist/kubernetes
cp -r mesos/_images .vuepress/dist/mesos
cp -r underly/_images .vuepress/dist/underly
echo "include: [_images]" > .vuepress/dist/_config.yml

View File

@ -170,14 +170,6 @@
* [ Docker](appendix/debug.md)
* [](appendix/resources.md)
* [](archive/README.md)
* [Mesos - ](mesos/README.md)
* [Mesos ](mesos/intro.md)
* [使](mesos/installation.md)
* [](mesos/architecture.md)
* [Mesos ](mesos/configuration.md)
* [](mesos/monitor.md)
* [](mesos/framework.md)
* [](mesos/summary.md)
* [Docker Machine](machine/README.md)
* [](machine/install.md)
* [使](machine/usage.md)

View File

@ -1,6 +0,0 @@
# Mesos -
Mesos UC Berkeley 使
Mesos Docker Mesos 线
Mesos 使

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 28 KiB

View File

@ -1,83 +0,0 @@
##
Mesos Mesos Mesos Mesos Mesos 使
Mesos 使
###
Mesos
![mesos ](_images/mesos-architecture.png)
Mesos -master-slave使 zookeeper HA
Mesos master Mesos slave
Mesos master
###
Mesos masterslaveframework
#### - master
使
API
#### - slave
LXCDocker
#### - framework
* scheduler
* executor使 Mesos shell Docker
HadoopSpark MPI
###
使
Mesos two-layer
####
master frameworkframework Mesos master framework Mesos master
Dominant Resource FairnessDRF
*DRF Dominant Resource Fairness: Fair Allocation of Multiple Resource Types*
####
offer offer `<1 CPU, 2 GB Mem>`
* slave master
* master offer framework
* framework offer 使使 master
* master framework slave使 framework executor
`<4 CPU, 8 GB Mem>` `<3 CPU, 6 GB Mem>` offer `<slave#1, 4 CPU, 8 GB Mem>` offer `<1 CPU, 2 GB Mem>` `<2 CPU, 4 GB Mem>`
offer offer 使 slave offer master offer
####
framework master offer
####
Mesos
### HA
master 使 ZooKeeper Mesos master
Mesos master slave framework
master slave Mesos

View File

@ -1,172 +0,0 @@
## Mesos
Mesos
Mesos master slave master slave
Mesos
###
* `--advertise_ip=VALUE` 访访 master
* `--advertise_port=VALUE` 访
* `--external_log_file=VALUE` Web
* `--firewall_rules=VALUE` endpoint `VALUE` JSON JSON
* `--ip=VALUE` IP
* `--log_dir=VALUE`
* `--logbufsecs=VALUE` buffer
* `--logging_level=VALUE`
* `--port=VALUE` master 5050slave 5051
### master
Mesos master
`mesos-master --help`
* `--quorum=VALUE` 使 replicated-Log ZooKeeper HA
* `--work_dir=VALUE`
* `--zk=VALUE` HA ZooKeepr `zk://username:password@host1:port1,host2:port2,.../path`
* `--acls=VALUE` ACL
* `--allocation_interval=VALUE` allocation 1sec
* `--allocator=VALUE` HierarchicalDRF
* `--[no-]authenticate` framework
* `--[no-]authenticate_slaves` slaves
* `--authenticators=VALUE` framework salves
* `--cluster=VALUE` Web
* `--credentials=VALUE`
* `--external_log_file=VALUE`
* `--framework_sorter=VALUE` framework
* `--hooks=VALUE` master hook
* `--hostname=VALUE` master 使
* `--[no-]log_auto_initialize` replicated
* `--modules=VALUE` JSON
* `--offer_timeout=VALUE` offer
* `--rate_limits=VALUE` framework query per second (qps)
* `--recovery_slave_removal_limit=VALUE` slave master 100%
* `--slave_removal_rate_limit=VALUE slave` 1/10mins
* `--registry=VALUE` `replicated_log` `in_memory`
* `--registry_fetch_timeout=VALUE` 访
* `--registry_store_timeout=VALUE`
* `--[no-]registry_strict` false
* `--roles=VALUE` framework
* `--[no-]root_submissions` root framework true
* `--slave_reregister_timeout=VALUE` lead master slave salve 10mins
* `--user_sorter=VALUE` drf
* `--webui_dir=VALUE` webui `/usr/local/share/mesos/webui`
* `--weights=VALUE`
* `--whitelist=VALUE` offer slave None
* `--zk_session_timeout=VALUE` session 10secs
* `--max_executors_per_slave=VALUE` `--with-network-isolator` slave
master `/tmp/mesos` `mesos_cluster`
```sh
mesos-master \
--zk=zk://10.0.0.2:2181,10.0.0.3:2181,10.0.0.4:2181/mesos \
--quorum=2 \
--work_dir=/tmp/mesos \
--cluster=mesos_cluster
```
### slave
slave
`mesos-slave --help`
* `--master=VALUE` master ZooKeeper
* `--attributes=VALUE`
* `--authenticatee=VALUE` master
* `--[no-]cgroups_enable_cfs` CFS CPU false
* `--cgroups_hierarchy=VALUE` cgroups `/sys/fs/cgroup`
* `--[no-]cgroups_limit_swap` swap false
* `--cgroups_root=VALUE` cgroups mesos
* `--container_disk_watch_interval=VALUE`
* `--containerizer_path=VALUE` `--isolation=external`
* `--containerizers=VALUE` mesosexternaldocker
* `--credential=VALUE`
* `--default_container_image=VALUE` 使
* `--default_container_info=VALUE`
* `--default_role=VALUE`
* `--disk_watch_interval=VALUE` 使 1mins
* `--docker=VALUE` docker
* `--docker_remove_delay=VALUE` 6hrs
* `--[no-]docker_kill_orphans` true
* `--docker_sock=VALUE` docker sock `/var/run/docker.sock`
* `--docker_mesos_image=VALUE` slave docker docker slave docker
* `--docker_sandbox_directory=VALUE` sandbox
* `--docker_stop_timeout=VALUE` kill 0secs
* `--[no-]enforce_container_disk_quota` false
* `--executor_registration_timeout=VALUE` slave 1mins
* `--executor_shutdown_grace_period=VALUE` 5secs
* `--external_log_file=VALUE`
* `--fetcher_cache_size=VALUE` fetcher cache 2 GB
* `--fetcher_cache_dir=VALUE` fetcher cache /tmp/mesos/fetch
* `--frameworks_home=VALUE`
* `--gc_delay=VALUE` 1weeks
* `--gc_disk_headroom=VALUE` 0.1
* `--hadoop_home=VALUE` hadoop HADOOP_HOME
* `--hooks=VALUE` master hook
* `--hostname=VALUE` slave 使
* `--isolation=VALUE` `posix/cpu,posix/mem` `cgroups/cpu,cgroups/mem``external`
* `--launcher_dir=VALUE` mesos `/usr/local/lib/mesos`
* `--image_providers=VALUE` 'APPC,DOCKER'
* `--oversubscribed_resources_interval=VALUE` slave
* `--modules=VALUE` JSON
* `--perf_duration=VALUE` perf perf_interval 10secs
* `--perf_events=VALUE` perf
* `--perf_interval=VALUE` perf
* `--qos_controller=VALUE` QoS
* `--qos_correction_interval_min=VALUE` Qos 0secs
* `--recover=VALUE` reconnectcleanup 退
* `--recovery_timeout=VALUE` slave 退 15mins
* `--registration_backoff_factor=VALUE` master 1secs 1mins
* `--resource_monitoring_interval=VALUE` 使 1secs
* `--resources=VALUE` slave [31000, 32000]
* `--[no-]revocable_cpu_low_priority` CPU true
* `--slave_subsystems=VALUE` slave cgroup memorycpuacct
* `--[no-]strict` true
* `--[no-]switch_user` true
* `--work_dir=VALUE` framework /tmp/mesos
`--with-network-isolator` 使 ` --with-network-isolator`
* `--ephemeral_ports_per_container=VALUE` 2 1024
* `--eth0_name=VALUE` public
* `--lo_name=VALUE` loopback
* `--egress_rate_limit_per_container=VALUE` fq_codel
* `--[no-]-egress_unique_flow_per_container` false
* `--[no-]network_enable_socket_statistics` socket false
slave Docker `10.0.0.10` 16 CPU64 GB [31000-32000] [32768-57344] 512 50 MB/s
```sh
mesos-slave \
--master=zk://10.0.0.2:2181,10.0.0.3:2181,10.0.0.4:2181/mesos \
--containerizers=docker \
--ip=10.0.0.10 \
--isolation=cgroups/cpu,cgroups/mem,network/port_mapping \
--resources=cpus:16;mem:64000;ports:[31000-32000];ephemeral_ports:[32768-57344] \
--ephemeral_ports_per_container=512 \
--egress_rate_limit_per_container=50000KB \
--egress_unique_flow_per_container
```
```sh
$ echo "57345 61000" > /proc/sys/net/ipv4/ip_local_port_range
```
* Mesos 使*

View File

@ -1,89 +0,0 @@
##
Mesos `应用` Mesos master 使
Mesos PaaS
Mesos Mesos
###
#### [Aurora](http://aurora.incubator.apache.org)
Mesos
REST webUI8081
#### [Marathon](https://github.com/mesosphere/marathon)
PaaS
bash
REST webUI8080
#### [Singularity](https://github.com/HubSpot/Singularity)
PaaS
REST webUI70998080
###
#### [Cray Chapel](https://github.com/nqn/mesos-chapel)
Chapel
#### [Dpark](https://github.com/douban/dpark)
Spark Python
#### [Hadoop](https://github.com/mesos/hadoop)
map-reduce
#### [Spark](http://spark.apache.org)
Hadoop 使
#### [Storm](https://github.com/mesos/storm)
###
#### [Chronos](https://github.com/airbnb/chronos)
Cron
#### [Jenkins](https://github.com/jenkinsci/mesos-plugin)
CI 使 mesos-jenkins jenkins Mesos
#### [JobServer](http://www.grandlogic.com/content/html_docs/jobserver.html)
Java
#### [GoDocker](https://bitbucket.org/osallou/go-docker)
Docker Mesos KubernetesSwarm
###
#### [ElasticSearch](https://github.com/mesos/elasticsearch)
APIElasticSearch + LogStash + Kibana ELK
#### [Hypertable](https://code.google.com/p/hypertable)
#### [Tachyon](http://tachyon-project.org/)
访

View File

@ -1,387 +0,0 @@
## Mesos 使
Mesos Marathon Mesos
Marathon Mesos framework Scala
Mesos ZooKeeper Mesos master 3 5 使 supervisord
ZooKeeper `leader`
###
mesoszookeeper marathon
Mesos -mesos master zookeeper mesos slave marathon
Docker
####
apache.org
```sh
$ git clone https://git-wip-us.apache.org/repos/asf/mesos.git
```
`src` `frameworks` `docs` `include ` Mesos 使 API
Java Linux
```sh
$ sudo apt-get update
$ sudo apt-get install -y openjdk-8-jdk autoconf libtool \
build-essential python-dev python-boto libcurl4-nss-dev \
libsasl2-dev maven libapr1-dev libsvn-dev
```
C++ configure Makefile
```sh
$ cd mesos
$ ./bootstrap
$ mkdir build
$ cd build && ../configure --with-network-isolator
$ make
$ make check && sudo make install
```
####
Ubuntu
```sh
$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF
$ DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
$ CODENAME=$(lsb_release -cs)
$ echo "deb http://repos.mesosphere.io/${DISTRO} ${CODENAME} main" | \
sudo tee /etc/apt/sources.list.d/mesosphere.list
```
zookeepermesosmarathon
```sh
$ sudo apt-get -y update && sudo apt-get -y install zookeeper mesos marathon
```
Marathon jdk 1.8+ Java JDK
```sh
$ sudo update-alternatives --config java
```
Mesos `/usr/sbin/` `mesos-master` `mesos-slave`
`service` 便
Mesos
```sh
$ sudo service mesos-master restart
```
`service` `/usr/bin/mesos-init-wrapper`
#### [ Docker](https://github.com/sekka1/mesosphere-docker)
* ZooKeeperhttps://registry.hub.docker.com/u/garland/zookeeper/
* Mesoshttps://registry.hub.docker.com/u/garland/mesosphere-docker-mesos-master/
* Marathonhttps://registry.hub.docker.com/u/garland/mesosphere-docker-marathon/
mesos-master master slave 使
```sh
$ docker pull garland/zookeeper
$ docker pull garland/mesosphere-docker-mesos-master
$ docker pull garland/mesosphere-docker-marathon
```
```sh
$ HOST_IP=10.0.0.2
```
Zookeepr
```sh
docker run -d \
-p 2181:2181 \
-p 2888:2888 \
-p 3888:3888 \
garland/zookeeper
```
Mesos Master
```sh
docker run --net="host" \
-p 5050:5050 \
-e "MESOS_HOSTNAME=${HOST_IP}" \
-e "MESOS_IP=${HOST_IP}" \
-e "MESOS_ZK=zk://${HOST_IP}:2181/mesos" \
-e "MESOS_PORT=5050" \
-e "MESOS_LOG_DIR=/var/log/mesos" \
-e "MESOS_QUORUM=1" \
-e "MESOS_REGISTRY=in_memory" \
-e "MESOS_WORK_DIR=/var/lib/mesos" \
-d \
garland/mesosphere-docker-mesos-master
```
Marathon
```sh
docker run \
-d \
-p 8080:8080 \
garland/mesosphere-docker-marathon --master zk://${HOST_IP}:2181/mesos --zk zk://${HOST_IP}:2181/marathon
```
Mesos slave
```sh
docker run -d \
--name mesos_slave_1 \
--entrypoint="mesos-slave" \
-e "MESOS_MASTER=zk://${HOST_IP}:2181/mesos" \
-e "MESOS_LOG_DIR=/var/log/mesos" \
-e "MESOS_LOGGING_LEVEL=INFO" \
garland/mesosphere-docker-mesos-master:latest
```
访 8080 使 Marathon
###
#### ZooKeepr
ZooKeepr 2181 ZooKeeper
`/etc/zookeeper/conf/` `myid` `zoo.cfg`
myid ZooKeeper 1-255`/var/lib/zookeeper/myid`
1
```sh
$ echo 1 | sudo dd of=/etc/zookeeper/conf/myid
```
ZooKeeper
zoo.cfg ZooKeeper
ZooKeeper `10.0.0.2``10.0.0.3``10.0.0.4` `2``3``4`
```sh
server.2=10.0.0.2:2888:3888
server.3=10.0.0.3:2888:3888
server.4=10.0.0.4:2888:3888
```
2888 3888
`/etc/hosts`
ZooKeeper
```sh
$ sudo service zookeeper start
```
#### Mesos
Mesos
* /etc/mesos/ zk
* /etc/mesos-master/ mesos-master
* /etc/mesos-slave/ mesos-master
`/etc/mesos/zk` ZooKeeper
```sh
zk://10.0.0.2:2181,10.0.0.3:2181,10.0.0.4:2181/mesos
```
`/etc/default/mesos``/etc/default/mesos-master``/etc/default/mesos-slave` Mesos `MESOS_OPTION_NAME`
#####
`/etc/mesos-master/`
mesos-master `mesos-master --help`
`key` `value` mesos-master `--key=value`
mesos-master loopback `127.0.0.1:5050` /etc/mesos-master/ip
mesos-master `work_dir` () /etc/mesos-master/work_dir ` /var/lib/mesos` `replicated_log`
quorum ZooKeeper ZooKeeper `2`
Mesos `/etc/mesos-master/cluster` `MesosCluster`
`/etc/mesos-master` `ip``quorum``work_dir``cluster`
```sh
$ sudo service mesos-master start
```
mesos-slave
#####
`/etc/mesos-slave/`
`/etc/mesos-slave/ip`
```sh
$ sudo service mesos-slave start
```
#### Marathon
Marathon Mesos `--master` `--zk`
/usr/bin marathon shell marathon
`/etc/marathon/conf` `/etc/default/marathon`
Mesos Marathon Mesos
```sh
$ sudo mkdir -p /etc/marathon/conf
$ sudo cp /etc/mesos/zk /etc/marathon/conf/master
```
Marathon ZooKeeper `/etc/marathon/conf/zk` ZooKeeper
```sh
zk://10.0.0.2:2181,10.0.0.2:2181,10.0.0.2:2181/marathon
```
marathon
```sh
$ sudo service marathon start
```
### 访 Mesos
Mesos Web 便
Mesos 访 5050 slave
![mesos slave ](_images/mesos.png)
Slaves
Marathon Frameworks
### 访 Marathon
Marathon Mesos web Frameworks marathon
访 8080 Marathon
![marathon ](_images/marathon.png)
REST API Marathon
![marathon ](_images/marathon_basic0.png)
使 Marathon REST API
JSON API
basic-0
```json
{
"id": "basic-0",
"cmd": "while [ true ] ; do echo 'Hello Marathon' ; sleep 5 ; done",
"cpus": 0.1,
"mem": 10.0,
"instances": 1
}
```
0.1 CPU 10 MB shell `Hello Marathon`
basic-0 Marathon json
```sh
$ curl -X POST http://marathon_host:8080/v2/apps -d @basic-0.json -H "Content-type: application/json"
{"id":"/basic-0","cmd":"while [ true ] ; do echo 'Hello Marathon' ; sleep 5 ; done","args":null,"user":null,"env":{},"instances":1,"cpus":0.1,"mem":10,"disk":0,"executor":"","constraints":[],"uris":[],"storeUrls":[],"ports":[0],"requirePorts":false,"backoffSeconds":1,"backoffFactor":1.15,"maxLaunchDelaySeconds":3600,"container":null,"healthChecks":[],"dependencies":[],"upgradeStrategy":{"minimumHealthCapacity":1,"maximumOverCapacity":1},"labels":{},"acceptedResourceRoles":null,"version":"2015-12-28T05:33:05.805Z","tasksStaged":0,"tasksRunning":0,"tasksHealthy":0,"tasksUnhealthy":0,"deployments":[{"id":"3ec3fbd5-11e4-479f-bd17-813d33e43e0c"}],"tasks":[]}%
```
Marathon REST API `http://marathon_host:8080/api-console/index.html`
Docker Mesos slave slave `--containerizers=docker,mesos`
```json
{
"id": "basic-3",
"cmd": "python3 -m http.server 8080",
"cpus": 0.5,
"mem": 32.0,
"container": {
"type": "DOCKER",
"volumes": [],
"docker": {
"image": "python:3",
"network": "BRIDGE",
"portMappings": [
{
"containerPort": 8080,
"hostPort": 31000,
"servicePort": 0,
"protocol": "tcp"
}
],
"privileged": false,
"parameters": [],
"forcePullImage": true
}
}
}
```
`python:3` `python3 -m http.server 8080` web 宿 31000
hostPort servicePort Mesos slave 31000 ~ 32000使 Marathon
slave Docker `mesos-SLAVE_ID`
```sh
$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1226b4ec8d7d python:3 "/bin/sh -c 'python3 " 3 days ago Up 3 days 0.0.0.0:10000->8080/tcp mesos-06db0fba-49dc-4d28-ad87-6c2d5a020866-S10.b581149e-2c43-46a2-b652-1a0bc10204b3
```

View File

@ -1,20 +0,0 @@
##
Mesos UC Berkeley AMP 2009 Apache Mesosphere Mesos CPU使
Mesos + Mesos Mesosphere Datacenter Operating System (DCOS)
Mesos C++ [https://mesos.apache.org](https://mesos.apache.org),代码仍在快速演化中,已经发布了正式版 1.0.0 版本。
Mesos
* AppleTwittereBay
* MarathonSingularityAurora
* HA ZooKeeper
* DockerLXC
* API PythonJavaC++
* WebUI便
Mesos 使 Mesos framework HadoopSpark Marathon PaaS ElasticSearch
Mesos 便
Mesos

View File

@ -1,110 +0,0 @@
##
Mesos
###
`/var/log/mesos`
使
使 `--log_dir`
###
Mesos 便
####
`http://MASTER_NODE:5050/metrics/snapshot` Mesos CPU使
`10.0.0.2` jq json
```sh
$ curl -s http://10.0.0.2:5050/metrics/snapshot |jq .
{
"system/mem_total_bytes": 4144713728,
"system/mem_free_bytes": 153071616,
"system/load_5min": 0.37,
"system/load_1min": 0.6,
"system/load_15min": 0.29,
"system/cpus_total": 4,
"registrar/state_store_ms/p9999": 45.4096616192,
"registrar/state_store_ms/p999": 45.399272192,
"registrar/state_store_ms/p99": 45.29537792,
"registrar/state_store_ms/p95": 44.8336256,
"registrar/state_store_ms/p90": 44.2564352,
"registrar/state_store_ms/p50": 34.362368,
...
"master/recovery_slave_removals": 1,
"master/slave_registrations": 0,
"master/slave_removals": 0,
"master/slave_removals/reason_registered": 0,
"master/slave_removals/reason_unhealthy": 0,
"master/slave_removals/reason_unregistered": 0,
"master/slave_reregistrations": 2,
"master/slave_shutdowns_canceled": 0,
"master/slave_shutdowns_completed": 1,
"master/slave_shutdowns_scheduled": 1
}
```
####
`http://SLAVE_NODE:5051/metrics/snapshot` Mesos
`10.0.0.10`
```sh
$ curl -s http://10.0.0.10:5051/metrics/snapshot |jq .
{
"system/mem_total_bytes": 16827785216,
"system/mem_free_bytes": 3377315840,
"system/load_5min": 0.11,
"system/load_1min": 0.16,
"system/load_15min": 0.13,
"system/cpus_total": 8,
"slave/valid_status_updates": 11,
"slave/valid_framework_messages": 0,
"slave/uptime_secs": 954125.458927872,
"slave/tasks_starting": 0,
"slave/tasks_staging": 0,
"slave/tasks_running": 1,
"slave/tasks_lost": 0,
"slave/tasks_killed": 2,
"slave/tasks_finished": 0,
"slave/executors_preempted": 0,
"slave/executor_directory_max_allowed_age_secs": 403050.709525191,
"slave/disk_used": 0,
"slave/disk_total": 88929,
"slave/disk_revocable_used": 0,
"slave/disk_revocable_total": 0,
"slave/disk_revocable_percent": 0,
"slave/disk_percent": 0,
"containerizer/mesos/container_destroy_errors": 0,
"slave/container_launch_errors": 6,
"slave/cpus_percent": 0.025,
"slave/cpus_revocable_percent": 0,
"slave/cpus_revocable_total": 0,
"slave/cpus_revocable_used": 0,
"slave/cpus_total": 8,
"slave/cpus_used": 0.2,
"slave/executors_registering": 0,
"slave/executors_running": 1,
"slave/executors_terminated": 8,
"slave/executors_terminating": 0,
"slave/frameworks_active": 1,
"slave/invalid_framework_messages": 0,
"slave/invalid_status_updates": 0,
"slave/mem_percent": 0.00279552715654952,
"slave/mem_revocable_percent": 0,
"slave/mem_revocable_total": 0,
"slave/mem_revocable_used": 0,
"slave/mem_total": 15024,
"slave/mem_used": 42,
"slave/recovery_errors": 0,
"slave/registered": 1,
"slave/tasks_failed": 6
}
```
`http://MASTER_NODE:5050/monitor/statistics.json`

View File

@ -1,7 +0,0 @@
##
Mesos 使 Mesos Mesos 使 DockerMesos
Mesos
Mesos