上传者: ssehs
|
上传时间: 2021-01-28 04:09:01
|
文件大小: 829KB
|
文件类型: PDF
We present Mesos, a platform for sharing commodity clusters between multiple diverse cluster computing
frameworks, such as Hadoop and MPI. Sharing improves
cluster utilization and avoids per-framework data replication. Mesos shares resources in a fine-grained manner, allowing frameworks to achieve data locality by
taking turns reading data stored on each machine. To
support the sophisticated schedulers of today’s frameworks, Mesos introduces a distributed two-level scheduling mechanism called resource offers. Mesos decides
how many resources to offer each framework, while
frameworks decide which resources to accept and which
computations to run on them. Our results show that
Mesos can achieve near-optimal data locality when sharing the cluster among diverse frameworks, can scale to
50,000 (emulated) nodes, and is resilient to failures.