O'Reilly、Cloudera 主办
Make Data Work
2017年7月12-13日:培训
2017年7月13-15日:会议
北京,中国

使用开源的Alluxio解耦计算与存储的架构 (The architecture of decoupling compute and storage with open source Alluxio)

此演讲使用中文 (This will be presented in Chinese)

Yupeng Fu (Alluxio)
11:15–11:55 Saturday, 2017-07-15
数据工程和架构 (Data engineering and architecture)
地点: 多功能厅6A+B(Function Room 6A+B) 观众水平 (Level): Non-technical

必要预备知识 (Prerequisite Knowledge)

A basic understanding of big data platform concepts

您将学到什么 (What you'll learn)

Understand the benefits of decoupling storage and computation

描述 (Description)

随着Spark、MapReduce和许多的框架在企业生产系统中得到广泛的部署,高效、灵活的计算和存储的架构成为IT和LOB从业者之间争论的热门话题。正如O’Reilly在2017年的最新趋势中指出,尽管在作为数据湖一部分的传统的超聚合环境中运行计算任务是有很好的理由,但是存储和计算的解耦正变得越来越受欢迎。例如,Alluxio、IBM、华为、EMC和Redhat的团队正联合在一起检查现实世界的应用案例并提供联合解决方案。在本演示中,我们将分享相应的决策依据和考虑的因素,如应用的工作负载模式、数据本地性、基础设施的成本、网络带宽以及云部署模式等。我们将共享生产系统里的最佳实践和解决方案。它们能最好地利用CPU、内存和不同层次的计算和存储系统的分离,以建立一个解决现实世界业务需求的多租户高性能平台。


Frameworks like Spark and MapReduce are increasingly being widely deployed at enterprise productions. As a result, efficient and flexible compute and storage architecture has become a hot topic for debate among both IT and LOB practitioners. Although there are good reasons to run compute in a traditional hyperconverge environment as a part of a data lake implementation, the decoupling of storage and computation is becoming more and more popular, as O’Reilly pointed out in a recent 2017 trend post. Recently Alluxio, IBM, Huawei, EMC, and Red Hat teams joined together to examine real-world application examples and provide joint solutions.

Yupeng Fu shares production best practices and solutions to best utilize CPUs, memory, and different tiers of disaggregated compute and storage systems to build out a multitenant high-performance platform that addresses real-world business demands. Along the way, Yupeng also outlines decision factors and considerations, such as application workload patterns, data locality, cost of infrastructure, network bandwidth, and cloud deployment, to name just a few.

Photo of Yupeng Fu

Yupeng Fu

Alluxio

Yupeng Fu is a software engineer at Alluxio and a PMC member of the Alluxio open source project. Previously, Yupeng worked at Palantir, where he led the efforts to build the company’s storage solution. Yupeng holds a BS and an MS from Tsinghua University and has completed coursework toward a PhD at UCSD.

联系OReillyData

关注OReillyData微信号获取最新会议信息并浏览前沿数据文章。

WeChat QRcode

 

Stay Connected Image 1
Stay Connected Image 3
Stay Connected Image 2

阅读关于大数据的最新理念。

ORB Data Site