ISBN: 978-981-11-3671-9 DOI: 10.18178/wcse.2017.06.003
A Distributed Cache Framework on Massively Parallel Computing Architecture
Abstract— Current high performance computers (HPC) mostly adopt massively parallel computing (MPP) architecture, which uses parallel file system to accommodate all the data, instead of having local storage located on the compute node side. MPP architecture prevents large scale applications dumping terabytes of temporal or intermediate data into local storage and causes severe I/O bottleneck. In this paper, we proposed a distributed cache framework to merge scattered memory spaces of up to thousands of compute nodes into a unified cache pool, so that data-intensive applications can store their intermediate data into those compute nodes of light I/O burden. We evaluate the framework with IOR and a realistic benchmark BTIO on a HPC system TH-1A, which indicates that our approach can bring significant performance boost to data-intensive applications.
Index Terms— HPC, Distributed Cache, Data-intensive Application, Cooperate Cache, MPP.
Jie Yu, Guangming Liu, Wenrui Dong, Xiaoyong Li
College of Computer Science, National University of Defense Technology, CHINA
Guangming Liu
National Supercomputer Centre in Tianjin, CHINA
Cite: Jie Yu, Guangming Liu, Wenrui Dong, Xiaoyong Li, "A Distributed Cache Framework on Massively Parallel Computing Architecture," Proceedings of 2017 the 7th International Workshop on Computer Science and Engineering, pp. 17-21, Beijing, 25-27 June, 2017.