Print Email Facebook Twitter Design and Experimental Evaluation of Distributed Heterogeneous Graph-Processing Systems Title Design and Experimental Evaluation of Distributed Heterogeneous Graph-Processing Systems Author Guo, Y. (TU Delft Data-Intensive Systems) Varbanescu, A.L. (Universiteit van Amsterdam) Epema, D.H.J. (TU Delft Data-Intensive Systems) Iosup, A. (TU Delft Data-Intensive Systems) Date 2016-07-21 Abstract Graph processing is increasingly used in a variety of domains, from engineering to logistics and from scientific computing to online gaming. To process graphs efficiently, GPU-enabled graph-processing systems such as TOTEM and Medusa exploit the GPU or the combined CPU+GPU capabilities of a single machine. Unlike scalable distributed CPU-based systems such as Pregel and GraphX, existing GPU-enabled systems are restricted to the resources of a single machine, including the limited amount of GPU memory, and thus cannot analyze the increasingly large-scale graphs we see in practice. To address this problem, we design and implement three families of distributed heterogeneous graph-processing systems that can use both the CPUs and GPUs of multiple machines. We further focus on graph partitioning, for which we compare existing graph-partitioning policies and a new policy specifically targeted at heterogeneity. We implement all our distributed heterogeneous systems based on the programming model of the single-machine TOTEM, to which we add (1) a new communication layer for CPUs and GPUs across multiple machines to support distributed graphs, and (2) a workload partitioning method that uses offline profiling to distribute the work on the CPUs and the GPUs. We conduct a comprehensive real-world performance evaluation for all three families. To ensure representative results, we select 3 typical algorithms and 5 datasets with different characteristics. Our results include algorithm run time, performance breakdown, scalability, graph partitioning time, and comparison with other graph-processing systems. They demonstrate the feasibility of distributed heterogeneous graph processing and show evidence of the high performance that can be achieved by combining CPUs and GPUs in a distributed environment. Subject Distributed Heterogeneous SystemsGraph Processing To reference this document use: http://resolver.tudelft.nl/uuid:c5534962-50ed-49c1-b279-e3ffe7799658 DOI https://doi.org/10.1109/ccgrid.2016.53 Publisher IEEE, Los Alamitos, CA ISBN 978-1-5090-2453-7 Source Proceedings - 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2016 Event CCGRID 2016, 2016-05-16 → 2016-05-19, Cartagena de Indias, Cartagena, Colombia Part of collection Institutional Repository Document type conference paper Rights © 2016 Y. Guo, A.L. Varbanescu, D.H.J. Epema, A. Iosup Files PDF CCGRID_2016_paper_53.pdf 374.19 KB Close viewer /islandora/object/uuid:c5534962-50ed-49c1-b279-e3ffe7799658/datastream/OBJ/view