Study of Parallelized Graph Clustering Algorithm Based on Spark

Authors

Dongjiang Liu, 1. Computer Network Information Center, Chinese Academy of Sciences, Beijing 100190, China;;2. University of Chinese Academy of Sciences, Beijing 100190, China;
Jianhui Li, 1. Computer Network Information Center, Chinese Academy of Sciences, Beijing 100190, China;;

Abstract

Abstract: The parallelized graph clustering algorithm is researched. A new parallelized graph clustering algorithm is proposed based on Spark. As the top operation of Spark occupies a lot of memory space, a new algorithm which is used to substitute the top operation is proposed to reduce the memory consumption. By improving bottom up hierarchical clustering algorithm, the speed of the proposed algorithm is improved. A new data filtering method based on the feature of graph data is proposed. By the method, the running time and memory space comsuption is reduced greatly. The reason of the high efficiency of this filtering method is explained. Simulation result indicates that the proposed algorithm is better than other parallelized graph clustering algorithms.

Recommended Citation

Liu, Dongjiang and Li, Jianhui (2020) "Study of Parallelized Graph Clustering Algorithm Based on Spark," Journal of System Simulation: Vol. 32: Iss. 6, Article 7.
DOI: 10.16182/j.issn1004731x.joss.18-0722
Available at: https://dc-china-simulation.researchcommons.org/journal/vol32/iss6/7

First Page

1038

Revised Date

2019-03-02

DOI Link

https://doi.org/10.16182/j.issn1004731x.joss.18-0722

Last Page

1050

CLC

TP391

Recommended Citation

Liu Dongjiang, Li Jianhui. Study of Parallelized Graph Clustering Algorithm Based on Spark[J]. Journal of System Simulation, 2020, 32(6): 1038-1050.

DOI

10.16182/j.issn1004731x.joss.18-0722