Comparison of Distributed K-Means and Distributed Fuzzy C-Means Algorithms for Text Clustering

Article Sidebar

PDF
DOI: https://doi.org/10.21924/cst.2.1.2017.46
Keywords:
K-Means, FCM, Tanimoto Distance, MapReduce, Hadoop

Main Article Content

I Made Artha Agastya
Universitas Gadjah Mada
Teguh Bharata Adji
Universitas Gadjah Mada
Noor Akhmad Setiawan
Universitas Gadjah Mada

Abstract

Text clustering has been developed in distributed system due to increasing data. The popular algorithms like K-Means (KM) and Fuzzy C-Means (FCM) are combined with MapReduce algorithm in Hadoop Environment to be distributable and parallelizable. The problem is performance comparison between Distributed KM (DKM) and Distributed FCM (DFCM) that use Tanimoto Distance Measure (TDM) has not been studied yet. It is important because TDM’s characteristics are scale invariant while allowing discrimination collinear vectors. This work compared the combination of TDM with DKM (DKM-T) and TDM with DFCM (DFCM-T) to acquire performance of both algorithms. The result shows that DFCM-T has better intra-cluster and inter-cluster densities than those of DKM-T. Moreover, DFCM-T has lower processing time than that of DKM-T when total nodes used are 4 and 8. DFCM-T and DKM-T could perform clustering of 1,400,000 text files in 16.18 and 9.74 minutes but the preprocessing times take hours.

Downloads

Download data is not yet available.

Article Details

How to Cite
Agastya, I. M. A., Adji, T. B., & Setiawan, N. A. (2017). Comparison of Distributed K-Means and Distributed Fuzzy C-Means Algorithms for Text Clustering. Communications in Science and Technology, 2(1). https://doi.org/10.21924/cst.2.1.2017.46
Issue
Vol. 2 No. 1 (2017)
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright

Open Access authors retain the copyrights of their papers, and all open access articles are distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution and reproduction in any medium, provided that the original work is properly cited.

The use of general descriptive names, trade names, trademarks, and so forth in this publication, even if not specifically identified, does not imply that these names are not protected by the relevant laws and regulations.

While the advice and information in this journal are believed to be true and accurate on the date of its going to press, neither the authors, the editors, nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Crossref
Scopus
Google Scholar
Europe PMC

References

1. X. Wu, X. Zhu, G. Wu, and W. Ding, “Data mining with big data,” Knowl. Data …, vol. 26, no. 1, pp. 97–107, 2014.
2. A. Gandomi and M. Haider, “Beyond the hype: Big data concepts, methods, and analytics,” Int. J. Inf. Manage., vol. 35, no. 2, pp. 137–144, 2015.
3. T. White, Hadoop : The Definitive Guide. 2015.
4. S. Sathya and N. Rajendran, “A Review on Text Mining Techniques,” Int. J. Comput. Sci. Trends Technol., vol. 3, no. 5, pp. 274–284, 2013.
5. R. C. Esteves Rui, “Using Mahout for clustering Wikipedia’s latest articles: A comparison between k-means and fuzzy c-means in the cloud,” Proc. - 2011 3rd IEEE Int. Conf. Cloud Comput. Technol. Sci. CloudCom 2011, pp. 565–569, 2011.
6. S. Madhukumar and N. Santhiyakumari, “Evaluation of k-Means and fuzzy C-means segmentation on MR images of brain,” Egypt. J. Radiol. Nucl. Med., vol. 46, no. 2, pp. 475–479, 2015.
7. S. K. Sahu and S. K. Jena, “A Study of K-Means and C-Means Clustering Algorithms for Intrusion Detection Product Development,” Int. J. Innov. Manag. Technol., vol. 5, no. 3, pp. 207–213, 2014.
8. S. Panda, S. Sahu, P. Jena, and S. Chattopadhyay, “Comparing fuzzy-C means and K-means clustering techniques: A comprehensive study,” Adv. Intell. Soft Comput., vol. 166 AISC, no. VOL. 1, pp. 451–460, 2012.
9. L. Sahu and B. R. Mohan, “An improved K-means algorithm using modified cosine distance measure for document clustering using Mahout with Hadoop,” 9th Int. Conf. Ind. Inf. Syst. ICIIS 2014, 2015.
10. E. Jain and S. K. Jain, “Using Mahout for clustering similar Twitter users: Performance evaluation of k-means and its comparison with fuzzy k-means,” Proc. - 5th IEEE Int. Conf. Comput. Commun. Technol. ICCCT 2014, pp. 29–33, 2015.
11. E. Jain and S. K. Jain, “Categorizing twitter users on the basis of their interests using hadoop/mahout platform,” 9th Int. Conf. Ind. Inf. Syst. ICIIS 2014, 2015.
12. P. Muniz De Avila et al., “Comparing K-Means and Mean Shift Algorithms Performance Using Mahout in a Private Cloud Environment,” J. Commun. Comput., vol. 11, pp. 45–51, 2014.
13. S. Owen, R. Anil, T. Dunning, and E. Friedman, Mahout in Action. 2011.
14. J. Ghosh and A. Strehl, “Similarity-based text clustering: A comparative study,” Group. Multidimens. Data Recent Adv. Clust., no. ii, pp. 73–97, 2006.
15. A. Huang, “Similarity measures for text document clustering,” Proc. Sixth New Zeal., no. April, pp. 49–56, 2008.
16. A. S. Joydeep, E. Strehl, J. Ghosh, R. Mooney, and A. Strehl, “Impact of Similarity Measures on Web-page Clustering,” Work. Artif. Intell. Web Search (AAAI 2000), 2000.
17. A. Rangrej, “Comparative Study of Clustering Techniques for Short Text Documents,” Media, pp. 111–112, 2011.
18. X. Zhang, J. Zhao, and Y. LeCun, “Character-level Convolutional Networks for Text Classification,” Proc. Annu. Conf. Int. Speech Commun. Assoc. INTERSPEECH, pp. 3057–3061, 2015.
19. M. Eroglu S., Toprak S., Urgan O, MD, Ozge E. Onur, MD, Arzu Denizbasi, MD, Haldun Akoglu, MD, Cigdem Ozpolat, MD, Ebru Akoglu, Hadoop Solutions, vol. 33. 2012.
20. ASF, “Apache Mahout: Scalable machine learning and data mining.” 2016.