Clustering methods<\/strong> are used to identify groups of similar objects in a multivariate data sets collected from fields such as marketing, bio-medical and geo-spatial. They are different types of clustering<\/strong> methods, including:<\/p>\n In this article, we provide an overview of clustering methods and quick start R code to perform cluster analysis in R:<\/p>\n Contents:<\/p>\n Contents:<\/p>\n We\u2019ll use mainly two R packages:<\/p>\n Accessory packages:<\/p>\n Install:<\/p>\n Load packages:<\/p>\n Read more: Data Preparation and Essential R Packages for Cluster Analysis<\/a><\/p>\n The classification of objects, into clusters, requires some methods for measuring the distance or the (dis)similarity between the objects. Chapter Clustering Distance Measures Essentials<\/a> covers the common distance measures used for assessing similarity between observations.<\/p>\n It\u2019s simple to compute and visualize distance matrix using the functions get_dist() and fviz_dist()<\/a> [factoextra R package]:<\/p>\n Read more: Clustering Distance Measures Essentials<\/a><\/p>\n<\/div>\n Partitioning algorithms are clustering techniques that subdivide the data sets into a set of k groups, where k is the number of groups pre-specified by the analyst.<\/p>\n There are different types of partitioning clustering methods. The most popular is the K-means clustering<\/a> (MacQueen 1967)<\/span>, in which, each cluster is represented by the center or means of the data points belonging to the cluster. The K-means method is sensitive to outliers.<\/p>\n An alternative to k-means clustering is the K-medoids clustering<\/a> or PAM (Partitioning Around Medoids, Kaufman & Rousseeuw, 1990), which is less sensitive to outliers compared to k-means.<\/p>\n Read more: Partitioning Clustering methods<\/a>.<\/p>\n The following R codes show how to determine the optimal number of clusters and how to compute k-means and PAM clustering in R.<\/p>\n Suggested number of cluster: 3<\/p>\n Compute and visualize k-means clustering<\/a><\/p>\n Similarly, the k-medoids\/pam clustering<\/a> can be computed as follow:<\/p>\n Hierarchical clustering is an alternative approach to partitioning clustering for identifying groups in the dataset. It does not require to pre-specify the number of clusters to be generated.<\/p>\n The result of hierarchical clustering is a tree-based representation of the objects, which is also known as dendrogram. Observations can be subdivided into groups by cutting the dendrogram at a desired similarity level.<\/p>\n R code to compute and visualize hierarchical clustering:<\/p>\n Read more: Hierarchical clustering<\/a><\/p>\n See also:<\/p>\n Clustering validation and evaluation strategies, consist of measuring the goodness of clustering results. Before applying any clustering algorithm to a data set, the first thing to do is to assess the clustering tendency<\/em>. That is, whether the data contains any inherent grouping structure.<\/p>\n If yes, then how many clusters are there. Next, you can perform hierarchical clustering or partitioning clustering (with a pre-specified number of clusters). Finally, you can use a number of measures, described in this chapter, to evaluate the goodness of the clustering results.<\/p>\n Read more: Cluster Validation Essentials<\/a><\/p>\n To assess the clustering tendency, the Hopkins\u2019 statistic and a visual approach can be used. This can be performed using the function R code:<\/p>\n Read more: Assessing Clustering Tendency<\/a><\/p>\n<\/div>\n There are different methods for determining the optimal number of clusters<\/a>.<\/p>\n In the R code below, we\u2019ll use the Read more: Determining the Optimal Number of Clusters<\/a><\/p>\n<\/div>\n A variety of measures has been proposed in the literature for evaluating clustering results. The term clustering validation is used to design the procedure of evaluating the results of a clustering algorithm.<\/p>\n The silhouette plot<\/em> is one of the many measures for inspecting and validating clustering results. Recall that the silhouette (\\(S_i\\)<\/span>) measures how similar an object \\(i\\)<\/span> is to the the other objects in its own cluster versus those in the neighbor cluster. \\(S_i\\)<\/span> values range from 1 to - 1:<\/p>\n In the following R code, we\u2019ll compute and evaluate the result of hierarchical clustering methods.<\/p>\n Read more: Cluster Validation Statistics<\/a><\/p>\n<\/div>\n Fuzzy clustering<\/em> is also known as soft method. Standard clustering approaches produce partitions (K-means, PAM), in which each observation belongs to only one cluster. This is known as hard clustering.<\/p>\n In Fuzzy clustering<\/em>, items can be a member of more than one cluster. Each item has a set of membership coefficients corresponding to the degree of being in a given cluster. The Fuzzy c-means<\/em> method is the most popular fuzzy clustering algorithm.<\/p>\n Read more: Fuzzy Clustering<\/a>.<\/p>\n<\/div>\n In model-based clustering<\/em>, the data are viewed as coming from a distribution that is mixture of two ore more clusters. It finds best fit of models to data and estimates the number of clusters.<\/p>\n Read more: Model-Based Clustering<\/a>.<\/p>\n<\/div>\n DBSCAN is a partitioning method that has been introduced in Ester et al. (1996). It can find out clusters of different shapes and sizes from data containing noise and outliers (Ester et al. 1996)<\/span>. The basic idea behind density-based clustering approach is derived from a human intuitive clustering method.<\/p>\n The description and implementation of DBSCAN in R are provided at this link: DBSCAN: Density-Based Clustering<\/a>.<\/p>\n\n
![\"Types](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/images\/types-of-clustering-methods.png\")
<\/p>\n\n
\n
\n
\n
Related Book <\/a><\/h4>Practical Guide to Cluster Analysis in R<\/div>\n
Installing and loading required R packages<\/h2>\n
\n
\n
install.packages(\"factoextra\")\r\ninstall.packages(\"cluster\")\r\ninstall.packages(\"magrittr\")<\/code><\/pre>\nlibrary(\"cluster\")\r\nlibrary(\"factoextra\")\r\nlibrary(\"magrittr\")<\/code><\/pre>\n<\/div>\nData preparation<\/h2>\n
\n
# Load and prepare the data\r\ndata(\"USArrests\")\r\n\r\nmy_data <- USArrests %>%\r\n na.omit() %>% # Remove missing values (NA)\r\n scale() # Scale variables\r\n\r\n# View the firt 3 rows\r\nhead(my_data, n = 3)<\/code><\/pre>\n## Murder Assault UrbanPop Rape\r\n## Alabama 1.2426 0.783 -0.521 -0.00342\r\n## Alaska 0.5079 1.107 -1.212 2.48420\r\n## Arizona 0.0716 1.479 0.999 1.04288<\/code><\/pre>\n<\/div>\nDistance measures<\/h2>\n
\n
get_dist()<\/code>: for computing a distance matrix between the rows of a data matrix. Compared to the standard dist()<\/code> function, it supports correlation-based distance measures including \u201cpearson\u201d, \u201ckendall\u201d and \u201cspearman\u201d methods.<\/li>\nfviz_dist()<\/code>: for visualizing a distance matrix<\/li>\n<\/ul>\nres.dist <- get_dist(USArrests, stand = TRUE, method = \"pearson\")\r\n\r\nfviz_dist(res.dist, \r\n gradient = list(low = \"#00AFBB\", mid = \"white\", high = \"#FC4E07\"))<\/code><\/pre>\n![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/figures\/060-types-of-clustering-methods-distance-matrix-1.png\")
<\/p>\nPartitioning clustering<\/h2>\n
\n
factoextra::fviz_nbclust()<\/code><\/li>\n<\/ol>\nlibrary(\"factoextra\")\r\nfviz_nbclust(my_data, kmeans, method = \"gap_stat\")<\/code><\/pre>\n![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/figures\/060-types-of-clustering-methods-optimal-number-of-clusters-1.png\")
<\/p>\nset.seed(123)\r\nkm.res <- kmeans(my_data, 3, nstart = 25)\r\n# Visualize\r\nlibrary(\"factoextra\")\r\nfviz_cluster(km.res, data = my_data,\r\n ellipse.type = \"convex\",\r\n palette = \"jco\",\r\n ggtheme = theme_minimal())<\/code><\/pre>\n![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/figures\/060-types-of-clustering-methods-k-means-plot-ggplot2-factoextra-1.png\")
<\/p>\n# Compute PAM\r\nlibrary(\"cluster\")\r\npam.res <- pam(my_data, 3)\r\n# Visualize\r\nfviz_cluster(pam.res)<\/code><\/pre>\n<\/div>\nHierarchical clustering<\/h2>\n
# Compute hierarchical clustering\r\nres.hc <- USArrests %>%\r\n scale() %>% # Scale the data\r\n dist(method = \"euclidean\") %>% # Compute dissimilarity matrix\r\n hclust(method = \"ward.D2\") # Compute hierachical clustering\r\n\r\n# Visualize using factoextra\r\n# Cut in 4 groups and color by groups\r\nfviz_dend(res.hc, k = 4, # Cut in four groups\r\n cex = 0.5, # label size\r\n k_colors = c(\"#2E9FDF\", \"#00AFBB\", \"#E7B800\", \"#FC4E07\"),\r\n color_labels_by_k = TRUE, # color labels by groups\r\n rect = TRUE # Add rectangle around groups\r\n )<\/code><\/pre>\n![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/figures\/060-types-of-clustering-methods-hierarchical-clustering-r-1.png\")
<\/p>\n\n
Clustering validation and evaluation<\/h2>\n
Assessing clustering tendency<\/h3>\n
get_clust_tendency()<\/code> [factoextra package], which creates an ordered dissimilarity image (ODI).<\/p>\n\n
gradient.color <- list(low = \"steelblue\", high = \"white\")\r\n\r\niris[, -5] %>% # Remove column 5 (Species)\r\n scale() %>% # Scale variables\r\n get_clust_tendency(n = 50, gradient = gradient.color)<\/code><\/pre>\n## $hopkins_stat\r\n## [1] 0.8\r\n## \r\n## $plot<\/code><\/pre>\n![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/figures\/060-types-of-clustering-methods-clustering-tendency-1.png\")
<\/p>\nDetermining the optimal number of clusters<\/h3>\n
NbClust<\/code> R package, which provides 30 indices for determining the best number of clusters. First, install it using install.packages(\"NbClust\")<\/code>, then type this:<\/p>\nset.seed(123)\r\n\r\n# Compute\r\nlibrary(\"NbClust\")\r\nres.nbclust <- USArrests %>%\r\n scale() %>%\r\n NbClust(distance = \"euclidean\",\r\n min.nc = 2, max.nc = 10, \r\n method = \"complete\", index =\"all\") <\/code><\/pre>\n# Visualize\r\nlibrary(factoextra)\r\nfviz_nbclust(res.nbclust, ggtheme = theme_minimal())<\/code><\/pre>\n## Among all indices: \r\n## ===================\r\n## * 2 proposed 0 as the best number of clusters\r\n## * 1 proposed 1 as the best number of clusters\r\n## * 9 proposed 2 as the best number of clusters\r\n## * 4 proposed 3 as the best number of clusters\r\n## * 6 proposed 4 as the best number of clusters\r\n## * 2 proposed 5 as the best number of clusters\r\n## * 1 proposed 8 as the best number of clusters\r\n## * 1 proposed 10 as the best number of clusters\r\n## \r\n## Conclusion\r\n## =========================\r\n## * According to the majority rule, the best number of clusters is 2 .<\/code><\/pre>\n![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/figures\/060-types-of-clustering-methods-determine-the-number-of-clusters-nbclust-1.png\")
<\/p>\nClustering validation statistics<\/h3>\n
\n
\n
set.seed(123)\r\n# Enhanced hierarchical clustering, cut in 3 groups\r\nres.hc <- iris[, -5] %>%\r\n scale() %>%\r\n eclust(\"hclust\", k = 3, graph = FALSE)\r\n\r\n# Visualize with factoextra\r\nfviz_dend(res.hc, palette = \"jco\",\r\n rect = TRUE, show_labels = FALSE)<\/code><\/pre>\n![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/figures\/060-types-of-clustering-methods-hierarchical-clustering-1.png\")
<\/p>\n\n
fviz_silhouette(res.hc)<\/code><\/pre>\n## cluster size ave.sil.width\r\n## 1 1 49 0.63\r\n## 2 2 30 0.44\r\n## 3 3 71 0.32<\/code><\/pre>\n![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/figures\/060-types-of-clustering-methods-silhouette-plot-1.png\")
<\/p>\n\n
# Silhouette width of observations\r\nsil <- res.hc$silinfo$widths[, 1:3]\r\n\r\n# Objects with negative silhouette\r\nneg_sil_index <- which(sil[, 'sil_width'] < 0)\r\nsil[neg_sil_index, , drop = FALSE]<\/code><\/pre>\n## cluster neighbor sil_width\r\n## 84 3 2 -0.0127\r\n## 122 3 2 -0.0179\r\n## 62 3 2 -0.0476\r\n## 135 3 2 -0.0530\r\n## 73 3 2 -0.1009\r\n## 74 3 2 -0.1476\r\n## 114 3 2 -0.1611\r\n## 72 3 2 -0.2304<\/code><\/pre>\nSee also:<\/h3>\n
\n
Advanced clustering methods<\/h2>\n
Hybrid clustering methods<\/h3>\n
\n
![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/figures\/060-types-of-clustering-methods-hcpc-1.png\")
<\/p>\n<\/div>\nFuzzy clustering<\/h3>\n
Model-based clustering<\/h3>\n
DBSCAN: Density-based clustering<\/h3>\n
![\"Density](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/images\/dbscan-idea.png\")
<\/p>\n![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/cluster-analysis\/figures\/060-types-of-clustering-methods-dbscan-1.png\")
<\/p>\n<\/div>\n<\/div>\n