{"id":8063,"date":"2018-10-21T11:55:57","date_gmt":"2018-10-21T09:55:57","guid":{"rendered":"https:\/\/www.datanovia.com\/en\/?post_type=dt_lessons&#038;p=8063"},"modified":"2018-10-22T19:04:03","modified_gmt":"2018-10-22T17:04:03","slug":"cluster-validation-statistics-must-know-methods","status":"publish","type":"dt_lessons","link":"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/","title":{"rendered":"Cluster Validation Statistics: Must Know Methods"},"content":{"rendered":"<div id=\"rdoc\">\n<p>The term <strong>cluster validation<\/strong> is used to design the procedure of evaluating the goodness of clustering algorithm results. This is important to avoid finding patterns in a random data, as well as, in the situation where you want to compare two clustering algorithms.<\/p>\n<p>Generally, clustering validation statistics can be categorized into 3 classes <span class=\"citation\">(Charrad et al. 2014,<span class=\"citation\">Brock et al. (2008)<\/span>, <span class=\"citation\">Theodoridis and Koutroumbas (2008)<\/span>)<\/span>:<\/p>\n<ol style=\"list-style-type: decimal;\">\n<li><strong>Internal cluster validation<\/strong>, which uses the internal information of the clustering process to evaluate the goodness of a clustering structure without reference to external information. It can be also used for estimating the number of clusters and the appropriate clustering algorithm without any external data.<\/li>\n<li><strong>External cluster validation<\/strong>, which consists in comparing the results of a cluster analysis to an externally known result, such as externally provided class labels. It measures the extent to which cluster labels match externally supplied class labels. Since we know the \u201ctrue\u201d cluster number in advance, this approach is mainly used for selecting the right clustering algorithm for a specific data set.<\/li>\n<li><strong>Relative cluster validation<\/strong>, which evaluates the clustering structure by varying different parameter values for the same algorithm (e.g.,: varying the number of clusters k). It\u2019s generally used for determining the optimal number of clusters.<\/li>\n<\/ol>\n<div class=\"block\">\n<p>In this chapter, we start by describing the different methods for clustering validation. Next, we\u2019ll demonstrate how to compare the quality of clustering results obtained with different clustering algorithms. Finally, we\u2019ll provide R scripts for validating clustering results.<\/p>\n<\/div>\n<div class=\"notice\">\n<p>In all the examples presented here, we\u2019ll apply k-means, PAM and hierarchical clustering. Note that, the functions used in this article can be applied to evaluate the validity of any other clustering methods.<\/p>\n<\/div>\n<p>Contents:<\/p>\n<div id=\"TOC\">\n<ul>\n<li><a href=\"#internal-measures-for-cluster-validation\">Internal measures for cluster validation<\/a>\n<ul>\n<li><a href=\"#silhouette-coefficient\">Silhouette coefficient<\/a><\/li>\n<li><a href=\"#dunn-index\">Dunn index<\/a><\/li>\n<\/ul>\n<\/li>\n<li><a href=\"#external-measures-for-clustering-validation\">External measures for clustering validation<\/a><\/li>\n<li><a href=\"#computing-cluster-validation-statistics-in-r\">Computing cluster validation statistics in R<\/a>\n<ul>\n<li><a href=\"#required-r-packages\">Required R packages<\/a><\/li>\n<li><a href=\"#data-preparation\">Data preparation<\/a><\/li>\n<li><a href=\"#clustering-analysis\">Clustering analysis<\/a><\/li>\n<li><a href=\"#clustering-validation\">Clustering validation<\/a><\/li>\n<li><a href=\"#external-clustering-validation\">External clustering validation<\/a><\/li>\n<\/ul>\n<\/li>\n<li><a href=\"#summary\">Summary<\/a><\/li>\n<li><a href=\"#references\">References<\/a><\/li>\n<\/ul>\n<\/div>\n<div class='dt-sc-hr-invisible-medium  '><\/div>\n<div class='dt-sc-ico-content type1'><div class='custom-icon' ><a href='https:\/\/www.datanovia.com\/en\/product\/practical-guide-to-cluster-analysis-in-r\/' target='_blank'><span class='fa fa-book'><\/span><\/a><\/div><h4><a href='https:\/\/www.datanovia.com\/en\/product\/practical-guide-to-cluster-analysis-in-r\/' target='_blank'> Related Book <\/a><\/h4>Practical Guide to Cluster Analysis in R<\/div>\n<div class='dt-sc-hr-invisible-medium  '><\/div>\n<div id=\"internal-measures-for-cluster-validation\" class=\"section level2\">\n<h2>Internal measures for cluster validation<\/h2>\n<p>In this section, we describe the most widely used clustering validation indices. Recall that the goal of partitioning clustering algorithms (Part @ref(partitioning-clustering)) is to split the data set into clusters of objects, such that:<\/p>\n<ul>\n<li>the objects in the same cluster are similar as much as possible,<\/li>\n<li>and the objects in different clusters are highly distinct<\/li>\n<\/ul>\n<div class=\"success\">\n<p>That is, we want the average distance within cluster to be as small as possible; and the average distance between clusters to be as large as possible.<\/p>\n<\/div>\n<p>Internal validation measures reflect often the <strong>compactness<\/strong>, the <strong>connectedness<\/strong> and the <strong>separation<\/strong> of the cluster partitions.<\/p>\n<div class=\"block\">\n<ol style=\"list-style-type: decimal;\">\n<li><strong>Compactness<\/strong> or cluster cohesion: Measures how close are the objects within the same cluster. A lower <strong>within-cluster variation<\/strong> is an indicator of a good compactness (i.e., a good clustering). The different indices for evaluating the compactness of clusters are base on distance measures such as the cluster-wise within average\/median distances between observations.<\/li>\n<li><strong>Separation<\/strong>: Measures how well-separated a cluster is from other clusters. The indices used as separation measures include:\n<ul>\n<li>distances between cluster centers<\/li>\n<li>the pairwise minimum distances between objects in different clusters<\/li>\n<\/ul>\n<\/li>\n<li><strong>Connectivity<\/strong>: corresponds to what extent items are placed in the same cluster as their nearest neighbors in the data space. The connectivity has a value between 0 and infinity and should be minimized.<\/li>\n<\/ol>\n<\/div>\n<p>Generally most of the indices used for internal clustering validation combine compactness and separation measures as follow:<\/p>\n<p><span class=\"math display\">\\[<br \/>\nIndex = \\frac{(\\alpha \\times Separation)}{(\\beta \\times Compactness)}<br \/>\n\\]<\/span><\/p>\n<p>Where <span class=\"math inline\">\\(\\alpha\\)<\/span> and <span class=\"math inline\">\\(\\beta\\)<\/span> are weights.<\/p>\n<div class=\"success\">\n<p>In this section, we\u2019ll describe the two commonly used indices for assessing the goodness of clustering: the <strong>silhouette width<\/strong> and the <strong>Dunn index<\/strong>. These internal measure can be used also to determine the optimal number of clusters in the data.<\/p>\n<\/div>\n<div id=\"silhouette-coefficient\" class=\"section level3\">\n<h3>Silhouette coefficient<\/h3>\n<p>The silhouette analysis measures how well an observation is clustered and it estimates the <strong>average distance between clusters<\/strong>. The silhouette plot displays a measure of how close each point in one cluster is to points in the neighboring clusters.<\/p>\n<p>For each observation <span class=\"math inline\">\\(i\\)<\/span>, the silhouette width <span class=\"math inline\">\\(s_i\\)<\/span> is calculated as follows:<\/p>\n<ol style=\"list-style-type: decimal;\">\n<li>For each observation <span class=\"math inline\">\\(i\\)<\/span>, calculate the average dissimilarity <span class=\"math inline\">\\(a_i\\)<\/span> between <span class=\"math inline\">\\(i\\)<\/span> and all other points of the cluster to which i belongs.<\/li>\n<li>For all other clusters <span class=\"math inline\">\\(C\\)<\/span>, to which i does not belong, calculate the average dissimilarity <span class=\"math inline\">\\(d(i, C)\\)<\/span> of <span class=\"math inline\">\\(i\\)<\/span> to all observations of C. The smallest of these <span class=\"math inline\">\\(d(i,C)\\)<\/span> is defined as <span class=\"math inline\">\\(b_i= \\min_C d(i,C)\\)<\/span>. The value of <span class=\"math inline\">\\(b_i\\)<\/span> can be seen as the dissimilarity between <span class=\"math inline\">\\(i\\)<\/span> and its \u201cneighbor\u201d cluster, i.e., the nearest one to which it does not belong.<\/li>\n<li>Finally the silhouette width of the observation <span class=\"math inline\">\\(i\\)<\/span> is defined by the formula: <span class=\"math inline\">\\(S_i = (b_i - a_i)\/max(a_i, b_i)\\)<\/span>.<\/li>\n<\/ol>\n<p>Silhouette width can be interpreted as follow:<\/p>\n<div class=\"block\">\n<ul>\n<li>Observations with a large <span class=\"math inline\"><em>S<\/em><sub><em>i<\/em><\/sub><\/span> (almost 1) are very well clustered.<\/li>\n<li>A small <span class=\"math inline\"><em>S<\/em><sub><em>i<\/em><\/sub><\/span> (around 0) means that the observation lies between two clusters.<\/li>\n<li>Observations with a negative <span class=\"math inline\"><em>S<\/em><sub><em>i<\/em><\/sub><\/span> are probably placed in the wrong cluster.<\/li>\n<\/ul>\n<\/div>\n<\/div>\n<div id=\"dunn-index\" class=\"section level3\">\n<h3>Dunn index<\/h3>\n<p>The <strong>Dunn index<\/strong> is another internal clustering validation measure which can be computed as follow:<\/p>\n<ol style=\"list-style-type: decimal;\">\n<li>For each cluster, compute the distance between each of the objects in the cluster and the objects in the other clusters<\/li>\n<li>Use the minimum of this pairwise distance as the inter-cluster separation (<em>min.separation<\/em>)<\/li>\n<li>For each cluster, compute the distance between the objects in the same cluster.<\/li>\n<li>Use the maximal intra-cluster distance (i.e maximum diameter) as the intra-cluster compactness<\/li>\n<li>Calculate the <em>Dunn index<\/em> (D) as follow:<\/li>\n<\/ol>\n<p><span class=\"math display\">\\[<br \/>\nD = \\frac{min.separation}{max.diameter}<br \/>\n\\]<\/span><\/p>\n<div class=\"success\">\n<p>If the data set contains compact and well-separated clusters, the diameter of the clusters is expected to be small and the distance between the clusters is expected to be large. Thus, Dunn index should be maximized.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div id=\"external-measures-for-clustering-validation\" class=\"section level2\">\n<h2>External measures for clustering validation<\/h2>\n<p>The aim is to compare the identified clusters (by k-means, pam or hierarchical clustering) to an external reference.<\/p>\n<p>It\u2019s possible to quantify the agreement between partitioning clusters and external reference using either the corrected <em>Rand index<\/em> and <em>Meila\u2019s variation index VI<\/em>, which are implemented in the R function <em>cluster.stats<\/em>()[<em>fpc<\/em> package].<\/p>\n<p>The corrected <em>Rand index<\/em> varies from -1 (no agreement) to 1 (perfect agreement).<\/p>\n<div class=\"success\">\n<p>External clustering validation, can be used to select suitable clustering algorithm for a given data set.<\/p>\n<\/div>\n<\/div>\n<div id=\"computing-cluster-validation-statistics-in-r\" class=\"section level2\">\n<h2>Computing cluster validation statistics in R<\/h2>\n<div id=\"required-r-packages\" class=\"section level3\">\n<h3>Required R packages<\/h3>\n<p>The following R packages are required in this chapter:<\/p>\n<ul>\n<li><em>factoextra<\/em> for data visualization<\/li>\n<li><em>fpc<\/em> for computing clustering validation statistics<\/li>\n<li><em>NbClust<\/em> for determining the optimal number of clusters in the data set.<\/li>\n<li>Install the packages:<\/li>\n<\/ul>\n<pre class=\"r\"><code>install.packages(c(\"factoextra\", \"fpc\", \"NbClust\"))<\/code><\/pre>\n<ul>\n<li>Load the packages:<\/li>\n<\/ul>\n<pre class=\"r\"><code>library(factoextra)\r\nlibrary(fpc)\r\nlibrary(NbClust)<\/code><\/pre>\n<\/div>\n<div id=\"data-preparation\" class=\"section level3\">\n<h3>Data preparation<\/h3>\n<p>We\u2019ll use the built-in R data set iris:<\/p>\n<pre class=\"r\"><code># Excluding the column \"Species\" at position 5\r\ndf &lt;- iris[, -5]\r\n# Standardize\r\ndf &lt;- scale(df)<\/code><\/pre>\n<\/div>\n<div id=\"clustering-analysis\" class=\"section level3\">\n<h3>Clustering analysis<\/h3>\n<p>We\u2019ll use the function <em>eclust<\/em>() [enhanced clustering, in <em>factoextra<\/em>] which provides several advantages:<\/p>\n<ul>\n<li>It simplifies the workflow of clustering analysis<\/li>\n<li>It can be used to compute hierarchical clustering and partitioning clustering in a single line function call<\/li>\n<li>Compared to the standard partitioning functions (kmeans, pam, clara and fanny) which requires the user to specify the optimal number of clusters, the function <em>eclust<\/em>() computes automatically the gap statistic for estimating the right number of clusters.<\/li>\n<li>It provides silhouette information for all partitioning methods and hierarchical clustering<\/li>\n<li>It draws beautiful graphs using ggplot2<\/li>\n<\/ul>\n<p>The simplified format the <em>eclust<\/em>() function is as follow:<\/p>\n<pre class=\"r\"><code>eclust(x, FUNcluster = \"kmeans\", hc_metric = \"euclidean\", ...)<\/code><\/pre>\n<div class=\"block\">\n<ul>\n<li><strong>x<\/strong>: numeric vector, data matrix or data frame<\/li>\n<li><strong>FUNcluster<\/strong>: a clustering function including \u201ckmeans\u201d, \u201cpam\u201d, \u201cclara\u201d, \u201cfanny\u201d, \u201chclust\u201d, \u201cagnes\u201d and \u201cdiana\u201d. Abbreviation is allowed.<\/li>\n<li><strong>hc_metric<\/strong>: character string specifying the metric to be used for calculating dissimilarities between observations. Allowed values are those accepted by the function <em>dist<\/em>() [including \u201ceuclidean\u201d, \u201cmanhattan\u201d, \u201cmaximum\u201d, \u201ccanberra\u201d, \u201cbinary\u201d, \u201cminkowski\u201d] and correlation based distance measures [\u201cpearson\u201d, \u201cspearman\u201d or \u201ckendall\u201d]. Used only when FUNcluster is a hierarchical clustering function such as one of \u201chclust\u201d, \u201cagnes\u201d or \u201cdiana\u201d.<\/li>\n<li><strong>\u2026<\/strong>: other arguments to be passed to FUNcluster.<\/li>\n<\/ul>\n<\/div>\n<p>The function <strong>eclust()<\/strong> returns an object of class <strong>eclust<\/strong> containing the result of the standard function used (e.g., kmeans, pam, hclust, agnes, diana, etc.).<\/p>\n<p>It includes also:<\/p>\n<ul>\n<li><strong>cluster<\/strong>: the cluster assignment of observations after cutting the tree<\/li>\n<li><strong>nbclust<\/strong>: the number of clusters<\/li>\n<li><strong>silinfo<\/strong>: the silhouette information of observations<\/li>\n<li><strong>size<\/strong>: the size of clusters<\/li>\n<li><strong>data<\/strong>: a matrix containing the original or the standardized data (if stand = TRUE)<\/li>\n<li><strong>gap_stat<\/strong>: containing gap statistics<\/li>\n<\/ul>\n<p>To compute a partitioning clustering, such as k-means clustering with k = 3, type this:<\/p>\n<pre class=\"r\"><code># K-means clustering\r\nkm.res &lt;- eclust(df, \"kmeans\", k = 3, nstart = 25, graph = FALSE)\r\n# Visualize k-means clusters\r\nfviz_cluster(km.res, geom = \"point\", ellipse.type = \"norm\",\r\n             palette = \"jco\", ggtheme = theme_minimal())<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/004-cluster-validation\/figures\/016-cluster-validation-statistics-k-means-clustering-1.png\" width=\"384\" \/><\/p>\n<p>To compute a hierarchical clustering, use this:<\/p>\n<pre class=\"r\"><code># Hierarchical clustering\r\nhc.res &lt;- eclust(df, \"hclust\", k = 3, hc_metric = \"euclidean\", \r\n                 hc_method = \"ward.D2\", graph = FALSE)\r\n\r\n# Visualize dendrograms\r\nfviz_dend(hc.res, show_labels = FALSE,\r\n         palette = \"jco\", as.ggplot = TRUE)<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/004-cluster-validation\/figures\/016-cluster-validation-statistics-hierarchical-clustering-1.png\" width=\"518.4\" \/><\/p>\n<\/div>\n<div id=\"clustering-validation\" class=\"section level3\">\n<h3>Clustering validation<\/h3>\n<div id=\"silhouette-plot\" class=\"section level4\">\n<h4>Silhouette plot<\/h4>\n<p>Recall that the silhouette coefficient (<span class=\"math inline\">\\(S_i\\)<\/span>) measures how similar an object <span class=\"math inline\">\\(i\\)<\/span> is to the the other objects in its own cluster versus those in the neighbor cluster. <span class=\"math inline\">\\(S_i\\)<\/span> values range from 1 to - 1:<\/p>\n<ul>\n<li>A value of <span class=\"math inline\">\\(S_i\\)<\/span> close to 1 indicates that the object is well clustered. In the other words, the object <span class=\"math inline\">\\(i\\)<\/span> is similar to the other objects in its group.<\/li>\n<li>A value of <span class=\"math inline\">\\(S_i\\)<\/span> close to -1 indicates that the object is poorly clustered, and that assignment to some other cluster would probably improve the overall results.<\/li>\n<\/ul>\n<p>It\u2019s possible to draw silhouette coefficients of observations using the function <em>fviz_silhouette<\/em>() [<em>factoextra<\/em> package], which will also print a summary of the silhouette analysis output. To avoid this, you can use the option <em>print.summary = FALSE<\/em>.<\/p>\n<\/p>\n<div class=\"error\">Here, there are contents hidden to non-premium members. Sign up now to read all of our premium contents and to be awarded a certificate of course completion.<br \/>\n<a href='https:\/\/www.datanovia.com\/en\/pricing\/' target='_self'  class='dt-sc-button   medium  '  style=\"background-color:#FF6600;border-color:#FF6600;color:#ffffff;\">Claim Your Membership Now<\/a><\/div>\n<p>\n<pre><code>##   cluster size ave.sil.width\r\n## 1       1   50          0.64\r\n## 2       2   47          0.35\r\n## 3       3   53          0.39<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/004-cluster-validation\/figures\/016-cluster-validation-statistics-silhouette-coefficient-1.png\" width=\"518.4\" \/><\/p>\n<p>Silhouette information can be extracted as follow:<\/p>\n<pre class=\"r\"><code># Silhouette information\r\nsilinfo &lt;- km.res$silinfo\r\nnames(silinfo)\r\n# Silhouette widths of each observation\r\nhead(silinfo$widths[, 1:3], 10)\r\n# Average silhouette width of each cluster\r\nsilinfo$clus.avg.widths\r\n# The total average (mean of all individual silhouette widths)\r\nsilinfo$avg.width\r\n# The size of each clusters\r\nkm.res$size<\/code><\/pre>\n<p>It can be seen that several samples, in cluster 2, have a negative silhouette coefficient. This means that they are not in the right cluster. We can find the name of these samples and determine the clusters they are closer (neighbor cluster), as follow:<\/p>\n<pre class=\"r\"><code># Silhouette width of observation\r\nsil &lt;- km.res$silinfo$widths[, 1:3]\r\n# Objects with negative silhouette\r\nneg_sil_index &lt;- which(sil[, 'sil_width'] &lt; 0)\r\nsil[neg_sil_index, , drop = FALSE]<\/code><\/pre>\n<pre><code>##     cluster neighbor sil_width\r\n## 112       2        3   -0.0106\r\n## 128       2        3   -0.0249<\/code><\/pre>\n<\/div>\n<div id=\"validation-statistics\" class=\"section level4\">\n<h4>Validation statistics<\/h4>\n<p>The function <em>cluster.stats<\/em>() [<em>fpc<\/em> package] and the function <em>NbClust()<\/em> [in <em>NbClust<\/em> package] can be used to compute <em>Dunn index<\/em> and many other cluster validation statistics or indices.<\/p>\n<p>The simplified format is:<\/p>\n<pre class=\"r\"><code>cluster.stats(d = NULL, clustering, al.clustering = NULL)<\/code><\/pre>\n<div class=\"block\">\n<ul>\n<li><strong>d<\/strong>: a distance object between cases as generated by the <strong>dist()<\/strong> function<\/li>\n<li><strong>clustering<\/strong>: vector containing the cluster number of each observation<\/li>\n<li><strong>alt.clustering<\/strong>: vector such as for clustering, indicating an alternative clustering<\/li>\n<\/ul>\n<\/div>\n<p>The function <em>cluster.stats<\/em>() returns a list containing many components useful for analyzing the intrinsic characteristics of a clustering:<\/p>\n<ul>\n<li><strong>cluster.number<\/strong>: number of clusters<\/li>\n<li><strong>cluster.size<\/strong>: vector containing the number of points in each cluster<\/li>\n<li><strong>average.distance<\/strong>, <strong>median.distance<\/strong>: vector containing the cluster-wise within average\/median distances<\/li>\n<li><strong>average.between<\/strong>: average distance between clusters. We want it to be as large as possible<\/li>\n<li><strong>average.within<\/strong>: average distance within clusters. We want it to be as small as possible<\/li>\n<li><strong>clus.avg.silwidths<\/strong>: vector of cluster average silhouette widths. Recall that, the <strong>silhouette width<\/strong> is also an estimate of the average distance between clusters. Its value is comprised between 1 and -1 with a value of 1 indicating a very good cluster.<\/li>\n<li><strong>within.cluster.ss<\/strong>: a generalization of the within clusters sum of squares (k-means objective function), which is obtained if d is a Euclidean distance matrix.<\/li>\n<li><strong>dunn, dunn2<\/strong>: Dunn index<\/li>\n<li><strong>corrected.rand, vi<\/strong>: Two indexes to assess the similarity of two clustering: the corrected Rand index and Meila\u2019s VI<\/li>\n<\/ul>\n<p>All the above elements can be used to evaluate the internal quality of clustering.<\/p>\n<p>In the following sections, we\u2019ll compute the clustering quality statistics for k-means. Look at the <strong>within.cluster.ss<\/strong> (within clusters sum of squares), the <strong>average.within<\/strong> (average distance within clusters) and <strong>clus.avg.silwidths<\/strong> (vector of cluster average silhouette widths).<\/p>\n<pre class=\"r\"><code>library(fpc)\r\n# Statistics for k-means clustering\r\nkm_stats &lt;- cluster.stats(dist(df),  km.res$cluster)\r\n# Dun index\r\nkm_stats$dunn<\/code><\/pre>\n<pre><code>## [1] 0.0265<\/code><\/pre>\n<p>To display all statistics, type this:<\/p>\n<pre class=\"r\"><code>km_stats<\/code><\/pre>\n<div class=\"notice\">\n<p>Read the documentation of <em>cluster.stats<\/em>() for details about all the available indices.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div id=\"external-clustering-validation\" class=\"section level3\">\n<h3>External clustering validation<\/h3>\n<p>Among the values returned by the function <strong>cluster.stats<\/strong>(), there are two indexes to assess the similarity of two clustering, namely the corrected Rand index and Meila\u2019s VI.<\/p>\n<p>We know that the iris data contains exactly 3 groups of species.<\/p>\n<p><span class=\"question\">Does the K-means clustering matches with the true structure of the data?<\/span><\/p>\n<p>We can use the function <strong>cluster.stats<\/strong>() to answer to this question.<\/p>\n<p>Let start by computing a cross-tabulation between k-means clusters and the reference Species labels:<\/p>\n<pre class=\"r\"><code>table(iris$Species, km.res$cluster)<\/code><\/pre>\n<pre><code>##             \r\n##               1  2  3\r\n##   setosa     50  0  0\r\n##   versicolor  0 11 39\r\n##   virginica   0 36 14<\/code><\/pre>\n<p>It can be seen that:<\/p>\n<ul>\n<li>All setosa species (n = 50) has been classified in cluster 1<\/li>\n<li>A large number of versicor species (n = 39 ) has been classified in cluster 3. Some of them ( n = 11) have been classified in cluster 2.<\/li>\n<li>A large number of virginica species (n = 36 ) has been classified in cluster 2. Some of them (n = 14) have been classified in cluster 3.<\/li>\n<\/ul>\n<p>It\u2019s possible to quantify the agreement between Species and k-means clusters using either the corrected Rand index and Meila\u2019s VI provided as follow:<\/p>\n<pre class=\"r\"><code>library(\"fpc\")\r\n# Compute cluster stats\r\nspecies &lt;- as.numeric(iris$Species)\r\nclust_stats &lt;- cluster.stats(d = dist(df), \r\n                             species, km.res$cluster)\r\n# Corrected Rand index\r\nclust_stats$corrected.rand<\/code><\/pre>\n<pre><code>## [1] 0.62<\/code><\/pre>\n<pre class=\"r\"><code># VI\r\nclust_stats$vi<\/code><\/pre>\n<pre><code>## [1] 0.748<\/code><\/pre>\n<div class=\"success\">\n<p>The corrected <strong>Rand index<\/strong> provides a measure for assessing the similarity between two partitions, adjusted for chance. Its range is -1 (no agreement) to 1 (perfect agreement). Agreement between the specie types and the cluster solution is 0.62 using <strong>Rand index<\/strong> and 0.748 using Meila\u2019s VI.<\/p>\n<\/div>\n<p>The same analysis can be computed for both PAM and hierarchical clustering:<\/p>\n<pre class=\"r\"><code># Agreement between species and pam clusters\r\npam.res &lt;- eclust(df, \"pam\", k = 3, graph = FALSE)\r\ntable(iris$Species, pam.res$cluster)\r\ncluster.stats(d = dist(iris.scaled), \r\n              species, pam.res$cluster)$vi\r\n# Agreement between species and HC clusters\r\nres.hc &lt;- eclust(df, \"hclust\", k = 3, graph = FALSE)\r\ntable(iris$Species, res.hc$cluster)\r\ncluster.stats(d = dist(iris.scaled), \r\n              species, res.hc$cluster)$vi<\/code><\/pre>\n<div class=\"success\">\n<p>External clustering validation, can be used to select suitable clustering algorithm for a given data set.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div id=\"summary\" class=\"section level2\">\n<h2>Summary<\/h2>\n<p>We described how to validate clustering results using the silhouette method and the Dunn index. This task is facilitated using the combination of two R functions: <em>eclust<\/em>() and <em>fviz_silhouette<\/em> in the factoextra package We also demonstrated how to assess the agreement between a clustering result and an external reference.<br \/>\nIn the next chapters, we\u2019ll show how to i) choose the appropriate clustering algorithm for your data; and ii) computing p-values for hierarchical clustering.<\/p>\n<\/div>\n<div id=\"references\" class=\"section level2 unnumbered\">\n<h2>References<\/h2>\n<div id=\"refs\" class=\"references\">\n<div id=\"ref-brock2008\">\n<p>Brock, Guy, Vasyl Pihur, Susmita Datta, and Somnath Datta. 2008. \u201cClValid: An R Package for Cluster Validation.\u201d <em>Journal of Statistical Software<\/em> 25 (4): 1\u201322. <a class=\"uri\" href=\"https:\/\/www.jstatsoft.org\/v025\/i04\">https:\/\/www.jstatsoft.org\/v025\/i04<\/a>.<\/p>\n<\/div>\n<div id=\"ref-charrad2014\">\n<p>Charrad, Malika, Nadia Ghazzali, V\u00e9ronique Boiteau, and Azam Niknafs. 2014. \u201cNbClust: An R Package for Determining the Relevant Number of Clusters in a Data Set.\u201d <em>Journal of Statistical Software<\/em> 61: 1\u201336. <a class=\"uri\" href=\"http:\/\/www.jstatsoft.org\/v61\/i06\/paper\">http:\/\/www.jstatsoft.org\/v61\/i06\/paper<\/a>.<\/p>\n<\/div>\n<div id=\"ref-theodoridis2008\">\n<p>Theodoridis, Sergios, and Konstantinos Koutroumbas. 2008. <em>Pattern Recognition<\/em>. 2nd ed. Academic Press.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<p><!--end rdoc--><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this article, we start by describing the different methods for clustering validation. Next, we&#8217;ll demonstrate how to compare the quality of clustering results obtained with different clustering algorithms. Finally, we&#8217;ll provide R scripts for validating clustering results.<\/p>\n","protected":false},"author":1,"featured_media":7994,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","class_list":["post-8063","dt_lessons","type-dt_lessons","status-publish","has-post-thumbnail","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Cluster Validation Statistics: Must Know Methods - Datanovia<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Cluster Validation Statistics: Must Know Methods - Datanovia\" \/>\n<meta property=\"og:description\" content=\"In this article, we start by describing the different methods for clustering validation. Next, we&#039;ll demonstrate how to compare the quality of clustering results obtained with different clustering algorithms. Finally, we&#039;ll provide R scripts for validating clustering results.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/\" \/>\n<meta property=\"og:site_name\" content=\"Datanovia\" \/>\n<meta property=\"article:modified_time\" content=\"2018-10-22T17:04:03+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/IMG_6422.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"512\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"13 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/\",\"url\":\"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/\",\"name\":\"Cluster Validation Statistics: Must Know Methods - Datanovia\",\"isPartOf\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/IMG_6422.jpg\",\"datePublished\":\"2018-10-21T09:55:57+00:00\",\"dateModified\":\"2018-10-22T17:04:03+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/#primaryimage\",\"url\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/IMG_6422.jpg\",\"contentUrl\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/IMG_6422.jpg\",\"width\":1024,\"height\":512},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.datanovia.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Lessons\",\"item\":\"https:\/\/www.datanovia.com\/en\/lessons\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Cluster Validation Statistics: Must Know Methods\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.datanovia.com\/en\/#website\",\"url\":\"https:\/\/www.datanovia.com\/en\/\",\"name\":\"Datanovia\",\"description\":\"Data Mining and Statistics for Decision Support\",\"publisher\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.datanovia.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.datanovia.com\/en\/#organization\",\"name\":\"Datanovia\",\"url\":\"https:\/\/www.datanovia.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png\",\"contentUrl\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png\",\"width\":98,\"height\":99,\"caption\":\"Datanovia\"},\"image\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Cluster Validation Statistics: Must Know Methods - Datanovia","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/","og_locale":"en_US","og_type":"article","og_title":"Cluster Validation Statistics: Must Know Methods - Datanovia","og_description":"In this article, we start by describing the different methods for clustering validation. Next, we'll demonstrate how to compare the quality of clustering results obtained with different clustering algorithms. Finally, we'll provide R scripts for validating clustering results.","og_url":"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/","og_site_name":"Datanovia","article_modified_time":"2018-10-22T17:04:03+00:00","og_image":[{"width":1024,"height":512,"url":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/IMG_6422.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"13 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/","url":"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/","name":"Cluster Validation Statistics: Must Know Methods - Datanovia","isPartOf":{"@id":"https:\/\/www.datanovia.com\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/#primaryimage"},"image":{"@id":"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/#primaryimage"},"thumbnailUrl":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/IMG_6422.jpg","datePublished":"2018-10-21T09:55:57+00:00","dateModified":"2018-10-22T17:04:03+00:00","breadcrumb":{"@id":"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/#primaryimage","url":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/IMG_6422.jpg","contentUrl":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/IMG_6422.jpg","width":1024,"height":512},{"@type":"BreadcrumbList","@id":"https:\/\/www.datanovia.com\/en\/lessons\/cluster-validation-statistics-must-know-methods\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.datanovia.com\/en\/"},{"@type":"ListItem","position":2,"name":"Lessons","item":"https:\/\/www.datanovia.com\/en\/lessons\/"},{"@type":"ListItem","position":3,"name":"Cluster Validation Statistics: Must Know Methods"}]},{"@type":"WebSite","@id":"https:\/\/www.datanovia.com\/en\/#website","url":"https:\/\/www.datanovia.com\/en\/","name":"Datanovia","description":"Data Mining and Statistics for Decision Support","publisher":{"@id":"https:\/\/www.datanovia.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.datanovia.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.datanovia.com\/en\/#organization","name":"Datanovia","url":"https:\/\/www.datanovia.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png","contentUrl":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png","width":98,"height":99,"caption":"Datanovia"},"image":{"@id":"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/"}}]}},"multi-rating":{"mr_rating_results":[]},"_links":{"self":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/dt_lessons\/8063","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/dt_lessons"}],"about":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/types\/dt_lessons"}],"author":[{"embeddable":true,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/comments?post=8063"}],"version-history":[{"count":0,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/dt_lessons\/8063\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/media\/7994"}],"wp:attachment":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/media?parent=8063"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}