The classification of observations into groups requires some methods for computing the distance<\/strong> or the (dis)similarity<\/strong> between each pair of observations. The result of this computation is known as a dissimilarity or distance matrix<\/strong>.<\/p>\n There are many methods to calculate this distance information. In this article, we describe the common distance measures<\/strong> and provide R codes for computing and visualizing distances.<\/p>\n<\/div>\n Contents:<\/p>\n The choice of distance measures is a critical step in clustering. It defines how the similarity of two elements (x, y) is calculated and it will influence the shape of the clusters.<\/p>\n The classical methods for distance measures are Euclidean<\/em> and Manhattan distances<\/em>, which are defined as follow:<\/p>\n \\[ \\[ Where, x<\/em> and y<\/em> are two vectors of length n<\/em>.<\/p>\n Other dissimilarity measures exist such as correlation-based distances<\/strong>, which is widely used for gene expression data analyses. Correlation-based distance is defined by subtracting the correlation coefficient from 1. Different types of correlation methods can be used such as:<\/p>\n \\[ Pearson correlation measures the degree of a linear relationship between two profiles.<\/p>\n<\/div>\n It\u2019s a special case of Pearson\u2019s correlation with \\(\\bar{x}\\)<\/span> and \\(\\bar{y}\\)<\/span> both replaced by zero:<\/p>\n \\[ The spearman correlation method computes the correlation between the rank of x and the rank of y variables.<\/p>\n \\[ Where \\(x'_i = rank(x_i)\\)<\/span> and \\(y'_i = rank(y)\\)<\/span>.<\/p>\n Kendall correlation method measures the correspondence between the ranking of x and y variables. The total number of possible pairings of x with y observations is \\(n(n-1)\/2\\)<\/span>, where n is the size of x and y. Begin by ordering the pairs by the x values. If x and y are correlated, then they would have the same relative rank orders. Now, for each \\(y_i\\)<\/span>, count the number of \\(y_j > y_i\\)<\/span> (concordant pairs (c)) and the number of \\(y_j < y_i\\)<\/span> (discordant pairs (d)).<\/p>\n Kendall correlation distance is defined as follow:<\/p>\n \\[ Where,<\/p>\n Note that,<\/p>\n In the formula above, x<\/em><\/span> and y<\/em><\/span> are two vectors of length n<\/em><\/span> and, means \\(\\bar{x}\\)<\/span><\/span> and \\(\\bar{y}\\)<\/span><\/span>, respectively. The distance between x and y is denoted d<\/em>(x<\/em>,\u2006y<\/em>)<\/span>.<\/p>\n<\/div>\n<\/div>\n The choice of distance measures is very important, as it has a strong influence on the clustering results. For most common clustering software, the default distance measure is the Euclidean distance.<\/p>\n Depending on the type of the data and the researcher questions, other dissimilarity measures might be preferred. For example, correlation-based distance is often used in gene expression data analysis.<\/p>\n Correlation-based distance considers two objects to be similar if their features are highly correlated, even though the observed values may be far apart in terms of Euclidean distance. The distance between two objects is 0 when they are perfectly correlated. Pearson\u2019s correlation is quite sensitive to outliers. This does not matter when clustering samples, because the correlation is over thousands of genes. When clustering genes, it is important to be aware of the possible impact of outliers. This can be mitigated by using Spearman\u2019s correlation instead of Pearson\u2019s correlation.<\/p>\n If we want to identify clusters of observations with the same overall profiles regardless of their magnitudes, then we should go with correlation-based distance<\/em> as a dissimilarity measure. This is particularly the case in gene expression data analysis, where we might want to consider genes similar when they are \u201cup\u201d and \u201cdown\u201d together. It is also the case, in marketing if we want to identify group of shoppers with the same preference in term of items, regardless of the volume of items they bought.<\/p>\n If Euclidean distance is chosen, then observations with high values of features will be clustered together. The same holds true for observations with low values of features.<\/p>\n<\/div>\n The value of distance measures is intimately related to the scale on which measurements are made. Therefore, variables are often scaled (i.e.\u00a0standardized) before measuring the inter-observation dissimilarities. This is particularly recommended when variables are measured in different scales (e.g: kilograms, kilometers, centimeters, \u2026); otherwise, the dissimilarity measures obtained will be severely affected.<\/p>\n The goal is to make the variables comparable. Generally variables are scaled to have i) standard deviation one and ii) mean zero.<\/p>\n The standardization of data is an approach widely used in the context of gene expression data analysis before clustering. We might also want to scale the data when the mean and\/or the standard deviation of variables are largely different.<\/p>\n When scaling variables, the data can be transformed as follow:<\/p>\n \\[ Where \\(center(x)\\)<\/span> can be the mean or the median of x values, and \\(scale(x)\\)<\/span> can be the standard deviation (SD), the interquartile range, or the MAD (median absolute deviation).<\/p>\n The R base function scale<\/em>() can be used to standardize the data. It takes a numeric matrix as an input and performs the scaling on the columns.<\/p>\n Standardization makes the four distance measure methods - Euclidean, Manhattan, Correlation and Eisen - more similar than they would be with non-transformed data.<\/p>\n Note that, when the data are standardized, there is a functional relationship between the Pearson correlation coefficient r<\/em>(x<\/em>,\u2006y<\/em>)<\/span> and the Euclidean distance.<\/p>\n With some maths, the relationship can be defined as follow:<\/p>\n \\[ Where x and y are two standardized m-vectors with zero mean and unit length.<\/p>\n Therefore, the result obtained with Pearson correlation measures and standardized Euclidean distances are comparable.<\/p>\n<\/div>\n<\/div>\n We\u2019ll use the USArrests data as demo data sets. We\u2019ll use only a subset of the data by taking 15 random rows among the 50 rows in the data set. This is done by using the function sample<\/em>(). Next, we standardize the data using the function scale<\/em>():<\/p>\n There are many R functions for computing distances between pairs of observations:<\/p>\n All these functions compute distances between rows of the data.<\/p>\n<\/div>\n<\/div>\n To compute Euclidean distance, you can use the R base dist<\/em>() function, as follow:<\/p>\n Note that, allowed values for the option method include one of: \u201ceuclidean\u201d, \u201cmaximum\u201d, \u201cmanhattan\u201d, \u201ccanberra\u201d, \u201cbinary\u201d, \u201cminkowski\u201d.<\/p>\n To make it easier to see the distance information generated by the dist<\/em>() function, you can reformat the distance vector into a matrix using the as.matrix<\/em>() function.<\/p>\n In this matrix, the value represent the distance between objects. The values on the diagonal of the matrix represent the distance between objects and themselves (which are zero).<\/p>\n In this data set, the columns are variables. Hence, if we want to compute pairwise distances between variables, we must start by transposing the data to have variables in the rows of the data set before using the dist<\/em>() function. The function t<\/em>() is used for transposing the data.<\/p>\n<\/div>\n<\/div>\n Correlation-based distances are commonly used in gene expression data analysis.<\/p>\n The function get_dist<\/em>()[factoextra<\/em> package] can be used to compute correlation-based distances. Correlation method can be either pearson<\/em>, spearman<\/em> or kendall<\/em>.<\/p>\n The function daisy<\/em>() [cluster<\/em> package] provides a solution (Gower\u2019s metric<\/em>) for computing the distance matrix, in the situation where the data contain no-numeric columns.<\/p>\n The R code below applies the daisy<\/em>() function on flower<\/em> data which contains factor<\/em>, ordered<\/em> and numeric<\/em> variables:<\/p>\n A simple solution for visualizing the distance matrices is to use the function fviz_dist<\/em>() [factoextra<\/em> package]. Other specialized methods, such as agglomerative hierarchical clustering or heatmap will be comprehensively described in the dedicated courses.<\/p>\n To use fviz_dist<\/em>() type this:<\/p>\n The color level is proportional to the value of the dissimilarity between observations: pure red if \\(dist(x_i, x_j) = 0\\)<\/span> and pure blue corresponds to the highest value of euclidean distance computed. Objects belonging to the same cluster are displayed in consecutive order.<\/p>\n<\/div>\n We described how to compute distance matrices using either Euclidean or correlation-based measures. It\u2019s generally recommended to standardize the variables before distance matrix computation. Standardization makes variable comparable, in the situation where they are measured in different scales.<\/p>\n<\/div>\n<\/div>\n <\/p>\n","protected":false},"excerpt":{"rendered":" In this article, we describe the common distance measures used to compute distance matrix for cluster analysis. We also provide R codes for computing and visualizing distances. <\/p>\n","protected":false},"author":1,"featured_media":8019,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","class_list":["post-7645","dt_lessons","type-dt_lessons","status-publish","has-post-thumbnail","hentry"],"yoast_head":"\n\n
\n
Related Book <\/a><\/h4>Practical Guide to Cluster Analysis in R<\/div>\n
Methods for measuring distances<\/h2>\n
\n
\nd_{euc}(x,y) = \\sqrt{\\sum_{i=1}^n(x_i - y_i)^2}
\n\\]<\/span><\/p>\n\n
\nd_{man}(x,y) = \\sum_{i=1}^n |{(x_i - y_i)|}
\n\\]<\/span><\/p>\n\n
\nd_{cor}(x, y) = 1 - \\frac{\\sum\\limits_{i=1}^n (x_i - \\bar{x})(y_i - \\bar{y})}{\\sqrt{\\sum\\limits_{i=1}^n(x_i - \\bar{x})^2 \\sum\\limits_{i=1}^n(y_i -\\bar{y})^2}}
\n\\]<\/span><\/p>\n\n
\nd_{eisen}(x, y) = 1 - \\frac{\\left|\\sum\\limits_{i=1}^n x_iy_i\\right|}{\\sqrt{\\sum\\limits_{i=1}^n x^2_i \\sum\\limits_{i=1}^n y^2_i}}
\n\\]<\/span><\/p>\n\n
\nd_{spear}(x, y) = 1 - \\frac{\\sum\\limits_{i=1}^n (x'_i - \\bar{x'})(y'_i - \\bar{y'})}{\\sqrt{\\sum\\limits_{i=1}^n(x'_i - \\bar{x'})^2 \\sum\\limits_{i=1}^n(y'_i -\\bar{y'})^2}}
\n\\]<\/span><\/p>\n\n
\nd_{kend}(x, y) = 1 - \\frac{n_c - n_d}{\\frac{1}{2}n(n-1)}
\n\\]<\/span><\/p>\n\n
\n
What type of distance measures should we choose?<\/h2>\n
Data standardization<\/h2>\n
\n\\frac{x_i - center(x)}{scale(x)}
\n\\]<\/span><\/p>\n
\nd_{euc}(x, y) = \\sqrt{2m[1 - r(x, y)]}
\n\\]<\/span><\/span><\/p>\nDistance matrix computation<\/h2>\n
Data preparation<\/h3>\n
# Subset of the data\r\nset.seed(123)\r\nss <- sample(1:50, 15) # Take 15 random rows\r\ndf <- USArrests[ss, ] # Subset the 15 rows\r\ndf.scaled <- scale(df) # Standardize the variables<\/code><\/pre>\n<\/div>\nR functions and packages<\/h3>\n
\n
Computing euclidean distance<\/h3>\n
dist.eucl <- dist(df.scaled, method = \"euclidean\")<\/code><\/pre>\n# Reformat as a matrix\r\n# Subset the first 3 columns and rows and Round the values\r\nround(as.matrix(dist.eucl)[1:3, 1:3], 1)<\/code><\/pre>\n## Iowa Rhode Island Maryland\r\n## Iowa 0.0 2.8 4.1\r\n## Rhode Island 2.8 0.0 3.6\r\n## Maryland 4.1 3.6 0.0<\/code><\/pre>\nComputing correlation based distances<\/h3>\n
# Compute\r\nlibrary(\"factoextra\")\r\ndist.cor <- get_dist(df.scaled, method = \"pearson\")\r\n\r\n# Display a subset\r\nround(as.matrix(dist.cor)[1:3, 1:3], 1)<\/code><\/pre>\n## Iowa Rhode Island Maryland\r\n## Iowa 0.0 0.4 1.9\r\n## Rhode Island 0.4 0.0 1.5\r\n## Maryland 1.9 1.5 0.0<\/code><\/pre>\n<\/div>\nComputing distances for mixed data<\/h3>\n
library(cluster)\r\n# Load data\r\ndata(flower)\r\nhead(flower, 3)<\/code><\/pre>\n## V1 V2 V3 V4 V5 V6 V7 V8\r\n## 1 0 1 1 4 3 15 25 15\r\n## 2 1 0 0 2 1 3 150 50\r\n## 3 0 1 0 3 3 1 150 50<\/code><\/pre>\n# Data structure\r\nstr(flower)<\/code><\/pre>\n## 'data.frame': 18 obs. of 8 variables:\r\n## $ V1: Factor w\/ 2 levels \"0\",\"1\": 1 2 1 1 1 1 1 1 2 2 ...\r\n## $ V2: Factor w\/ 2 levels \"0\",\"1\": 2 1 2 1 2 2 1 1 2 2 ...\r\n## $ V3: Factor w\/ 2 levels \"0\",\"1\": 2 1 1 2 1 1 1 2 1 1 ...\r\n## $ V4: Factor w\/ 5 levels \"1\",\"2\",\"3\",\"4\",..: 4 2 3 4 5 4 4 2 3 5 ...\r\n## $ V5: Ord.factor w\/ 3 levels \"1\"<\"2\"<\"3\": 3 1 3 2 2 3 3 2 1 2 ...\r\n## $ V6: Ord.factor w\/ 18 levels \"1\"<\"2\"<\"3\"<\"4\"<..: 15 3 1 16 2 12 13 7 4 14 ...\r\n## $ V7: num 25 150 150 125 20 50 40 100 25 100 ...\r\n## $ V8: num 15 50 50 50 15 40 20 15 15 60 ...<\/code><\/pre>\n# Distance matrix\r\ndd <- daisy(flower)\r\nround(as.matrix(dd)[1:3, 1:3], 2)<\/code><\/pre>\n## 1 2 3\r\n## 1 0.00 0.89 0.53\r\n## 2 0.89 0.00 0.51\r\n## 3 0.53 0.51 0.00<\/code><\/pre>\n<\/div>\n<\/div>\nVisualizing distance matrices<\/h2>\n
library(factoextra)\r\nfviz_dist(dist.eucl)<\/code><\/pre>\n![](\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/001-data-clustering-basics\/figures\/005-clustering-distance-measures-visualize-distance-measures-1.png\")
<\/p>\n\n
Summary<\/h2>\n