{"id":8086,"date":"2018-10-25T20:31:30","date_gmt":"2018-10-25T18:31:30","guid":{"rendered":"https:\/\/www.datanovia.com\/en\/?post_type=dt_lessons&#038;p=8086"},"modified":"2018-10-25T23:44:54","modified_gmt":"2018-10-25T21:44:54","slug":"fuzzy-c-means-clustering-algorithm","status":"publish","type":"dt_lessons","link":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/","title":{"rendered":"Fuzzy C-Means Clustering Algorithm"},"content":{"rendered":"<div id=\"rdoc\">\n<p>In our previous article, we described the basic concept of <strong>fuzzy clustering<\/strong> and we showed how to compute fuzzy clustering. In this current article, we\u2019ll present the <strong>fuzzy c-means clustering algorithm<\/strong>, which is very similar to the <a href=\"https:\/\/www.datanovia.com\/en\/lessons\/k-means-clustering-in-r-algorith-and-practical-examples\/\">k-means algorithm<\/a> and the aim is to minimize the objective function defined as follow:<\/p>\n<p><span class=\"math\"><span class=\"math display\">\\[<br \/>\n\\sum\\limits_{j=1}^k \\sum\\limits_{x_i \\in C_j} u_{ij}^m (x_i - \\mu_j)^2<br \/>\n\\]<\/span><\/span><\/p>\n<p>Where,<\/p>\n<ul>\n<li><span class=\"math\"><span class=\"math inline\">\\(u_{ij}\\)<\/span><\/span> is the degree to which an observation <span class=\"math\"><span class=\"math inline\">\\(x_i\\)<\/span><\/span> belongs to a cluster <span class=\"math\"><span class=\"math inline\">\\(c_j\\)<\/span><\/span><\/li>\n<li><span class=\"math\"><span class=\"math inline\">\\(\\mu_j\\)<\/span><\/span> is the center of the cluster j<\/li>\n<li><span class=\"math\"><span class=\"math inline\">\\(u_{ij}\\)<\/span><\/span> is the degree to which an observation <span class=\"math\"><span class=\"math inline\">\\(x_i\\)<\/span><\/span> belongs to a cluster <span class=\"math\"><span class=\"math inline\">\\(c_j\\)<\/span><\/span><\/li>\n<li><span class=\"math\"><span class=\"math inline\">\\(m\\)<\/span><\/span> is the fuzzifier.<\/li>\n<\/ul>\n<p><span class=\"notice\">It can be seen that, FCM differs from k-means by using the membership values <span class=\"math\"><span class=\"math inline\">\\(u_{ij}\\)<\/span><\/span> and the fuzzifier <span class=\"math\"><span class=\"math inline\">\\(m\\)<\/span><\/span>.<\/span><\/p>\n<p>The variable <span class=\"math\"><span class=\"math inline\">\\(u_{ij}^m\\)<\/span><\/span> is defined as follow:<\/p>\n<p><span class=\"math\"><span class=\"math display\">\\[<br \/>\nu_{ij}^m = \\frac{1}{\\sum\\limits_{l=1}^k \\left( \\frac{| x_i - c_j |}{| x_i - c_k |}\\right)^{\\frac{2}{m-1}}}<br \/>\n\\]<\/span><\/span><\/p>\n<p>The degree of belonging, <span class=\"math\"><span class=\"math inline\">\\(u_{ij}\\)<\/span><\/span>, is linked inversely to the distance from x to the cluster center.<\/p>\n<p>The parameter <span class=\"math\"><span class=\"math inline\">\\(m\\)<\/span><\/span> is a real number greater than 1 (<span class=\"math\"><span class=\"math inline\">\\(1.0 &lt; m &lt; \\infty\\)<\/span><\/span>) and it defines the level of cluster fuzziness. Note that, a value of <span class=\"math\"><span class=\"math inline\">\\(m\\)<\/span><\/span> close to 1 gives a cluster solution which becomes increasingly similar to the solution of hard clustering such as k-means; whereas a value of <span class=\"math\"><span class=\"math inline\">\\(m\\)<\/span><\/span> close to infinite leads to complete fuzzyness.<\/p>\n<p><span class=\"success\">Note that, a good choice is to use <strong>m = 2.0<\/strong> (Hathaway and Bezdek 2001).<\/span><\/p>\n<p>In <strong>fuzzy clustering<\/strong> the centroid of a cluster is he mean of all points, weighted by their degree of belonging to the cluster:<\/p>\n<p><span class=\"math\"><span class=\"math display\">\\[<br \/>\nC_j = \\frac{\\sum\\limits_{x \\in C_j} u_{ij}^m x}{\\sum\\limits_{x \\in C_j} u_{ij}^m}<br \/>\n\\]<\/span><\/span><\/p>\n<p>Where,<\/p>\n<ul>\n<li><span class=\"math\"><span class=\"math inline\">\\(C_j\\)<\/span><\/span> is the centroid of the cluster j<\/li>\n<li><span class=\"math\"><span class=\"math inline\">\\(u_{ij}\\)<\/span><\/span> is the degree to which an observation <span class=\"math\"><span class=\"math inline\">\\(x_i\\)<\/span><\/span> belongs to a cluster <span class=\"math\"><span class=\"math inline\">\\(c_j\\)<\/span><\/span><\/li>\n<\/ul>\n<p>The algorithm of fuzzy clustering can be summarize as follow:<\/p>\n<ol style=\"list-style-type: decimal;\">\n<li>Specify a number of clusters k (by the analyst)<\/li>\n<li>Assign randomly to each point coefficients for being in the clusters.<\/li>\n<li>Repeat until the maximum number of iterations (given by \u201cmaxit\u201d) is reached, or when the algorithm has converged (that is, the coefficients\u2019 change between two iterations is no more than <span class=\"math\"><span class=\"math inline\">\\(\\epsilon\\)<\/span><\/span>, the given sensitivity threshold):\n<ul>\n<li>Compute the centroid for each cluster, using the formula above.<\/li>\n<li>For each point, compute its coefficients of being in the clusters, using the formula above.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>The algorithm minimizes intra-cluster variance as well, but has the same problems as k-means; the minimum is a local minimum, and the results depend on the initial choice of weights. Hence, different initializations may lead to different results.<\/p>\n<p>Using a mixture of Gaussians along with the expectation-maximization algorithm is a more statistically formalized method which includes some of these ideas: partial membership in classes.<\/p>\n<\/div>\n<p><!--end rdoc--><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This current article presents the fuzzy c-means clustering algorithm.<\/p>\n","protected":false},"author":1,"featured_media":7863,"parent":8079,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","class_list":["post-8086","dt_lessons","type-dt_lessons","status-publish","has-post-thumbnail","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Fuzzy C-Means Clustering Algorithm - Datanovia<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Fuzzy C-Means Clustering Algorithm - Datanovia\" \/>\n<meta property=\"og:description\" content=\"This current article presents the fuzzy c-means clustering algorithm.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/\" \/>\n<meta property=\"og:site_name\" content=\"Datanovia\" \/>\n<meta property=\"article:modified_time\" content=\"2018-10-25T21:44:54+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/La_Chan_en_Hivers_6.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"512\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/\",\"url\":\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/\",\"name\":\"Fuzzy C-Means Clustering Algorithm - Datanovia\",\"isPartOf\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/La_Chan_en_Hivers_6.jpg\",\"datePublished\":\"2018-10-25T18:31:30+00:00\",\"dateModified\":\"2018-10-25T21:44:54+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/#primaryimage\",\"url\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/La_Chan_en_Hivers_6.jpg\",\"contentUrl\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/La_Chan_en_Hivers_6.jpg\",\"width\":1024,\"height\":512},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.datanovia.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Lessons\",\"item\":\"https:\/\/www.datanovia.com\/en\/lessons\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Fuzzy Clustering Essentials\",\"item\":\"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Fuzzy C-Means Clustering Algorithm\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.datanovia.com\/en\/#website\",\"url\":\"https:\/\/www.datanovia.com\/en\/\",\"name\":\"Datanovia\",\"description\":\"Data Mining and Statistics for Decision Support\",\"publisher\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.datanovia.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.datanovia.com\/en\/#organization\",\"name\":\"Datanovia\",\"url\":\"https:\/\/www.datanovia.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png\",\"contentUrl\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png\",\"width\":98,\"height\":99,\"caption\":\"Datanovia\"},\"image\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Fuzzy C-Means Clustering Algorithm - Datanovia","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/","og_locale":"en_US","og_type":"article","og_title":"Fuzzy C-Means Clustering Algorithm - Datanovia","og_description":"This current article presents the fuzzy c-means clustering algorithm.","og_url":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/","og_site_name":"Datanovia","article_modified_time":"2018-10-25T21:44:54+00:00","og_image":[{"width":1024,"height":512,"url":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/La_Chan_en_Hivers_6.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/","url":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/","name":"Fuzzy C-Means Clustering Algorithm - Datanovia","isPartOf":{"@id":"https:\/\/www.datanovia.com\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/#primaryimage"},"image":{"@id":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/#primaryimage"},"thumbnailUrl":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/La_Chan_en_Hivers_6.jpg","datePublished":"2018-10-25T18:31:30+00:00","dateModified":"2018-10-25T21:44:54+00:00","breadcrumb":{"@id":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/#primaryimage","url":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/La_Chan_en_Hivers_6.jpg","contentUrl":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/10\/La_Chan_en_Hivers_6.jpg","width":1024,"height":512},{"@type":"BreadcrumbList","@id":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/fuzzy-c-means-clustering-algorithm\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.datanovia.com\/en\/"},{"@type":"ListItem","position":2,"name":"Lessons","item":"https:\/\/www.datanovia.com\/en\/lessons\/"},{"@type":"ListItem","position":3,"name":"Fuzzy Clustering Essentials","item":"https:\/\/www.datanovia.com\/en\/lessons\/fuzzy-clustering-essentials\/"},{"@type":"ListItem","position":4,"name":"Fuzzy C-Means Clustering Algorithm"}]},{"@type":"WebSite","@id":"https:\/\/www.datanovia.com\/en\/#website","url":"https:\/\/www.datanovia.com\/en\/","name":"Datanovia","description":"Data Mining and Statistics for Decision Support","publisher":{"@id":"https:\/\/www.datanovia.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.datanovia.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.datanovia.com\/en\/#organization","name":"Datanovia","url":"https:\/\/www.datanovia.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png","contentUrl":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png","width":98,"height":99,"caption":"Datanovia"},"image":{"@id":"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/"}}]}},"multi-rating":{"mr_rating_results":[]},"_links":{"self":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/dt_lessons\/8086","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/dt_lessons"}],"about":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/types\/dt_lessons"}],"author":[{"embeddable":true,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/comments?post=8086"}],"version-history":[{"count":0,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/dt_lessons\/8086\/revisions"}],"up":[{"embeddable":true,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/dt_lessons\/8079"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/media\/7863"}],"wp:attachment":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/media?parent=8086"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}