Comparing Dendrograms in R: Tanglegrams & Correlation

Two linkage methods give two different trees — line them up with a tanglegram, measure their agreement with cophenetic and Baker correlation, and decide which clustering to trust

Machine Learning

Compare two hierarchical clustering trees in R with the dendextend package. Draw a tanglegram to align two dendrograms side by side and spot where they disagree, read the entanglement score, then quantify their similarity with the cophenetic and Baker correlation. Includes comparing several linkage methods at once with a correlation matrix and a corrplot.

Published

June 24, 2026

Modified

July 7, 2026

TipKey takeaways
  • The clustering tree you get depends on the linkage method — average and Ward’s can group the same observations differently. Before you trust a dendrogram, compare it against an alternative.
  • A tanglegram plots two dendrograms face to face and draws a line between each pair of matching labels — the cleaner the lines, the more the two trees agree.
  • Entanglement is a single number (0 to 1) summarising how tangled those lines are: lower is better (0 = perfectly aligned).
  • A tanglegram is visual and can mislead — back it up with a number. cor.dendlist() gives the cophenetic or Baker correlation between trees (−1 to 1; near 1 = very similar).
  • You can compare many trees at once: build a dendlist, get the full correlation matrix, and read it off a corrplot.
Get the book — Practical Guide to Cluster Analysis in R (PDF)

Introduction

You ran hierarchical clustering, got a dendrogram, and grouped your data. Then a reviewer asks the obvious question: would a different linkage method have given different clusters? It often does — average linkage and Ward’s method can put the same observations in different branches. So before you build a story on one tree, you should compare two trees and see how much they actually agree.

This lesson shows the two tools the dendextend package gives you for that:

  • tanglegram() — a visual, side-by-side comparison of two dendrograms.
  • cor.dendlist() — a correlation matrix that turns “how similar are these trees?” into a number.

Everything here is base R plus dendextend. We use the built-in USArrests data so you can run every block as-is.

The data

We’ll use R’s built-in USArrests (arrests per 100,000 residents for Murder, Assault and Rape, plus the percent UrbanPop, for each US state). Different variables are on very different scales, so we standardize them first with scale() — exactly as you would before any distance-based clustering.

A tanglegram with 50 states would be an unreadable wall of lines. To keep the plots legible, we take a small random subset of 10 states with sample(). The set.seed(123) makes that subset reproducible, so your trees match the ones below.

# Standardize all four variables
df <- scale(USArrests)

# Keep a readable random subset of 10 states
set.seed(123)
ss <- sample(1:50, 10)
df <- df[ss, ]

head(df, 3)
               Murder   Assault    UrbanPop        Rape
New Mexico  0.8292944  1.370809  0.30812248  1.16031965
Iowa       -1.2829727 -1.377048 -0.58999237 -1.06038781
Indiana    -0.1350014 -0.693084 -0.03730631 -0.02476943

Build two dendrograms

To have something to compare, we cluster the same data twice with two different linkage methods"average" and "ward.D2" — and convert each hclust() result into a dendrogram with as.dendrogram(). dendlist() then bundles the two dendrograms into a single list object that the comparison functions understand.

library(dendextend)

# One distance matrix, reused for both trees
res.dist <- dist(df, method = "euclidean")

# Two hierarchical clusterings with different linkage
hc1 <- hclust(res.dist, method = "average")
hc2 <- hclust(res.dist, method = "ward.D2")

# Turn each into a dendrogram
dend1 <- as.dendrogram(hc1)
dend2 <- as.dendrogram(hc2)

# Hold both in a dendlist
dend_list <- dendlist(dend1, dend2)
dend_list
[[1]]
'dendrogram' with 2 branches and 10 members total, at height 2.912926 

[[2]]
'dendrogram' with 2 branches and 10 members total, at height 5.908419 

attr(,"class")
[1] "dendlist"

Compare them visually: the tanglegram

A tanglegram draws the two dendrograms facing each other and connects each label on the left to the same label on the right. The functions involved, all from dendextend:

  • untangle() — heuristically rotates branches to find the layout with the fewest crossing lines (we use method = "step1side").
  • tanglegram() — draws the two aligned trees with the connecting lines.
  • entanglement() — scores the alignment quality on a 0 to 1 scale, where 1 is fully tangled and 0 is a perfect, crossing-free alignment. Lower is better.

Read the result like this: straight, horizontal connecting lines mean the two trees place that state the same way; crossing lines mark states the two methods disagree about; and dashed branches are subtrees that exist in one tree but not the other.

# Align the two trees, then draw them side by side
dendlist(dend1, dend2) %>%
  untangle(method = "step1side") %>%  # find the best-aligned layout
  tanglegram()                        # draw the two dendrograms

A tanglegram with the average-linkage dendrogram on the left and the ward.D2 dendrogram on the right, ten US states labelled down the middle. Most connecting lines run straight across, a few cross, and dashed branches mark subtrees unique to one tree.

Now put a number on how well they line up:

# Alignment quality: lower = better (0 is perfect, 1 is fully tangled)
dendlist(dend1, dend2) %>%
  untangle(method = "step1side") %>%
  entanglement()
[1] 0

The entanglement here is 0 — these two trees align perfectly with no crossing lines, so for this subset the choice between average and Ward’s linkage barely changes the grouping.

tanglegram() takes many styling options. A few useful ones: turn off the dashed “unique branch” highlighting, drop the per-line colours, or colour the common subtrees so shared structure pops out.

dendlist(dend1, dend2) %>%
  untangle(method = "step1side") %>%
  tanglegram(
    highlight_distinct_edges = FALSE,    # turn off dashed lines
    common_subtrees_color_lines = FALSE, # turn off line colours
    common_subtrees_color_branches = TRUE # colour the shared branches
  )
Warning

A tanglegram can flatter two trees. Getting the connecting lines to run horizontally does not prove the trees are identical, or even topologically similar — untangle() is allowed to rotate branches to reduce crossings. Always confirm a visual impression with a correlation, which we compute next.

Measure agreement: dendrogram correlation

cor.dendlist() computes a correlation matrix between the trees in a dendlist. It supports two measures, both ranging from −1 to 1, where values near 0 mean the trees are not statistically similar and values near 1 mean they agree strongly:

  • Cophenetic correlation — correlates the cophenetic distances (the height at which each pair of observations first ends up in the same branch). This is the classic measure.
  • Baker’s gamma — a rank-based measure built from the same fusion heights; less sensitive to the exact distances, more to the order in which observations merge.
# Cophenetic correlation matrix
cor.dendlist(dend_list, method = "cophenetic")
          [,1]      [,2]
[1,] 1.0000000 0.9925544
[2,] 0.9925544 1.0000000
# Baker correlation matrix
cor.dendlist(dend_list, method = "baker")
          [,1]      [,2]
[1,] 1.0000000 0.9895528
[2,] 0.9895528 1.0000000

The off-diagonal entry is the number you care about — here about 0.99 for both measures. Both are very close to 1, confirming the tanglegram: the average and Ward’s trees are nearly the same.

For just two trees you can skip the matrix and get the single coefficient directly:

# Cophenetic correlation coefficient
cor_cophenetic(dend1, dend2)
[1] 0.9925544
# Baker correlation coefficient
cor_bakers_gamma(dend1, dend2)
[1] 0.9895528

Compare several linkage methods at once

The matrix really earns its keep when you compare more than two trees. Build one tree per linkage method, give each a name in dendlist(), and cor.dendlist() returns the full pairwise matrix. A corrplot then makes the agreements (and disagreements) jump out at a glance.

# Four trees on the same data, different linkage methods
res.dist <- dist(df, method = "euclidean")
dend1 <- as.dendrogram(hclust(res.dist, method = "complete"))
dend2 <- as.dendrogram(hclust(res.dist, method = "single"))
dend3 <- as.dendrogram(hclust(res.dist, method = "average"))
dend4 <- as.dendrogram(hclust(res.dist, method = "centroid"))

# Name them so the matrix is readable
dend_list <- dendlist(
  "Complete" = dend1, "Single" = dend2,
  "Average"  = dend3, "Centroid" = dend4
)

# Full pairwise (cophenetic) correlation matrix
cors <- cor.dendlist(dend_list)
round(cors, 2)
         Complete Single Average Centroid
Complete     1.00   0.46    0.45     0.30
Single       0.46   1.00    0.23     0.17
Average      0.45   0.23    1.00     0.31
Centroid     0.30   0.17    0.31     1.00
library(corrplot)
corrplot(cors, "pie", "lower")

A lower-triangle pie corrplot of the correlation matrix between four linkage-method dendrograms. All pies are small — no two methods agree strongly; complete vs single is the largest (about 0.46) and centroid is the most isolated.

Read the plot: on this small subset no two linkage methods agree strongly — the highest off-diagonal correlation is only about 0.46 (complete vs single), and centroid linkage is the most isolated of all (0.17–0.31 with the others). In other words, the four methods build genuinely different trees here. That is exactly the kind of disagreement you want to surface before committing to one clustering — on this data, the choice of linkage method clearly matters.

Try it live

Compare two linkage methods yourself. Swap "ward.D2" for "complete", "single" or "centroid", rerun, and watch the tanglegram and the entanglement score change. Click Run.

🟢 With an AI agent

Ask Prova “compare two clustering trees on my data and tell me whether the choice of linkage matters” — it answers with dendextend code you can run on your own data frame, draws the tanglegram, and reads the cophenetic correlation back to you. The runtime is the judge. Ask Prova →

Common issues

My tanglegram is an unreadable wall of lines. Tanglegrams only stay legible for small trees (roughly 10–30 labels). With 50+ observations the connecting lines overlap into noise. Take a random subset (as we did with sample()), or compare the trees with the correlation matrix instead — it scales to any size.

What does the entanglement number actually mean? It measures how crossed the connecting lines are after alignment, on a 0-to-1 scale: 0 is a clean, crossing-free layout and 1 is maximally tangled. Lower is better. It describes the visual layout quality, not how similar the trees are — for similarity, use the cophenetic or Baker correlation.

Cophenetic or Baker — which correlation should I report? Use cophenetic by default; it’s the standard and reflects the actual merge heights. Reach for Baker’s gamma when you care about the order in which observations fuse rather than the exact distances, or when a few large distances would dominate the cophenetic value. They usually agree closely (here, both about 0.99).

Frequently asked questions

A tanglegram is a plot that places two dendrograms facing each other and draws a line connecting each matching label. It’s used to visually compare two clustering trees: straight horizontal lines mark labels the two trees place the same way, crossing lines mark disagreements, and dashed branches mark subtrees unique to one tree. In R, draw one with dendextend::tanglegram().

Build each tree with hclust(), convert them with as.dendrogram(), and put both in a dendlist(). Then visualize the comparison with tanglegram() and quantify it with cor.dendlist() (or cor_cophenetic() for a single pair). The visual and the number together tell you whether the two trees really agree.

It’s the correlation between the two trees’ cophenetic distances — the height at which each pair of observations first lands in the same branch. It ranges from −1 to 1; near 1 means the two trees are very similar, near 0 means they are not statistically similar. Compute it with cor.dendlist(dend_list, method = "cophenetic") or cor_cophenetic(dend1, dend2).

Entanglement runs from 0 to 1 and measures how tangled the tanglegram’s connecting lines are after alignment — lower is better, with 0 meaning a perfect, crossing-free alignment. There’s no hard threshold, but values near 0 (say, below 0.1) indicate the two trees align cleanly. It measures layout quality, not tree similarity.

Both measure how similar two dendrograms are, from −1 to 1. Cophenetic correlation uses the actual merge heights (cophenetic distances). Baker’s gamma is rank-based, built from the order in which observations merge, so it’s less sensitive to the exact distances. They usually agree; report cophenetic by default and use Baker when ranks matter more than magnitudes.

Test your understanding

On the standardized 10-state USArrests subset, build one dendrogram with complete linkage and one with single linkage, then print their cophenetic correlation. Is it higher or lower than the ~0.99 we saw for average vs Ward’s?

# Put "complete" in the first hclust() and "single" in the second.
library(dendextend) df <- scale(USArrests) set.seed(123) df <- df[sample(1:50, 10), ] res.dist <- dist(df, method = "euclidean") dc <- as.dendrogram(hclust(res.dist, method = "complete")) ds <- as.dendrogram(hclust(res.dist, method = "single")) cor_cophenetic(dc, ds)
library(dendextend)
df <- scale(USArrests)
set.seed(123)
df <- df[sample(1:50, 10), ]

res.dist <- dist(df, method = "euclidean")
dc <- as.dendrogram(hclust(res.dist, method = "complete"))
ds <- as.dendrogram(hclust(res.dist, method = "single"))
cor_cophenetic(dc, ds)
  1. Conceptual. A tanglegram shows all connecting lines running perfectly horizontal, yet the cophenetic correlation between the two trees is only 0.6. How is that possible? (Answer: untangle() rotates branches to minimize line crossings, so a clean-looking tanglegram reflects a good layout, not necessarily similar trees. The correlation is the trustworthy measure of similarity — always confirm the picture with a number.)

  2. Conceptual. You build four trees (complete, single, average, centroid) and the correlation matrix shows every off-diagonal value below 0.5, with centroid the most isolated. What should you conclude? (Answer: The four linkage methods produce noticeably different clusterings on this data — none agree strongly. Your grouping is highly sensitive to the linkage method, so report which method you used and why, and don’t treat any single dendrogram as the one true answer.)

Conclusion

A single dendrogram is only one of several plausible trees for your data. dendextend lets you check it against an alternative: bundle two trees with dendlist(), line them up visually with tanglegram() and its entanglement score, then settle the question with a number — the cophenetic or Baker correlation from cor.dendlist(). Scale that to several linkage methods at once with a correlation matrix and a corrplot, and you’ll know before you commit whether your clusters are robust to the choice of method, or an artefact of it.

Reuse

Citation

BibTeX citation:
@online{2026,
  author = {},
  title = {Comparing {Dendrograms} in {R:} {Tanglegrams} \&
    {Correlation}},
  date = {2026-06-24},
  url = {https://www.datanovia.com/learn/machine-learning/clustering/comparing-dendrograms},
  langid = {en}
}
For attribution, please cite this work as:
“Comparing Dendrograms in R: Tanglegrams & Correlation.” 2026. June 24. https://www.datanovia.com/learn/machine-learning/clustering/comparing-dendrograms.