Multicollinearity and VIF in R

When predictors are correlated with each other, regression coefficients turn unstable. Detect it with the variance inflation factor (VIF), read the rule of thumb, and fix it by dropping the redundant predictor

Detect and fix multicollinearity in linear regression in R. Compute the variance inflation factor (VIF) with car::vif(), apply the VIF > 5 (or 10) rule of thumb, find the redundant predictors, and refit a simpler model without compromising fit. Worked on the Boston housing data.

TipKey takeaways

• Multicollinearity is when predictors are correlated with each other — it makes regression coefficients unstable (large standard errors, flipping signs) even though the model’s overall fit is fine.
• Detect it with the variance inflation factor (VIF) — car::vif(model) — one score per predictor.
• Rule of thumb: VIF = 1 means no collinearity; VIF > 5 (or 10) signals a problematic amount.
• Fix it by removing the redundant high-VIF predictor — the information is duplicated, so the simpler model fits about as well.
• Multicollinearity affects interpretation (which predictor matters), not the model’s predictions.

Introduction

In multiple regression, each coefficient is the effect of a predictor holding the others constant — but that only works cleanly when the predictors carry distinct information. When two or more predictors are strongly correlated with each other (multicollinearity), their effects can’t be separated: coefficients become unstable, standard errors balloon, and a predictor can even flip sign.

The scenario: a Boston housing model uses a dozen neighbourhood predictors, several of which measure related things (property-tax rate and highway access, for instance). Which ones are redundant — and is the model trustworthy? The variance inflation factor answers that.

The data and a full model

We use the Boston housing dataset from the MASS package — median home value (medv) and 13 neighbourhood predictors. Fit a model with all of them:

data("Boston", package = "MASS")

model <- lm(medv ~ ., data = Boston)
summary(model)$adj.r.squared      # the model fits well overall

[1] 0.7337897

The model explains ~73% of the variation — good overall fit. But are the predictors stepping on each other?

Detect multicollinearity with VIF

car::vif() gives one variance inflation factor per predictor — how much each coefficient’s variance is inflated by collinearity with the others:

library(car)

data("Boston", package = "MASS")
model <- lm(medv ~ ., data = Boston)

sort(vif(model), decreasing = TRUE)

     tax      rad      nox    indus      dis      age    lstat       zn 
9.008554 7.484496 4.393720 3.991596 3.955945 3.100826 2.941491 2.298758 
      rm  ptratio     crim    black     chas 
1.933744 1.799084 1.792192 1.348521 1.073995 

Read it against the rule of thumb (1 = none, > 5 or 10 = problematic):

• tax = 9.0 and rad = 7.5 are both above 5 — the property-tax rate and the highway-accessibility index are strongly correlated with each other (high-tax areas cluster near highways), so the model can’t cleanly separate their effects.
• Everything else is below 5 — acceptable.

Visualize the VIFs

A quick bar chart against the threshold makes the culprits obvious:

library(ggpubr)
library(car)

data("Boston", package = "MASS")
model <- lm(medv ~ ., data = Boston)

vif.df <- data.frame(predictor = names(vif(model)), VIF = as.numeric(vif(model)))

ggbarplot(vif.df, x = "predictor", y = "VIF", fill = "#3a86d4",
          sort.val = "desc", rotate = TRUE, xlab = "", ylab = "VIF") +
  geom_hline(yintercept = 5, linetype = "dashed", color = "red")

Fix it: remove the redundant predictor

The high-VIF predictors carry redundant information, so drop one and refit. Remove tax and re-check — the remaining VIFs fall into the safe range, and the fit is essentially unchanged:

library(car)

data("Boston", package = "MASS")

model2 <- lm(medv ~ . - tax, data = Boston)
sort(vif(model2), decreasing = TRUE)

     nox      dis    indus      age    lstat      rad       zn       rm 
4.369271 3.954446 3.226015 3.098044 2.940800 2.837494 2.184240 1.923075 
    crim  ptratio    black     chas 
1.791940 1.788839 1.347564 1.058220 

summary(model2)$adj.r.squared     # ≈ 0.729 — barely changed from 0.734

[1] 0.7285204

Removing tax pulls rad down to ~2.8 (they were collinear with each other), every VIF is now under 5, and adjusted R² drops only from 0.734 to 0.729 — a simpler, more stable, equally good model. That’s the whole point: the dropped predictor’s information was redundant.

Report

Multicollinearity was assessed with variance inflation factors. Two predictors exceeded the VIF > 5 threshold — tax (VIF = 9.0) and rad (VIF = 7.5) — indicating redundancy. Removing tax brought all VIFs below 5 while adjusted R² was essentially unchanged (0.734 → 0.729), yielding a more stable model.

NoteThe math (optional)

For predictor \(x_j\), regress it on all the other predictors and take that auxiliary model’s \(R_j^2\). Then \(\text{VIF}_j = \dfrac{1}{1 - R_j^2}\). If \(x_j\) is unrelated to the others, \(R_j^2 = 0\) and VIF = 1; if it is perfectly predicted by them, \(R_j^2 \to 1\) and VIF \(\to \infty\). The coefficient’s standard error is inflated by \(\sqrt{\text{VIF}_j}\) — so VIF = 9 means its standard error is 3× larger than it would be without collinearity, which is why the estimate becomes unstable.

Try it live

Remove rad instead of tax — does that also fix the collinearity? Compare the VIFs. The sandbox boots on first Run.

🟢 With an AI agent

Ask Prova “check my regression for multicollinearity and tell me which predictor to drop” — it computes the VIFs and explains which predictors are redundant, with code you can run on your own model. The runtime is the judge. Ask Prova →

Common issues

You panicked about a high VIF when you only care about prediction. Multicollinearity inflates coefficient standard errors and muddies interpretation — but it does not bias the model’s predictions. If you only need accurate predict() values, you can often leave it alone.

A predictor flipped sign or became non-significant. Classic multicollinearity symptom — the predictor shares information with another. Check VIF; the unstable coefficient usually has a high one.

You removed both correlated predictors. Drop one at a time and re-check VIF — removing one often pulls its partner back into the safe range (here, dropping tax fixed rad too). Removing both can discard real signal.

Frequently asked questions

NoteWhat is the variance inflation factor (VIF)?

VIF measures how much a predictor’s coefficient variance is inflated by its correlation with the other predictors. VIF = 1 means no collinearity; it rises as the predictor becomes more redundant. Compute it with car::vif(model) — one value per predictor.

NoteWhat is a good VIF value / what VIF is too high?

VIF = 1 is ideal (no collinearity). The common rule of thumb is that VIF above 5 (or, more lenient, 10) indicates problematic multicollinearity worth addressing. Values between 1 and 5 are generally fine.

NoteHow do I fix multicollinearity in R?

Remove the redundant high-VIF predictor and refit (lm(y ~ . - tax)), then re-check VIF — dropping one often fixes its correlated partner too. Alternatives when you must keep all predictors: combine them into an index, or use dimension-reduction/penalized methods (principal-component or ridge regression).

NoteDoes multicollinearity affect predictions?

No — multicollinearity inflates the standard errors of coefficients and makes their individual interpretation unreliable, but the model’s overall fit and its predictions stay valid. It’s a problem for explaining which predictor matters, not for predicting the outcome.

Test your understanding

ImportantPractice

1. Run it. In the live cell, remove rad instead of tax. Does the tax VIF drop below 5? What does that tell you about the tax–rad relationship?
2. Conceptual. A predictor has VIF = 12 but is the one you most care about scientifically. List two options other than simply deleting it.

NoteHint

Fill the blank with tax, then rad. Either removal should bring the other one’s VIF down, because they’re collinear with each other. For question 2, think about combining variables or using a method built for correlated predictors.

NoteSolution

Removing either tax or rad brings the other below 5 — they are collinear with each other, so one of them carries the shared information. For question 2: instead of deleting a scientifically important high-VIF predictor, you can (a) combine the correlated predictors into a single index/score, or (b) use a method designed for correlated predictors — principal-component regression or ridge (penalized) regression — which stabilise the estimates without dropping the variable.

Conclusion

You can now detect and fix multicollinearity in R: compute VIF with car::vif(), apply the > 5 (or 10) rule of thumb to spot redundant predictors, and remove the worst offender to get a simpler, more stable model with essentially the same fit. Multicollinearity is about trusting which predictor matters — check it whenever you interpret a multiple-regression model’s coefficients.

Related lessons

• Multiple linear regression in R — the model VIF diagnoses. · Linear regression assumptions & diagnostics in R — the other checks every model needs. · Correlation test in R — measure the pairwise correlation behind collinearity.
• Go further: Regression analysis · Biostatistics

Was this page helpful?

Prove you can do it. Master the whole Regression in R series — track your path, build projects, and earn a certificate.

Start free →

Go Pro — unlimited Prova on your own data and a verifiable certificate that proves the skill.

from $15/mo billed yearly

Go Pro →

✓ You're Pro — keep going. The runtime is the judge.

Ready to level up?

Try this on your own data with Prova — the runtime is the judge.Open Prova →

Get new R & Python lessons by email

Practical, reproducible, no spam. Unsubscribe anytime.

Double opt-in. We never share your email.

Share this pageXLinkedInRedditHN

Note

Every result on this page was produced by the code shown, run at build time against a pinned R environment — edit any block and Run to reproduce it yourself.