T-Test Essentials: Definition, Formula and Calculation

T-test Effect Size using Cohen’s d Measure

This article describe the t-test effect size. The most commonly used measure of effect size for a t-test is the Cohen’s d (Cohen 1998).

The d statistic redefines the difference in means as the number of standard deviations that separates those means. The formula looks like this (Navarro 2015):

\[
d = \frac{\mbox{(mean 1)} - \mbox{(mean 2)}}{\mbox{std dev}}
\]

In this article, you will learn:

  • Cohen’s d formula to calculate the effect size for one-sample t-test, for independent t-test (with pooled standard deviation or not) and for paired samples t-test (also known as repeated measures t-test).
  • Effect size interpretation describing the critical value corresponding to small, medium and large effect sizes.
  • Calculation of the Cohen’s d in R

Contents:

Related Book

Practical Statistics in R II - Comparing Groups: Numerical Variables

Prerequisites

Load the required R package for computing the Cohen’s d:

library(rstatix)

Demo dataset:

head(ToothGrowth, 3)
##    len supp dose
## 1  4.2   VC  0.5
## 2 11.5   VC  0.5
## 3  7.3   VC  0.5

Effect size interpretation

T-test conventional effect sizes, poposed by Cohen, are: 0.2 (small efect), 0.5 (moderate effect) and 0.8 (large effect) (Cohen 1998, Navarro (2015)). This means that if two groups’ means don’t differ by 0.2 standard deviations or more, the difference is trivial, even if it is statistically significant.

d-value rough interpretation
0.2 Small effect
0.5 Moderate effect
0.8 Large effect

Cohen’s d for one-sample t-test

To calculate an effect size, called Cohen's d, for the one-sample t-test you need to divide the mean difference by the standard deviation of the difference, as shown below. Note that, here: sd(x-mu) = sd(x).

Cohen’s d formula:

\[
d = \frac{m-\mu}{s}
\]

  • \(m\) is the sample mean
  • \(s\) is the sample standard deviation with \(n-1\) degrees of freedom
  • \(\mu\) is the theoretical mean against which the mean of our sample is compared (default value is mu = 0).

Calculation:

ToothGrowth %>% cohens_d(len ~ 1, mu = 0)
## # A tibble: 1 x 6
##   .y.   group1 group2     effsize     n magnitude
## * <chr> <chr>  <chr>        <dbl> <int> <ord>    
## 1 len   1      null model    2.46    60 large

Cohen’s d for independent t-test

The independent samples t-test comes in two different forms:

  • the standard Student’s t-test, which assumes that the variance of the two groups are equal.
  • the Welch’s t-test, which is less restrictive compared to the original Student’s test. This is the test where you do not assume that the variance is the same in the two groups, which results in the fractional degrees of freedom.

Cohen’s d for Student t-test

There are multiple version of Cohen’s d for Student t-test. The most commonly used version of the Student t-test effect size, comparing two groups (\(A\) and \(B\)), is calculated by dividing the mean difference between the groups by the pooled standard deviation.

Cohen’s d formula:

\[
d = \frac{m_A - m_B}{SD_{pooled}}
\]

where,

  • \(m_A\) and \(m_B\) represent the mean value of the group A and B, respectively.
  • \(n_A\) and \(n_B\) represent the sizes of the group A and B, respectively.
  • \(SD_{pooled}\) is an estimator of the pooled standard deviation of the two groups. It can be calculated as follow :
    \[
    SD_{pooled} = \sqrt{\frac{\sum{(x-m_A)^2}+\sum{(x-m_B)^2}}{n_A+n_B-2}}
    \]

Calculation. If the option var.equal = TRUE, then the pooled SD is used when compting the Cohen’s d.

ToothGrowth %>% cohens_d(len ~ supp, var.equal = TRUE)
## # A tibble: 1 x 7
##   .y.   group1 group2 effsize    n1    n2 magnitude
## * <chr> <chr>  <chr>    <dbl> <int> <int> <ord>    
## 1 len   OJ     VC       0.495    30    30 small

Note that, for small sample size (< 50), the Cohen’s d tends to over-inflate results. There exists a Hedge’s Corrected version of the Cohen’s d (Hedges and Olkin 1985), which reduces effect sizes for small samples by a few percentage points. The correction is introduced by multiplying the usual value of d by (N-3)/(N-2.25) (for unpaired t-test) and by (n1-2)/(n1-1.25) for paired t-test; where N is the total size of the two groups being compared (N = n1 + n2).

ToothGrowth %>% cohens_d(
  len ~ supp, var.equal = TRUE, 
  hedges.correction = TRUE
  )
## # A tibble: 1 x 7
##   .y.   group1 group2 effsize    n1    n2 magnitude
## * <chr> <chr>  <chr>    <dbl> <int> <int> <ord>    
## 1 len   OJ     VC       0.488    30    30 small

Cohen’s d for Welch test

The Welch test is a variant of t-test used when the equality of variance can’t be assumed. The effect size can be computed by dividing the mean difference between the groups by the “averaged” standard deviation.

Cohen’s d formula:

\[
d = \frac{m_A - m_B}{\sqrt{(Var_1 + Var_2)/2}}
\]

where,

  • \(m_A\) and \(m_B\) represent the mean value of the group A and B, respectively.
  • \(Var_1\) and \(Var_2\) are the variance of the two groups.

Calculation:

ToothGrowth %>% cohens_d(len ~ supp, var.equal = FALSE)
## # A tibble: 1 x 7
##   .y.   group1 group2 effsize    n1    n2 magnitude
## * <chr> <chr>  <chr>    <dbl> <int> <int> <ord>    
## 1 len   OJ     VC       0.495    30    30 small

Cohen’s d for paired samples t-test

The effect size for a paired-samples t-test can be calculated by dividing the mean difference by the standard deviation of the difference, as shown below.

Cohen’s d formula:

\[
d = \frac{mean_D}{SD_D}
\]

Where D is the differences of the paired samples values.

Calculation:

ToothGrowth %>% cohens_d(len ~ supp, paired = TRUE)
## # A tibble: 1 x 7
##   .y.   group1 group2 effsize    n1    n2 magnitude
## * <chr> <chr>  <chr>    <dbl> <int> <int> <ord>    
## 1 len   OJ     VC       0.603    30    30 moderate

Summary

This article shows how to compute and interpret the t-test effect using the Cohen’s d statistic. We describe the formula of the Cohen’s d for one-sample, two-samples and paired samples t-test. Examples of R codes are provided for the calculations.

Related article

T-test in R

References

Cohen, J. 1998. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates.

Hedges, Larry, and Ingram Olkin. 1985. “Statistical Methods in Meta-Analysis.” In Stat Med. Vol. 20. doi:10.2307/1164953.

Navarro, Daniel. 2015. Learning Statistics with R: A Tutorial for Psychology Students and Other Beginners (Version 0.5).

How to Do Paired T-test in R (Prev Lesson)
Back to T-Test Essentials: Definition, Formula and Calculation

No Comments

Give a comment