{"id":10873,"date":"2019-11-29T08:25:52","date_gmt":"2019-11-29T06:25:52","guid":{"rendered":"https:\/\/www.datanovia.com\/en\/?post_type=dt_lessons&#038;p=10873"},"modified":"2019-11-29T08:25:52","modified_gmt":"2019-11-29T06:25:52","slug":"repeated-measures-anova-in-r","status":"publish","type":"dt_lessons","link":"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/","title":{"rendered":"Repeated Measures ANOVA in R"},"content":{"rendered":"<div id=\"rdoc\">\n<p>The <strong>repeated-measures ANOVA<\/strong> is used for analyzing data where same subjects are measured more than once. This test is also referred to as a <em>within-subjects ANOVA<\/em> or <em>ANOVA with repeated measures<\/em>. The \u201cwithin-subjects\u201d term means that the same individuals are measured on the same outcome variable under different time points or conditions.<\/p>\n<p>For example, you might have measured 10 individuals\u2019 self-esteem score (the outcome or dependent variable) on three time points during a specific diet to determine whether their self-esteem improved.<\/p>\n<p>This chapter describes the different types of repeated measures ANOVA, including:<\/p>\n<ul>\n<li><strong>One-way repeated measures ANOVA<\/strong>, an extension of the paired-samples t-test for comparing the means of three or more levels of a <em>within-subjects<\/em> variable.<\/li>\n<li><strong>two-way repeated measures ANOVA<\/strong> used to evaluate simultaneously the effect of two within-subject factors on a continuous outcome variable.<\/li>\n<li><strong>three-way repeated measures ANOVA<\/strong> used to evaluate simultaneously the effect of three within-subject factors on a continuous outcome variable.<\/li>\n<\/ul>\n<div class=\"block\">\n<p>The main goal of two-way and three-way repeated measures ANOVA is, respectively, to evaluate if there is a statistically significant interaction effect between two and three within-subjects factors in explaining a continuous outcome variable.<\/p>\n<\/div>\n<p>You will learn how to:<\/p>\n<ul>\n<li><strong>Compute and interpret the different repeated measures ANOVA in R<\/strong>.<\/li>\n<li><strong>Check repeated measures ANOVA test assumptions<\/strong><\/li>\n<li><strong>Perform post-hoc tests<\/strong>, multiple pairwise comparisons between groups to identify which groups are different<\/li>\n<li><strong>Visualize the data<\/strong> using box plots, add ANOVA and pairwise comparisons p-values to the plot<\/li>\n<\/ul>\n<p>Contents:<\/p>\n<div id=\"TOC\">\n<ul>\n<li><a href=\"#assumptions\">Assumptions<\/a><\/li>\n<li><a href=\"#prerequisites\">Prerequisites<\/a><\/li>\n<li><a href=\"#one-way-repeated-measures-anova\">One-way repeated measures ANOVA<\/a>\n<ul>\n<li><a href=\"#data-preparation\">Data preparation<\/a><\/li>\n<li><a href=\"#summary-statistics\">Summary statistics<\/a><\/li>\n<li><a href=\"#visualization\">Visualization<\/a><\/li>\n<li><a href=\"#check-assumptions\">Check assumptions<\/a><\/li>\n<li><a href=\"#computation\">Computation<\/a><\/li>\n<li><a href=\"#post-hoc-tests\">Post-hoc tests<\/a><\/li>\n<li><a href=\"#report\">Report<\/a><\/li>\n<\/ul>\n<\/li>\n<li><a href=\"#two-way-repeated-measures-anova\">Two-way repeated measures ANOVA<\/a>\n<ul>\n<li><a href=\"#data-preparation-1\">Data preparation<\/a><\/li>\n<li><a href=\"#summary-statistics-1\">Summary statistics<\/a><\/li>\n<li><a href=\"#visualization-1\">Visualization<\/a><\/li>\n<li><a href=\"#chek-assumptions\">Chek assumptions<\/a><\/li>\n<li><a href=\"#computation-1\">Computation<\/a><\/li>\n<li><a href=\"#post-hoc-tests-1\">Post-hoc tests<\/a><\/li>\n<li><a href=\"#report-1\">Report<\/a><\/li>\n<\/ul>\n<\/li>\n<li><a href=\"#three-way\">Three-way repeated measures ANOVA<\/a>\n<ul>\n<li><a href=\"#data-preparation-2\">Data preparation<\/a><\/li>\n<li><a href=\"#summary-statistics-2\">Summary statistics<\/a><\/li>\n<li><a href=\"#visualization-2\">Visualization<\/a><\/li>\n<li><a href=\"#check-assumptions-1\">Check assumptions<\/a><\/li>\n<li><a href=\"#computation-2\">Computation<\/a><\/li>\n<li><a href=\"#post-hoc-tests-2\">Post-hoc tests<\/a><\/li>\n<li><a href=\"#report-2\">Report<\/a><\/li>\n<\/ul>\n<\/li>\n<li><a href=\"#summary\">Summary<\/a><\/li>\n<\/ul>\n<\/div>\n<div class='dt-sc-hr-invisible-medium  '><\/div>\n<div class='dt-sc-ico-content type1'><div class='custom-icon' ><a href='https:\/\/www.datanovia.com\/en\/product\/practical-statistics-in-r-for-comparing-groups-numerical-variables\/' target='_blank'><span class='fa fa-book'><\/span><\/a><\/div><h4><a href='https:\/\/www.datanovia.com\/en\/product\/practical-statistics-in-r-for-comparing-groups-numerical-variables\/' target='_blank'> Related Book <\/a><\/h4>Practical Statistics in R II - Comparing Groups: Numerical Variables<\/div>\n<div class='dt-sc-hr-invisible-medium  '><\/div>\n<div id=\"assumptions\" class=\"section level2\">\n<h2>Assumptions<\/h2>\n<p>The repeated measures ANOVA makes the following assumptions about the data:<\/p>\n<ul>\n<li><strong>No significant outliers<\/strong> in any cell of the design. This can be checked by visualizing the data using box plot methods and by using the function <code>identify_outliers()<\/code> [rstatix package].<\/li>\n<li><strong>Normality<\/strong>: the outcome (or dependent) variable should be approximately normally distributed in each cell of the design. This can be checked using the <strong>Shapiro-Wilk normality test<\/strong> (<code>shapiro_test()<\/code> [rstatix]) or by visual inspection using <strong>QQ plot<\/strong> (<code>ggqqplot()<\/code> [ggpubr package]).<\/li>\n<li><strong>Assumption of sphericity<\/strong>: the variance of the differences between groups should be equal. This can be checked using the <strong>Mauchly\u2019s test of sphericity<\/strong>, which is automatically reported when using the R function <code>anova_test()<\/code> [rstatix package]. Read more in Chapter @ref(mauchly-s-test-of-sphericity-in-r).<\/li>\n<\/ul>\n<p>Before computing repeated measures ANOVA test, you need to perform some preliminary tests to check if the assumptions are met.<\/p>\n<div class=\"warning\">\n<p>Note that, if the above assumptions are not met there are a non-parametric alternative (<em>Friedman test<\/em>) to the one-way repeated measures ANOVA.<\/p>\n<p>Unfortunately, there are no non-parametric alternatives to the two-way and the three-way repeated measures ANOVA. Thus, in the situation where the assumptions are not met, you could consider running the two-way\/three-way repeated measures ANOVA on the transformed and non-transformed data to see if there are any meaningful differences.<\/p>\n<p>If both tests lead you to the same conclusions, you might not choose to transform the outcome variable and carry on with the two-way\/three-way repeated measures ANOVA on the original data.<\/p>\n<p>It\u2019s also possible to perform robust ANOVA test using the <strong>WRS2<\/strong> R package.<\/p>\n<p>No matter your choice, you should report what you did in your results.<\/p>\n<\/div>\n<\/div>\n<div id=\"prerequisites\" class=\"section level2\">\n<h2>Prerequisites<\/h2>\n<p>Make sure that you have installed the following R packages:<\/p>\n<ul>\n<li><code>tidyverse<\/code> for data manipulation and visualization<\/li>\n<li><code>ggpubr<\/code> for creating easily publication ready plots<\/li>\n<li><code>rstatix<\/code> provides pipe-friendly R functions for easy statistical analyses<\/li>\n<li><code>datarium<\/code>: contains required data sets for this chapter<\/li>\n<\/ul>\n<p>Start by loading the following R packages:<\/p>\n<pre class=\"r\"><code>library(tidyverse)\r\nlibrary(ggpubr)\r\nlibrary(rstatix)<\/code><\/pre>\n<p>Key R functions:<\/p>\n<ul>\n<li><code>anova_test()<\/code> [rstatix package], a wrapper around <code>car::Anova()<\/code> for making easy the computation of repeated measures ANOVA. Key arguments for performing repeated measures ANOVA:\n<ul>\n<li><code>data<\/code>: data frame<\/li>\n<li><code>dv<\/code>: (numeric) the dependent (or outcome) variable name.<\/li>\n<li><code>wid<\/code>: variable name specifying the case\/sample identifier.<\/li>\n<li><code>within<\/code>: within-subjects factor or grouping variable<\/li>\n<\/ul>\n<\/li>\n<li><code>get_anova_table()<\/code> [rstatix package]. Extracts the ANOVA table from the output of <code>anova_test()<\/code>. It returns ANOVA table that is automatically corrected for eventual deviation from the sphericity assumption. The default is to apply automatically the Greenhouse-Geisser sphericity correction to only within-subject factors violating the sphericity assumption (i.e., Mauchly\u2019s test p-value is significant, p &lt;= 0.05). Read more in Chapter @ref(mauchly-s-test-of-sphericity-in-r).<\/li>\n<\/ul>\n<\/div>\n<div id=\"one-way-repeated-measures-anova\" class=\"section level2\">\n<h2>One-way repeated measures ANOVA<\/h2>\n<div id=\"data-preparation\" class=\"section level3\">\n<h3>Data preparation<\/h3>\n<p>We\u2019ll use the self-esteem score dataset measured over three time points. The data is available in the datarium package.<\/p>\n<pre class=\"r\"><code># Data preparation\r\n# Wide format\r\ndata(\"selfesteem\", package = \"datarium\")\r\nhead(selfesteem, 3)<\/code><\/pre>\n<pre><code>## # A tibble: 3 x 4\r\n##      id    t1    t2    t3\r\n##   &lt;int&gt; &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt;\r\n## 1     1  4.01  5.18  7.11\r\n## 2     2  2.56  6.91  6.31\r\n## 3     3  3.24  4.44  9.78<\/code><\/pre>\n<pre class=\"r\"><code># Gather columns t1, t2 and t3 into long format\r\n# Convert id and time into factor variables\r\nselfesteem &lt;- selfesteem %&gt;%\r\n  gather(key = \"time\", value = \"score\", t1, t2, t3) %&gt;%\r\n  convert_as_factor(id, time)\r\nhead(selfesteem, 3)<\/code><\/pre>\n<pre><code>## # A tibble: 3 x 3\r\n##   id    time  score\r\n##   &lt;fct&gt; &lt;fct&gt; &lt;dbl&gt;\r\n## 1 1     t1     4.01\r\n## 2 2     t1     2.56\r\n## 3 3     t1     3.24<\/code><\/pre>\n<div class=\"block\">\n<p>The one-way repeated measures ANOVA can be used to determine whether the means self-esteem scores are significantly different between the three time points.<\/p>\n<\/div>\n<\/div>\n<div id=\"summary-statistics\" class=\"section level3\">\n<h3>Summary statistics<\/h3>\n<p>Compute some summary statistics of the self-esteem <code>score<\/code> by groups (<code>time<\/code>): mean and sd (standard deviation)<\/p>\n<pre class=\"r\"><code>selfesteem %&gt;%\r\n  group_by(time) %&gt;%\r\n  get_summary_stats(score, type = \"mean_sd\")<\/code><\/pre>\n<pre><code>## # A tibble: 3 x 5\r\n##   time  variable     n  mean    sd\r\n##   &lt;fct&gt; &lt;chr&gt;    &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt;\r\n## 1 t1    score       10  3.14 0.552\r\n## 2 t2    score       10  4.93 0.863\r\n## 3 t3    score       10  7.64 1.14<\/code><\/pre>\n<\/div>\n<div id=\"visualization\" class=\"section level3\">\n<h3>Visualization<\/h3>\n<p>Create a box plot and add points corresponding to individual values:<\/p>\n<pre class=\"r\"><code>bxp &lt;- ggboxplot(selfesteem, x = \"time\", y = \"score\", add = \"point\")\r\nbxp<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/r-statistics-2-comparing-groups-means\/figures\/046-repeated-measure-anova-one-way-boxplot-1.png\" width=\"336\" \/><\/p>\n<\/div>\n<div id=\"check-assumptions\" class=\"section level3\">\n<h3>Check assumptions<\/h3>\n<div id=\"outliers\" class=\"section level4\">\n<h4>Outliers<\/h4>\n<p>Outliers can be easily identified using box plot methods, implemented in the R function <code>identify_outliers()<\/code> [rstatix package].<\/p>\n<pre class=\"r\"><code>selfesteem %&gt;%\r\n  group_by(time) %&gt;%\r\n  identify_outliers(score)<\/code><\/pre>\n<pre><code>## # A tibble: 2 x 5\r\n##   time  id    score is.outlier is.extreme\r\n##   &lt;fct&gt; &lt;fct&gt; &lt;dbl&gt; &lt;lgl&gt;      &lt;lgl&gt;     \r\n## 1 t1    6      2.05 TRUE       FALSE     \r\n## 2 t2    2      6.91 TRUE       FALSE<\/code><\/pre>\n<div class=\"success\">\n<p>There were no extreme outliers.<\/p>\n<\/div>\n<div class=\"warning\">\n<p>Note that, in the situation where you have extreme outliers, this can be due to: 1) data entry errors, measurement errors or unusual values.<\/p>\n<p>You can include the outlier in the analysis anyway if you do not believe the result will be substantially affected. This can be evaluated by comparing the result of the ANOVA with and without the outlier.<\/p>\n<p>It\u2019s also possible to keep the outliers in the data and perform robust ANOVA test using the WRS2 package.<\/p>\n<\/div>\n<\/div>\n<div id=\"normality-assumption\" class=\"section level4\">\n<h4>Normality assumption<\/h4>\n<p>The normality assumption can be checked by computing Shapiro-Wilk test for each time point. If the data is normally distributed, the p-value should be greater than 0.05.<\/p>\n<pre class=\"r\"><code>selfesteem %&gt;%\r\n  group_by(time) %&gt;%\r\n  shapiro_test(score)<\/code><\/pre>\n<pre><code>## # A tibble: 3 x 4\r\n##   time  variable statistic     p\r\n##   &lt;fct&gt; &lt;chr&gt;        &lt;dbl&gt; &lt;dbl&gt;\r\n## 1 t1    score        0.967 0.859\r\n## 2 t2    score        0.876 0.117\r\n## 3 t3    score        0.923 0.380<\/code><\/pre>\n<div class=\"success\">\n<p>The self-esteem score was normally distributed at each time point, as assessed by Shapiro-Wilk\u2019s test (p &gt; 0.05).<\/p>\n<\/div>\n<p>Note that, if your sample size is greater than 50, the normal QQ plot is preferred because at larger sample sizes the Shapiro-Wilk test becomes very sensitive even to a minor deviation from normality.<\/p>\n<p>QQ plot draws the correlation between a given data and the normal distribution. Create QQ plots for each time point:<\/p>\n<pre class=\"r\"><code>ggqqplot(selfesteem, \"score\", facet.by = \"time\")<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/r-statistics-2-comparing-groups-means\/figures\/046-repeated-measure-anova-qq-plot-1.png\" width=\"480\" \/><\/p>\n<div class=\"success\">\n<p>From the plot above, as all the points fall approximately along the reference line, we can assume normality.<\/p>\n<\/div>\n<\/div>\n<div id=\"assumption-of-sphericity\" class=\"section level4\">\n<h4>Assumption of sphericity<\/h4>\n<p>As mentioned in previous sections, the assumption of sphericity will be automatically checked during the computation of the ANOVA test using the R function <code>anova_test()<\/code> [rstatix package]. The Mauchly\u2019s test is internally used to assess the sphericity assumption.<\/p>\n<p>By using the function <code>get_anova_table()<\/code> [rstatix] to extract the ANOVA table, the Greenhouse-Geisser sphericity correction is automatically applied to factors violating the sphericity assumption.<\/p>\n<\/div>\n<\/div>\n<div id=\"computation\" class=\"section level3\">\n<h3>Computation<\/h3>\n<pre class=\"r\"><code>res.aov &lt;- anova_test(data = selfesteem, dv = score, wid = id, within = time)\r\nget_anova_table(res.aov)<\/code><\/pre>\n<pre><code>## ANOVA Table (type III tests)\r\n## \r\n##   Effect DFn DFd    F        p p&lt;.05   ges\r\n## 1   time   2  18 55.5 2.01e-08     * 0.829<\/code><\/pre>\n<div class=\"success\">\n<p>The self-esteem score was statistically significantly different at the different time points during the diet, F(2, 18) = 55.5, p &lt; 0.0001, eta2[g] = 0.83.<\/p>\n<\/div>\n<p>where,<\/p>\n<ul>\n<li><code>F<\/code> Indicates that we are comparing to an F-distribution (F-test); <code>(2, 18)<\/code> indicates the degrees of freedom in the numerator (DFn) and the denominator (DFd), respectively; <code>55.5<\/code> indicates the obtained F-statistic value<\/li>\n<li><code>p<\/code> specifies the p-value<\/li>\n<li><code>ges<\/code> is the generalized effect size (amount of variability due to the within-subjects factor)<\/li>\n<\/ul>\n<\/div>\n<div id=\"post-hoc-tests\" class=\"section level3\">\n<h3>Post-hoc tests<\/h3>\n<p>You can perform multiple <strong>pairwise paired t-tests<\/strong> between the levels of the within-subjects factor (here <code>time<\/code>). P-values are adjusted using the Bonferroni multiple testing correction method.<\/p>\n<pre class=\"r\"><code># pairwise comparisons\r\npwc &lt;- selfesteem %&gt;%\r\n  pairwise_t_test(\r\n    score ~ time, paired = TRUE,\r\n    p.adjust.method = \"bonferroni\"\r\n    )\r\npwc<\/code><\/pre>\n<pre><code>## # A tibble: 3 x 10\r\n##   .y.   group1 group2    n1    n2 statistic    df           p    p.adj p.adj.signif\r\n## * &lt;chr&gt; &lt;chr&gt;  &lt;chr&gt;  &lt;int&gt; &lt;int&gt;     &lt;dbl&gt; &lt;dbl&gt;       &lt;dbl&gt;    &lt;dbl&gt; &lt;chr&gt;       \r\n## 1 score t1     t2        10    10     -4.97     9 0.000772    0.002    **          \r\n## 2 score t1     t3        10    10    -13.2      9 0.000000334 0.000001 ****        \r\n## 3 score t2     t3        10    10     -4.87     9 0.000886    0.003    **<\/code><\/pre>\n<div class=\"success\">\n<p>All the pairwise differences are statistically significant.<\/p>\n<\/div>\n<\/div>\n<div id=\"report\" class=\"section level3\">\n<h3>Report<\/h3>\n<p>We could report the result as follow:<\/p>\n<p>The self-esteem score was statistically significantly different at the different time points, F(2, 18) = 55.5, p &lt; 0.0001, generalized eta squared = 0.82.<\/p>\n<p>Post-hoc analyses with a Bonferroni adjustment revealed that all the pairwise differences, between time points, were statistically significantly different (p &lt;= 0.05).<\/p>\n<pre class=\"r\"><code># Visualization: box plots with p-values\r\npwc &lt;- pwc %&gt;% add_xy_position(x = \"time\")\r\nbxp + \r\n  stat_pvalue_manual(pwc) +\r\n  labs(\r\n    subtitle = get_test_label(res.aov, detailed = TRUE),\r\n    caption = get_pwc_label(pwc)\r\n  )<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/r-statistics-2-comparing-groups-means\/figures\/046-repeated-measure-anova-one-way-boxplots-with-p-values-1.png\" width=\"480\" \/><\/p>\n<\/div>\n<\/div>\n<div id=\"two-way-repeated-measures-anova\" class=\"section level2\">\n<h2>Two-way repeated measures ANOVA<\/h2>\n<div id=\"data-preparation-1\" class=\"section level3\">\n<h3>Data preparation<\/h3>\n<p>We\u2019ll use the <code>selfesteem2<\/code> dataset [in datarium package] containing the self-esteem score measures of 12 individuals enrolled in 2 successive short-term trials (4 weeks): control (placebo) and special diet trials.<\/p>\n<p>Each participant performed all two trials. The order of the trials was counterbalanced and sufficient time was allowed between trials to allow any effects of previous trials to have dissipated.<\/p>\n<p>The self-esteem score was recorded at three time points: at the beginning (t1), midway (t2) and at the end (t3) of the trials.<\/p>\n<p>The question is to investigate if this short-term diet treatment can induce a significant increase of self-esteem score over time. In other terms, we wish to know if there is significant interaction between <code>diet<\/code> and <code>time<\/code> on the self-esteem score.<\/p>\n<div class=\"block\">\n<p>The two-way repeated measures ANOVA can be performed in order to determine whether there is a significant interaction between diet and time on the self-esteem score.<\/p>\n<\/div>\n<p>Load and show one random row by treatment group:<\/p>\n<pre class=\"r\"><code># Wide format\r\nset.seed(123)\r\ndata(\"selfesteem2\", package = \"datarium\")\r\nselfesteem2 %&gt;% sample_n_by(treatment, size = 1)<\/code><\/pre>\n<pre><code>## # A tibble: 2 x 5\r\n##   id    treatment    t1    t2    t3\r\n##   &lt;fct&gt; &lt;fct&gt;     &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt;\r\n## 1 4     ctr          92    92    89\r\n## 2 10    Diet         90    93    95<\/code><\/pre>\n<pre class=\"r\"><code># Gather the columns t1, t2 and t3 into long format.\r\n# Convert id and time into factor variables\r\nselfesteem2 &lt;- selfesteem2 %&gt;%\r\n  gather(key = \"time\", value = \"score\", t1, t2, t3) %&gt;%\r\n  convert_as_factor(id, time)\r\n# Inspect some random rows of the data by groups\r\nset.seed(123)\r\nselfesteem2 %&gt;% sample_n_by(treatment, time, size = 1)<\/code><\/pre>\n<pre><code>## # A tibble: 6 x 4\r\n##   id    treatment time  score\r\n##   &lt;fct&gt; &lt;fct&gt;     &lt;fct&gt; &lt;dbl&gt;\r\n## 1 4     ctr       t1       92\r\n## 2 10    ctr       t2       84\r\n## 3 5     ctr       t3       68\r\n## 4 11    Diet      t1       93\r\n## 5 12    Diet      t2       80\r\n## 6 1     Diet      t3       88<\/code><\/pre>\n<div class=\"block\">\n<p>In this example, the effect of \u201ctime\u201d on self-esteem score is our <strong>focal variable<\/strong>, that is our primary concern.<\/p>\n<p>However, it is thought that the effect \u201ctime\u201d will be different if treatment is performed or not. In this setting, the \u201ctreatment\u201d variable is considered as <strong>moderator variable<\/strong>.<\/p>\n<\/div>\n<\/div>\n<div id=\"summary-statistics-1\" class=\"section level3\">\n<h3>Summary statistics<\/h3>\n<p>Group the data by <code>treatment<\/code> and <code>time<\/code>, and then compute some summary statistics of the <code>score<\/code> variable: mean and sd (standard deviation).<\/p>\n<pre class=\"r\"><code>selfesteem2 %&gt;%\r\n  group_by(treatment, time) %&gt;%\r\n  get_summary_stats(score, type = \"mean_sd\")<\/code><\/pre>\n<pre><code>## # A tibble: 6 x 6\r\n##   treatment time  variable     n  mean    sd\r\n##   &lt;fct&gt;     &lt;fct&gt; &lt;chr&gt;    &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt;\r\n## 1 ctr       t1    score       12  88    8.08\r\n## 2 ctr       t2    score       12  83.8 10.2 \r\n## 3 ctr       t3    score       12  78.7 10.5 \r\n## 4 Diet      t1    score       12  87.6  7.62\r\n## 5 Diet      t2    score       12  87.8  7.42\r\n## 6 Diet      t3    score       12  87.7  8.14<\/code><\/pre>\n<\/div>\n<div id=\"visualization-1\" class=\"section level3\">\n<h3>Visualization<\/h3>\n<p>Create box plots of the score colored by treatment groups:<\/p>\n<pre class=\"r\"><code>bxp &lt;- ggboxplot(\r\n  selfesteem2, x = \"time\", y = \"score\",\r\n  color = \"treatment\", palette = \"jco\"\r\n  )\r\nbxp<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/r-statistics-2-comparing-groups-means\/figures\/046-repeated-measure-anova-two-way-boxplots-1.png\" width=\"384\" \/><\/p>\n<\/div>\n<div id=\"chek-assumptions\" class=\"section level3\">\n<h3>Chek assumptions<\/h3>\n<div id=\"outliers-1\" class=\"section level4\">\n<h4>Outliers<\/h4>\n<pre class=\"r\"><code>selfesteem2 %&gt;%\r\n  group_by(treatment, time) %&gt;%\r\n  identify_outliers(score)<\/code><\/pre>\n<pre><code>## [1] treatment  time       id         score      is.outlier is.extreme\r\n## &lt;0 rows&gt; (or 0-length row.names)<\/code><\/pre>\n<div class=\"success\">\n<p>There were no extreme outliers.<\/p>\n<\/div>\n<\/div>\n<div id=\"normality-assumption-1\" class=\"section level4\">\n<h4>Normality assumption<\/h4>\n<p>Compute Shapiro-Wilk test for each combinations of factor levels:<\/p>\n<pre class=\"r\"><code>selfesteem2 %&gt;%\r\n  group_by(treatment, time) %&gt;%\r\n  shapiro_test(score)<\/code><\/pre>\n<pre><code>## # A tibble: 6 x 5\r\n##   treatment time  variable statistic      p\r\n##   &lt;fct&gt;     &lt;fct&gt; &lt;chr&gt;        &lt;dbl&gt;  &lt;dbl&gt;\r\n## 1 ctr       t1    score        0.828 0.0200\r\n## 2 ctr       t2    score        0.868 0.0618\r\n## 3 ctr       t3    score        0.887 0.107 \r\n## 4 Diet      t1    score        0.919 0.279 \r\n## 5 Diet      t2    score        0.923 0.316 \r\n## 6 Diet      t3    score        0.886 0.104<\/code><\/pre>\n<div class=\"success\">\n<p>The self-esteem score was normally distributed at each time point (p &gt; 0.05), except for ctr treatment at t1, as assessed by Shapiro-Wilk\u2019s test.<\/p>\n<\/div>\n<p>Create QQ plot for each cell of design:<\/p>\n<pre class=\"r\"><code>ggqqplot(selfesteem2, \"score\", ggtheme = theme_bw()) +\r\n  facet_grid(time ~ treatment, labeller = \"label_both\")<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/r-statistics-2-comparing-groups-means\/figures\/046-repeated-measure-anova-two-way-qq-plot-1.png\" width=\"480\" \/><\/p>\n<div class=\"success\">\n<p>From the plot above, as all the points fall approximately along the reference line, we can assume normality.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div id=\"computation-1\" class=\"section level3\">\n<h3>Computation<\/h3>\n<pre class=\"r\"><code>res.aov &lt;- anova_test(\r\n  data = selfesteem2, dv = score, wid = id,\r\n  within = c(treatment, time)\r\n  )\r\nget_anova_table(res.aov)<\/code><\/pre>\n<pre><code>## ANOVA Table (type III tests)\r\n## \r\n##           Effect  DFn  DFd    F        p p&lt;.05   ges\r\n## 1      treatment 1.00 11.0 15.5 2.00e-03     * 0.059\r\n## 2           time 1.31 14.4 27.4 5.03e-05     * 0.049\r\n## 3 treatment:time 2.00 22.0 30.4 4.63e-07     * 0.050<\/code><\/pre>\n<div class=\"success\">\n<p>There is a statistically significant two-way interactions between treatment and time, F(2, 22) = 30.4, p &lt; 0.0001.<\/p>\n<\/div>\n<\/div>\n<div id=\"post-hoc-tests-1\" class=\"section level3\">\n<h3>Post-hoc tests<\/h3>\n<p>A <strong>significant two-way interaction<\/strong> indicates that the impact that one factor (e.g., treatment) has on the outcome variable (e.g., self-esteem score) depends on the level of the other factor (e.g., time) (and vice versa). So, you can decompose a significant two-way interaction into:<\/p>\n<ul>\n<li><strong>Simple main effect<\/strong>: run one-way model of the first variable (factor A) at each level of the second variable (factor B),<\/li>\n<li><strong>Simple pairwise comparisons<\/strong>: if the simple main effect is significant, run multiple pairwise comparisons to determine which groups are different.<\/li>\n<\/ul>\n<p>For a <strong>non-significant two-way interaction<\/strong>, you need to determine whether you have any statistically significant <strong>main effects<\/strong> from the ANOVA output.<\/p>\n<div id=\"procedure-for-a-significant-two-way-interaction\" class=\"section level4\">\n<h4>Procedure for a significant two-way interaction<\/h4>\n<p><strong>Effect of treatment<\/strong>. In our example, we\u2019ll analyze the effect of <code>treatment<\/code> on self-esteem <code>score<\/code> at every <code>time<\/code> point.<\/p>\n<div class=\"warning\">\n<p>Note that, the <code>treatment<\/code> factor variable has only two levels (\u201cctr\u201d and \u201cDiet\u201d); thus, ANOVA test and paired t-test will give the same p-values.<\/p>\n<\/div>\n<pre class=\"r\"><code># Effect of treatment at each time point\r\none.way &lt;- selfesteem2 %&gt;%\r\n  group_by(time) %&gt;%\r\n  anova_test(dv = score, wid = id, within = treatment) %&gt;%\r\n  get_anova_table() %&gt;%\r\n  adjust_pvalue(method = \"bonferroni\")\r\none.way<\/code><\/pre>\n<pre><code>## # A tibble: 3 x 9\r\n##   time  Effect      DFn   DFd      F       p `p&lt;.05`      ges   p.adj\r\n##   &lt;fct&gt; &lt;chr&gt;     &lt;dbl&gt; &lt;dbl&gt;  &lt;dbl&gt;   &lt;dbl&gt; &lt;chr&gt;      &lt;dbl&gt;   &lt;dbl&gt;\r\n## 1 t1    treatment     1    11  0.376 0.552   \"\"      0.000767 1      \r\n## 2 t2    treatment     1    11  9.03  0.012   *       0.052    0.036  \r\n## 3 t3    treatment     1    11 30.9   0.00017 *       0.199    0.00051<\/code><\/pre>\n<pre class=\"r\"><code># Pairwise comparisons between treatment groups\r\npwc &lt;- selfesteem2 %&gt;%\r\n  group_by(time) %&gt;%\r\n  pairwise_t_test(\r\n    score ~ treatment, paired = TRUE,\r\n    p.adjust.method = \"bonferroni\"\r\n    )\r\npwc<\/code><\/pre>\n<pre><code>## # A tibble: 3 x 11\r\n##   time  .y.   group1 group2    n1    n2 statistic    df       p   p.adj p.adj.signif\r\n## * &lt;fct&gt; &lt;chr&gt; &lt;chr&gt;  &lt;chr&gt;  &lt;int&gt; &lt;int&gt;     &lt;dbl&gt; &lt;dbl&gt;   &lt;dbl&gt;   &lt;dbl&gt; &lt;chr&gt;       \r\n## 1 t1    score ctr    Diet      12    12     0.613    11 0.552   0.552   ns          \r\n## 2 t2    score ctr    Diet      12    12    -3.00     11 0.012   0.012   *           \r\n## 3 t3    score ctr    Diet      12    12    -5.56     11 0.00017 0.00017 ***<\/code><\/pre>\n<div class=\"success\">\n<p>Considering the Bonferroni adjusted p-value (p.adj), it can be seen that the simple main effect of treatment was not significant at the time point t1 (p = 1). It becomes significant at t2 (p = 0.036) and t3 (p = 0.00051).<\/p>\n<p>Pairwise comparisons show that the mean self-esteem score was significantly different between ctr and Diet group at t2 (p = 0.12) and t3 (p = 0.00017) but not at t1 (p = 0.55).<\/p>\n<\/div>\n<p><strong>Effect of time<\/strong>. Note that, it\u2019s also possible to perform the same analysis for the <code>time<\/code> variable at each level of <code>treatment<\/code>. You don\u2019t necessarily need to do this analysis.<\/p>\n<p>The R code:<\/p>\n<pre class=\"r\"><code># Effect of time at each level of treatment\r\none.way2 &lt;- selfesteem2 %&gt;%\r\n  group_by(treatment) %&gt;%\r\n  anova_test(dv = score, wid = id, within = time) %&gt;%\r\n  get_anova_table() %&gt;%\r\n  adjust_pvalue(method = \"bonferroni\")\r\n# Pairwise comparisons between time points\r\npwc2 &lt;- selfesteem2 %&gt;%\r\n  group_by(treatment) %&gt;%\r\n  pairwise_t_test(\r\n    score ~ time, paired = TRUE,\r\n    p.adjust.method = \"bonferroni\"\r\n    )\r\npwc2<\/code><\/pre>\n<div class=\"success\">\n<p>After executing the R code above, you can see that the effect of <code>time<\/code> is significant only for the control trial, F(2, 22) = 39.7, p &lt; 0.0001. Pairwise comparisons show that all comparisons between time points were statistically significant for control trial.<\/p>\n<\/div>\n<\/div>\n<div id=\"procedure-for-non-significant-two-way-interaction\" class=\"section level4\">\n<h4>Procedure for non-significant two-way interaction<\/h4>\n<p>If the interaction is not significant, you need to interpret the main effects for each of the two variables: <code>treatment<\/code> and <code>time<\/code>. A significant main effect can be followed up with pairwise comparisons.<\/p>\n<div class=\"success\">\n<p>In our example (see ANOVA table in <code>res.aov<\/code>), there was a statistically significant main effects of treatment (F(1, 11) = 15.5, p = 0.002) and time (F(2, 22) = 27.4, p &lt; 0.0001) on the self-esteem score.<\/p>\n<\/div>\n<p>Pairwise paired t-test comparisons:<\/p>\n<pre class=\"r\"><code># comparisons for treatment variable\r\nselfesteem2 %&gt;%\r\n  pairwise_t_test(\r\n    score ~ treatment, paired = TRUE, \r\n    p.adjust.method = \"bonferroni\"\r\n    )\r\n# comparisons for time variable\r\nselfesteem2 %&gt;%\r\n  pairwise_t_test(\r\n    score ~ time, paired = TRUE, \r\n    p.adjust.method = \"bonferroni\"\r\n    )<\/code><\/pre>\n<div class=\"success\">\n<p>All pairwise comparisons are significant.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div id=\"report-1\" class=\"section level3\">\n<h3>Report<\/h3>\n<p>We could report the result as follow:<\/p>\n<p>A two-way repeated measures ANOVA was performed to evaluate the effect of different diet treatments over time on self-esteem score.<\/p>\n<p>There was a statistically significant interaction between <code>treatment<\/code> and <code>time<\/code> on self-esteem score, F(2, 22) = 30.4, p &lt; 0.0001. Therefore, the effect of <code>treatment<\/code> variable was analyzed at each <code>time<\/code> point. P-values were adjusted using the Bonferroni multiple testing correction method. The effect of treatment was significant at t2 (p = 0.036) and t3 (p = 0.00051) but not at the time point t1 (p = 1).<\/p>\n<p>Pairwise comparisons, using paired t-test, show that the mean self-esteem score was significantly different between ctr and Diet trial at time points t2 (p = 0.012) and t3 (p = 0.00017) but not at t1 (p = 0.55).<\/p>\n<pre class=\"r\"><code># Visualization: box plots with p-values\r\npwc &lt;- pwc %&gt;% add_xy_position(x = \"time\")\r\nbxp + \r\n  stat_pvalue_manual(pwc, tip.length = 0, hide.ns = TRUE) +\r\n  labs(\r\n    subtitle = get_test_label(res.aov, detailed = TRUE),\r\n    caption = get_pwc_label(pwc)\r\n  )<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/r-statistics-2-comparing-groups-means\/figures\/046-repeated-measure-anova-two-way-boxplots-with-p-values-1.png\" width=\"528\" \/><\/p>\n<\/div>\n<\/div>\n<div id=\"three-way\" class=\"section level2\">\n<h2>Three-way repeated measures ANOVA<\/h2>\n<div id=\"data-preparation-2\" class=\"section level3\">\n<h3>Data preparation<\/h3>\n<p>We\u2019ll use the <code>weightloss<\/code> dataset [datarium package]. In this study, a researcher wanted to assess the effects of Diet and Exercises on weight loss in 10 sedentary individuals.<\/p>\n<p>The participants were enrolled in four trials: (1) no diet and no exercises; (2) diet only; (3) exercises only; and (4) diet and exercises combined.<\/p>\n<p>Each participant performed all four trials. The order of the trials was counterbalanced and sufficient time was allowed between trials to allow any effects of previous trials to have dissipated.<\/p>\n<p>Each trial lasted nine weeks and the weight loss score was measured at the beginning (t1), at the midpoint (t2) and at the end (t3) of each trial.<\/p>\n<div class=\"block\">\n<p>Three-way repeated measures ANOVA can be performed in order to determine whether there is a significant interaction between diet, exercises and time on the weight loss score.<\/p>\n<\/div>\n<p>Load the data and show some random rows by groups:<\/p>\n<pre class=\"r\"><code># Wide format\r\nset.seed(123)\r\ndata(\"weightloss\", package = \"datarium\")\r\nweightloss %&gt;% sample_n_by(diet, exercises, size = 1)<\/code><\/pre>\n<pre><code>## # A tibble: 4 x 6\r\n##   id    diet  exercises    t1    t2    t3\r\n##   &lt;fct&gt; &lt;fct&gt; &lt;fct&gt;     &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt;\r\n## 1 4     no    no         11.1   9.5  11.1\r\n## 2 10    no    yes        10.2  11.8  17.4\r\n## 3 5     yes   no         11.6  13.4  13.9\r\n## 4 11    yes   yes        12.7  12.7  15.1<\/code><\/pre>\n<pre class=\"r\"><code># Gather the columns t1, t2 and t3 into long format.\r\n# Convert id and time into factor variables\r\nweightloss &lt;- weightloss %&gt;%\r\n  gather(key = \"time\", value = \"score\", t1, t2, t3) %&gt;%\r\n  convert_as_factor(id, time)\r\n# Inspect some random rows of the data by groups\r\nset.seed(123)\r\nweightloss %&gt;% sample_n_by(diet, exercises, time, size = 1)<\/code><\/pre>\n<pre><code>## # A tibble: 12 x 5\r\n##   id    diet  exercises time  score\r\n##   &lt;fct&gt; &lt;fct&gt; &lt;fct&gt;     &lt;fct&gt; &lt;dbl&gt;\r\n## 1 4     no    no        t1     11.1\r\n## 2 10    no    no        t2     10.7\r\n## 3 5     no    no        t3     12.3\r\n## 4 11    no    yes       t1     10.2\r\n## 5 12    no    yes       t2     13.2\r\n## 6 1     no    yes       t3     15.8\r\n## # \u2026 with 6 more rows<\/code><\/pre>\n<div class=\"success\">\n<p>In this example, the effect of the \u201ctime\u201d is our <strong>focal variable<\/strong>, that is our primary concern.<\/p>\n<p>It is thought that the effect of \u201ctime\u201d on the weight loss score will depend on two other factors, \u201cdiet\u201d and \u201cexercises\u201d, which are called <strong>moderator variables<\/strong>.<\/p>\n<\/div>\n<\/div>\n<div id=\"summary-statistics-2\" class=\"section level3\">\n<h3>Summary statistics<\/h3>\n<p>Group the data by <code>diet<\/code>, <code>exercises<\/code> and <code>time<\/code>, and then compute some summary statistics of the <code>score<\/code> variable: mean and sd (standard deviation)<\/p>\n<pre class=\"r\"><code>weightloss %&gt;%\r\n  group_by(diet, exercises, time) %&gt;%\r\n  get_summary_stats(score, type = \"mean_sd\")<\/code><\/pre>\n<pre><code>## # A tibble: 12 x 7\r\n##   diet  exercises time  variable     n  mean    sd\r\n##   &lt;fct&gt; &lt;fct&gt;     &lt;fct&gt; &lt;chr&gt;    &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt;\r\n## 1 no    no        t1    score       12  10.9 0.868\r\n## 2 no    no        t2    score       12  11.6 1.30 \r\n## 3 no    no        t3    score       12  11.4 0.935\r\n## 4 no    yes       t1    score       12  10.8 1.27 \r\n## 5 no    yes       t2    score       12  13.4 1.01 \r\n## 6 no    yes       t3    score       12  16.8 1.53 \r\n## # \u2026 with 6 more rows<\/code><\/pre>\n<\/div>\n<div id=\"visualization-2\" class=\"section level3\">\n<h3>Visualization<\/h3>\n<p>Create box plots:<\/p>\n<pre class=\"r\"><code>bxp &lt;- ggboxplot(\r\n  weightloss, x = \"exercises\", y = \"score\",\r\n  color = \"time\", palette = \"jco\",\r\n  facet.by = \"diet\", short.panel.labs = FALSE\r\n  )\r\nbxp<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/r-statistics-2-comparing-groups-means\/figures\/046-repeated-measure-anova-three-way-boxplots-1.png\" width=\"480\" \/><\/p>\n<\/div>\n<div id=\"check-assumptions-1\" class=\"section level3\">\n<h3>Check assumptions<\/h3>\n<div id=\"outliers-2\" class=\"section level4\">\n<h4>Outliers<\/h4>\n<pre class=\"r\"><code>weightloss %&gt;%\r\n  group_by(diet, exercises, time) %&gt;%\r\n  identify_outliers(score)<\/code><\/pre>\n<pre><code>## # A tibble: 5 x 7\r\n##   diet  exercises time  id    score is.outlier is.extreme\r\n##   &lt;fct&gt; &lt;fct&gt;     &lt;fct&gt; &lt;fct&gt; &lt;dbl&gt; &lt;lgl&gt;      &lt;lgl&gt;     \r\n## 1 no    no        t3    2      13.2 TRUE       FALSE     \r\n## 2 yes   no        t1    1      10.2 TRUE       FALSE     \r\n## 3 yes   no        t1    3      13.2 TRUE       FALSE     \r\n## 4 yes   no        t1    4      10.2 TRUE       FALSE     \r\n## 5 yes   no        t2    10     15.3 TRUE       FALSE<\/code><\/pre>\n<div class=\"success\">\n<p>There were no extreme outliers.<\/p>\n<\/div>\n<\/div>\n<div id=\"normality-assumption-2\" class=\"section level4\">\n<h4>Normality assumption<\/h4>\n<p>Compute Shapiro-Wilk test for each combinations of factor levels:<\/p>\n<pre class=\"r\"><code>weightloss %&gt;%\r\n  group_by(diet, exercises, time) %&gt;%\r\n  shapiro_test(score)<\/code><\/pre>\n<pre><code>## # A tibble: 12 x 6\r\n##   diet  exercises time  variable statistic     p\r\n##   &lt;fct&gt; &lt;fct&gt;     &lt;fct&gt; &lt;chr&gt;        &lt;dbl&gt; &lt;dbl&gt;\r\n## 1 no    no        t1    score        0.917 0.264\r\n## 2 no    no        t2    score        0.957 0.743\r\n## 3 no    no        t3    score        0.965 0.851\r\n## 4 no    yes       t1    score        0.922 0.306\r\n## 5 no    yes       t2    score        0.912 0.229\r\n## 6 no    yes       t3    score        0.953 0.674\r\n## # \u2026 with 6 more rows<\/code><\/pre>\n<div class=\"success\">\n<p>The weight loss score was normally distributed, as assessed by Shapiro-Wilk\u2019s test of normality (p &gt; .05).<\/p>\n<\/div>\n<p>Create QQ plot for each cell of design:<\/p>\n<pre class=\"r\"><code>ggqqplot(weightloss, \"score\", ggtheme = theme_bw()) +\r\n  facet_grid(diet + exercises ~ time, labeller = \"label_both\")<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/r-statistics-2-comparing-groups-means\/figures\/046-repeated-measure-anova-three-way-qq-plot-1.png\" width=\"576\" \/><\/p>\n<div class=\"success\">\n<p>From the plot above, as all the points fall approximately along the reference line, we can assume normality.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div id=\"computation-2\" class=\"section level3\">\n<h3>Computation<\/h3>\n<pre class=\"r\"><code>res.aov &lt;- anova_test(\r\n  data = weightloss, dv = score, wid = id,\r\n  within = c(diet, exercises, time)\r\n  )\r\nget_anova_table(res.aov)<\/code><\/pre>\n<pre><code>## ANOVA Table (type III tests)\r\n## \r\n##                Effect  DFn  DFd       F        p p&lt;.05   ges\r\n## 1                diet 1.00 11.0   6.021 3.20e-02     * 0.028\r\n## 2           exercises 1.00 11.0  58.928 9.65e-06     * 0.284\r\n## 3                time 2.00 22.0 110.942 3.22e-12     * 0.541\r\n## 4      diet:exercises 1.00 11.0  75.356 2.98e-06     * 0.157\r\n## 5           diet:time 1.38 15.2   0.603 5.01e-01       0.013\r\n## 6      exercises:time 2.00 22.0  20.826 8.41e-06     * 0.274\r\n## 7 diet:exercises:time 2.00 22.0  14.246 1.07e-04     * 0.147<\/code><\/pre>\n<div class=\"success\">\n<p>From the output above, it can be seen that there is a statistically significant three-way interactions between diet, exercises and time, F(2, 22) = 14.24, p = 0.00011.<\/p>\n<\/div>\n<div class=\"warning\">\n<p>Note that, if the three-way interaction is not statistically significant, you need to consult the two-way interactions in the output.<\/p>\n<p>In our example, there was a statistically significant two-way diet:exercises interaction (p &lt; 0.0001), and two-way exercises:time (p &lt; 0.0001). The two-way diet:time interaction was not statistically significant (p = 0.5).<\/p>\n<\/div>\n<\/div>\n<div id=\"post-hoc-tests-2\" class=\"section level3\">\n<h3>Post-hoc tests<\/h3>\n<p>If there is a significant three-way interaction effect, you can decompose it into:<\/p>\n<ul>\n<li><strong>Simple two-way interaction<\/strong>: run two-way interaction at each level of third variable,<\/li>\n<li><strong>Simple simple main effect<\/strong>: run one-way model at each level of second variable, and<\/li>\n<li><strong>simple simple pairwise comparisons<\/strong>: run pairwise or other post-hoc comparisons if necessary.<\/li>\n<\/ul>\n<div id=\"compute-simple-two-way-interaction\" class=\"section level4\">\n<h4>Compute simple two-way interaction<\/h4>\n<p>You are free to decide which two variables will form the simple two-way interactions and which variable will act as the third (moderator) variable. In the following R code, we have considered the simple two-way interaction of <code>exercises*time<\/code> at each level of <code>diet<\/code>.<\/p>\n<p>Group the data by <code>diet<\/code> and analyze the simple two-way interaction between <code>exercises<\/code> and <code>time<\/code>:<\/p>\n<pre class=\"r\"><code># Two-way ANOVA at each diet level\r\ntwo.way &lt;- weightloss %&gt;%\r\n  group_by(diet) %&gt;%\r\n  anova_test(dv = score, wid = id, within = c(exercises, time))\r\ntwo.way<\/code><\/pre>\n<pre><code>## # A tibble: 2 x 2\r\n##   diet  anova     \r\n##   &lt;fct&gt; &lt;list&gt;    \r\n## 1 no    &lt;anov_tst&gt;\r\n## 2 yes   &lt;anov_tst&gt;<\/code><\/pre>\n<pre class=\"r\"><code># Extract anova table\r\nget_anova_table(two.way)<\/code><\/pre>\n<pre><code>## # A tibble: 6 x 8\r\n##   diet  Effect           DFn   DFd     F        p `p&lt;.05`   ges\r\n##   &lt;fct&gt; &lt;chr&gt;          &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt;    &lt;dbl&gt; &lt;chr&gt;   &lt;dbl&gt;\r\n## 1 no    exercises          1    11 72.8  3.53e- 6 *       0.526\r\n## 2 no    time               2    22 71.7  2.32e-10 *       0.587\r\n## 3 no    exercises:time     2    22 28.9  6.92e- 7 *       0.504\r\n## 4 yes   exercises          1    11 13.4  4.00e- 3 *       0.038\r\n## 5 yes   time               2    22 20.5  9.30e- 6 *       0.49 \r\n## 6 yes   exercises:time     2    22  2.57 9.90e- 2 \"\"      0.06<\/code><\/pre>\n<div class=\"success\">\n<p>There was a statistically significant simple two-way interaction between exercises and time for \u201cdiet no\u201d trial, F(2, 22) = 28.9, p &lt; 0.0001, but not for \u201cdiet yes\u201d trial, F(2, 22) = 2.6, p = 0.099.<\/p>\n<\/div>\n<div class=\"warning\">\n<p>Note that, it\u2019s recommended to adjust the p-value for multiple testing. One common approach is to apply a Bonferroni adjustment to downgrade the level at which you declare statistical significance.<\/p>\n<p>This can be done by dividing the current level you declare statistical significance at (i.e., p &lt; 0.05) by the number of simple two-way interaction you are computing (i.e., 2).<\/p>\n<p>Thus, you only declare a two-way interaction as statistically significant when p &lt; 0.025 (i.e., p &lt; 0.05\/2). Applying this to our current example, we would still make the same conclusions.<\/p>\n<\/div>\n<\/div>\n<div id=\"compute-simple-simple-main-effect\" class=\"section level4\">\n<h4>Compute simple simple main effect<\/h4>\n<p>A statistically significant simple two-way interaction can be followed up with <strong>simple simple main effects<\/strong>.<\/p>\n<p>In our example, you could therefore investigate the effect of <code>time<\/code> on the weight loss <code>score<\/code> at every level of <code>exercises<\/code> or investigate the effect of <code>exercises<\/code> at every level of <code>time<\/code>.<\/p>\n<div class=\"warning\">\n<p>You will only need to consider the result of the simple simple main effect analyses for the \u201cdiet no\u201d trial as this was the only simple two-way interaction that was statistically significant (see previous section).<\/p>\n<\/div>\n<p>Group the data by <code>diet<\/code> and <code>exercises<\/code>, and analyze the simple main effect of <code>time<\/code>. The Bonferroni adjustment will be considered leading to statistical significance being accepted at the p &lt; 0.025 level (that is 0.05 divided by the number of tests (here 2) considered for \u201cdiet:no\u201d trial.<\/p>\n<pre class=\"r\"><code># Effect of time at each diet X exercises cells\r\ntime.effect &lt;- weightloss %&gt;%\r\n  group_by(diet, exercises) %&gt;%\r\n  anova_test(dv = score, wid = id, within = time)\r\ntime.effect<\/code><\/pre>\n<pre><code>## # A tibble: 4 x 3\r\n##   diet  exercises anova     \r\n##   &lt;fct&gt; &lt;fct&gt;     &lt;list&gt;    \r\n## 1 no    no        &lt;anov_tst&gt;\r\n## 2 no    yes       &lt;anov_tst&gt;\r\n## 3 yes   no        &lt;anov_tst&gt;\r\n## 4 yes   yes       &lt;anov_tst&gt;<\/code><\/pre>\n<pre class=\"r\"><code># Extract anova table\r\nget_anova_table(time.effect) %&gt;%\r\n  filter(diet == \"no\")<\/code><\/pre>\n<pre><code>## # A tibble: 2 x 9\r\n##   diet  exercises Effect   DFn   DFd     F        p `p&lt;.05`   ges\r\n##   &lt;fct&gt; &lt;fct&gt;     &lt;chr&gt;  &lt;dbl&gt; &lt;dbl&gt; &lt;dbl&gt;    &lt;dbl&gt; &lt;chr&gt;   &lt;dbl&gt;\r\n## 1 no    no        time       2    22  1.32 2.86e- 1 \"\"      0.075\r\n## 2 no    yes       time       2    22 78.8  9.30e-11 *       0.801<\/code><\/pre>\n<div class=\"success\">\n<p>There was a statistically significant simple simple main effect of time on weight loss score for \u201cdiet:no,exercises:yes\u201d group (p &lt; 0.0001), but not for when neither diet nor exercises was performed (p = 0.286).<\/p>\n<\/div>\n<\/div>\n<div id=\"compute-simple-simple-comparisons\" class=\"section level4\">\n<h4>Compute simple simple comparisons<\/h4>\n<p>A statistically significant simple simple main effect can be followed up by <strong>multiple pairwise comparisons<\/strong> to determine which group means are different.<\/p>\n<p>Group the data by <code>diet<\/code> and <code>exercises<\/code>, and perform pairwise comparisons between <code>time<\/code> points with Bonferroni adjustment:<\/p>\n<pre class=\"r\"><code># Pairwise comparisons\r\npwc &lt;- weightloss %&gt;%\r\n  group_by(diet, exercises) %&gt;%\r\n  pairwise_t_test(score ~ time, paired = TRUE, p.adjust.method = \"bonferroni\") %&gt;%\r\n  select(-df, -statistic) # Remove details\r\n# Show comparison results for \"diet:no,exercises:yes\" groups\r\npwc %&gt;% filter(diet == \"no\", exercises == \"yes\") %&gt;%\r\n  select(-p)     # remove p columns<\/code><\/pre>\n<pre><code>## # A tibble: 3 x 9\r\n##   diet  exercises .y.   group1 group2    n1    n2        p.adj p.adj.signif\r\n##   &lt;fct&gt; &lt;fct&gt;     &lt;chr&gt; &lt;chr&gt;  &lt;chr&gt;  &lt;int&gt; &lt;int&gt;        &lt;dbl&gt; &lt;chr&gt;       \r\n## 1 no    yes       score t1     t2        12    12 0.000741     ***         \r\n## 2 no    yes       score t1     t3        12    12 0.0000000121 ****        \r\n## 3 no    yes       score t2     t3        12    12 0.000257     ***<\/code><\/pre>\n<p>In the pairwise comparisons table above, we are interested only in the simple simple comparisons for \u201cdiet:no,exercises:yes\u201d groups. In our example, there are three possible combinations of group differences. We could report the pairwise comparison results as follow.<\/p>\n<div class=\"success\">\n<p>All simple simple pairwise comparisons were run between the different time points for \u201cdiet:no,exercises:yes\u201d trial. The Bonferroni adjustment was applied. The mean weight loss score was significantly different in all time point comparisons when exercises are performed (p &lt; 0.05).<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div id=\"report-2\" class=\"section level3\">\n<h3>Report<\/h3>\n<p>A three-way repeated measures ANOVA was performed to evaluate the effects of diet, exercises and time on weight loss. There was a statistically significant three-way interaction between diet, exercises and time, F(2, 22) = 14.2, p = 0.00011.<\/p>\n<p>For the simple two-way interactions and simple simple main effects analyses, a Bonferroni adjustment was applied leading to statistical significance being accepted at the p &lt; 0.025 level.<\/p>\n<p>There was a statistically significant simple two-way interaction between exercises and time for \u201cdiet no\u201d trial, F(2, 22) = 28.9, p &lt; 0.0001, but not for \u201cdiet yes\u201d\" trial, F(2, 22) = 2.6, p = 0.099.<\/p>\n<p>There was a statistically significant simple simple main effect of time on weight loss score for \u201cdiet:no,exercises:yes\u201d trial (p &lt; 0.0001), but not for when neither diet nor exercises was performed (p = 0.286).<\/p>\n<p>All simple simple pairwise comparisons were run between the different time points for \u201cdiet:no,exercises:yes\u201d trial with a Bonferroni adjustment applied. The mean weight loss score was significantly different in all time point comparisons when exercises are performed (p &lt; 0.05).<\/p>\n<pre class=\"r\"><code># Visualization: box plots with p-values\r\npwc &lt;- pwc %&gt;% add_xy_position(x = \"exercises\")\r\npwc.filtered &lt;- pwc %&gt;% \r\n  filter(diet == \"no\", exercises == \"yes\")\r\nbxp + \r\n  stat_pvalue_manual(pwc.filtered, tip.length = 0, hide.ns = TRUE) +\r\n  labs(\r\n    subtitle = get_test_label(res.aov, detailed = TRUE),\r\n    caption = get_pwc_label(pwc)\r\n  )<\/code><\/pre>\n<p><img decoding=\"async\" src=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/dn-tutorials\/r-statistics-2-comparing-groups-means\/figures\/046-repeated-measure-anova-three-way-boxplot-with-p-values-1.png\" width=\"576\" \/><\/p>\n<\/div>\n<\/div>\n<div id=\"summary\" class=\"section level2\">\n<h2>Summary<\/h2>\n<p>This chapter describes how to compute, interpret and report repeated measures ANOVA in R. We also explain the assumptions made by repeated measures ANOVA tests and provide practical examples of R codes to check whether the test assumptions are met.<\/p>\n<\/div>\n<\/div>\n<p><!--end rdoc--><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The repeated-measures ANOVA is used for analyzing data where same subjects are measured more than once. This chapter describes the different types of repeated measures ANOVA, including:<\/p>\n<p>1) One-way repeated measures ANOVA, an extension of the paired-samples t-test for comparing the means of three or more levels of a within-subjects variable.<br \/>\n2) two-way repeated measures ANOVA used to evaluate simultaneously the effect of two within-subject factors on a continuous outcome variable.<br \/>\n3) three-way repeated measures ANOVA used to evaluate simultaneously the effect of three within-subject factors on a continuous outcome variable. <\/p>\n","protected":false},"author":1,"featured_media":9104,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","class_list":["post-10873","dt_lessons","type-dt_lessons","status-publish","has-post-thumbnail","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Repeated Measures ANOVA in R: The Ultimate Guide - Datanovia<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Repeated Measures ANOVA in R: The Ultimate Guide - Datanovia\" \/>\n<meta property=\"og:description\" content=\"The repeated-measures ANOVA is used for analyzing data where same subjects are measured more than once. This chapter describes the different types of repeated measures ANOVA, including:    1) One-way repeated measures ANOVA, an extension of the paired-samples t-test for comparing the means of three or more levels of a within-subjects variable. 2) two-way repeated measures ANOVA used to evaluate simultaneously the effect of two within-subject factors on a continuous outcome variable.  3) three-way repeated measures ANOVA used to evaluate simultaneously the effect of three within-subject factors on a continuous outcome variable.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/\" \/>\n<meta property=\"og:site_name\" content=\"Datanovia\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2019\/05\/X25660197_558627331143882_7150400490780998089_n.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"512\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"25 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/\",\"url\":\"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/\",\"name\":\"Repeated Measures ANOVA in R: The Ultimate Guide - Datanovia\",\"isPartOf\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2019\/05\/X25660197_558627331143882_7150400490780998089_n.jpg\",\"datePublished\":\"2019-11-29T06:25:52+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/#primaryimage\",\"url\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2019\/05\/X25660197_558627331143882_7150400490780998089_n.jpg\",\"contentUrl\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2019\/05\/X25660197_558627331143882_7150400490780998089_n.jpg\",\"width\":1024,\"height\":512},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.datanovia.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Lessons\",\"item\":\"https:\/\/www.datanovia.com\/en\/lessons\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Repeated Measures ANOVA in R\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.datanovia.com\/en\/#website\",\"url\":\"https:\/\/www.datanovia.com\/en\/\",\"name\":\"Datanovia\",\"description\":\"Data Mining and Statistics for Decision Support\",\"publisher\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.datanovia.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.datanovia.com\/en\/#organization\",\"name\":\"Datanovia\",\"url\":\"https:\/\/www.datanovia.com\/en\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png\",\"contentUrl\":\"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png\",\"width\":98,\"height\":99,\"caption\":\"Datanovia\"},\"image\":{\"@id\":\"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Repeated Measures ANOVA in R: The Ultimate Guide - Datanovia","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/","og_locale":"en_US","og_type":"article","og_title":"Repeated Measures ANOVA in R: The Ultimate Guide - Datanovia","og_description":"The repeated-measures ANOVA is used for analyzing data where same subjects are measured more than once. This chapter describes the different types of repeated measures ANOVA, including:    1) One-way repeated measures ANOVA, an extension of the paired-samples t-test for comparing the means of three or more levels of a within-subjects variable. 2) two-way repeated measures ANOVA used to evaluate simultaneously the effect of two within-subject factors on a continuous outcome variable.  3) three-way repeated measures ANOVA used to evaluate simultaneously the effect of three within-subject factors on a continuous outcome variable.","og_url":"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/","og_site_name":"Datanovia","og_image":[{"width":1024,"height":512,"url":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2019\/05\/X25660197_558627331143882_7150400490780998089_n.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"25 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/","url":"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/","name":"Repeated Measures ANOVA in R: The Ultimate Guide - Datanovia","isPartOf":{"@id":"https:\/\/www.datanovia.com\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/#primaryimage"},"image":{"@id":"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/#primaryimage"},"thumbnailUrl":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2019\/05\/X25660197_558627331143882_7150400490780998089_n.jpg","datePublished":"2019-11-29T06:25:52+00:00","breadcrumb":{"@id":"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/#primaryimage","url":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2019\/05\/X25660197_558627331143882_7150400490780998089_n.jpg","contentUrl":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2019\/05\/X25660197_558627331143882_7150400490780998089_n.jpg","width":1024,"height":512},{"@type":"BreadcrumbList","@id":"https:\/\/www.datanovia.com\/en\/lessons\/repeated-measures-anova-in-r\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.datanovia.com\/en\/"},{"@type":"ListItem","position":2,"name":"Lessons","item":"https:\/\/www.datanovia.com\/en\/lessons\/"},{"@type":"ListItem","position":3,"name":"Repeated Measures ANOVA in R"}]},{"@type":"WebSite","@id":"https:\/\/www.datanovia.com\/en\/#website","url":"https:\/\/www.datanovia.com\/en\/","name":"Datanovia","description":"Data Mining and Statistics for Decision Support","publisher":{"@id":"https:\/\/www.datanovia.com\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.datanovia.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.datanovia.com\/en\/#organization","name":"Datanovia","url":"https:\/\/www.datanovia.com\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/","url":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png","contentUrl":"https:\/\/www.datanovia.com\/en\/wp-content\/uploads\/2018\/09\/datanovia-logo.png","width":98,"height":99,"caption":"Datanovia"},"image":{"@id":"https:\/\/www.datanovia.com\/en\/#\/schema\/logo\/image\/"}}]}},"multi-rating":{"mr_rating_results":[]},"_links":{"self":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/dt_lessons\/10873","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/dt_lessons"}],"about":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/types\/dt_lessons"}],"author":[{"embeddable":true,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/comments?post=10873"}],"version-history":[{"count":0,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/dt_lessons\/10873\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/media\/9104"}],"wp:attachment":[{"href":"https:\/\/www.datanovia.com\/en\/wp-json\/wp\/v2\/media?parent=10873"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}