Test whether a yes/no outcome differs across three or more related conditions — McNemar extended to repeated measures, with pairwise post-hoc
Learn to run Cochran’s Q test in R — the extension of McNemar’s test to three or more paired proportions, for a dichotomous outcome measured on the same subjects across several conditions or times. Run it with rstatix cochran_qtest() and follow up with pairwise McNemar comparisons.
rstatix::cochran_qtest() on long-format data: outcome ~ condition | id.pairwise_mcnemar_test()) to see which conditions differ.Cochran’s Q test is the categorical counterpart of the repeated-measures ANOVA for a binary outcome: it compares a yes/no (success/failure) proportion across three or more related groups — the same subjects measured under several conditions, treatments, or time points. It generalises McNemar’s test (which handles two) to many.
The scenario here: 73 subjects each attempt three tasks, each scored success/failure — do the success rates differ across the tasks? This lesson runs it with rstatix on the taskachievment data.
Cochran’s Q hypotheses. H₀: the proportion of success is the same across all conditions. Hₐ: at least one condition’s proportion differs.
We use the taskachievment dataset from datarium — 73 participants, each with a success (1) / failure (0) outcome on Task1, Task2, Task3. Cochran’s Q needs long format (one row per participant × task), built with base R:
library(rstatix)
data("taskachievment", package = "datarium")
n <- nrow(taskachievment)
long <- data.frame(
id = factor(rep(taskachievment$participant, 3)),
task = factor(rep(c("Task1", "Task2", "Task3"), each = n)),
outcome = c(taskachievment$Task1, taskachievment$Task2, taskachievment$Task3)
)
head(long, 3) id task outcome
1 1 Task1 0
2 2 Task1 0
3 3 Task1 1Summarize the success rate per task (the mean of a 0/1 outcome is the proportion of successes):
library(rstatix)
data("taskachievment", package = "datarium")
n <- nrow(taskachievment)
long <- data.frame(
id = factor(rep(taskachievment$participant, 3)),
task = factor(rep(c("Task1", "Task2", "Task3"), each = n)),
outcome = c(taskachievment$Task1, taskachievment$Task2, taskachievment$Task3)
)
aggregate(outcome ~ task, data = long, FUN = mean) task outcome
1 Task1 0.2602740
2 Task2 0.4383562
3 Task3 0.6164384cochran_qtest() takes the long data with an outcome ~ condition | id formula:
library(rstatix)
data("taskachievment", package = "datarium")
n <- nrow(taskachievment)
long <- data.frame(
id = factor(rep(taskachievment$participant, 3)),
task = factor(rep(c("Task1", "Task2", "Task3"), each = n)),
outcome = c(taskachievment$Task1, taskachievment$Task2, taskachievment$Task3)
)
long %>% cochran_qtest(outcome ~ task | id)# A tibble: 1 × 6
.y. n statistic df p method
* <chr> <int> <dbl> <dbl> <dbl> <chr>
1 outcome 73 39 2 0.00000000340 Cochran's Q testThe success proportions differ significantly across the three tasks, Q(2) = 39, p < 0.0001.
Follow a significant Cochran’s Q with pairwise McNemar tests (each pair is a paired 2×2), Bonferroni- adjusted:
library(rstatix)
data("taskachievment", package = "datarium")
n <- nrow(taskachievment)
long <- data.frame(
id = factor(rep(taskachievment$participant, 3)),
task = factor(rep(c("Task1", "Task2", "Task3"), each = n)),
outcome = c(taskachievment$Task1, taskachievment$Task2, taskachievment$Task3)
)
long %>% pairwise_mcnemar_test(outcome ~ task | id)# A tibble: 3 × 6
group1 group2 p p.adj p.adj.signif method
* <chr> <chr> <dbl> <dbl> <chr> <chr>
1 Task1 Task2 0.000874 0.00262 ** McNemar test
2 Task1 Task3 0.000000944 0.00000283 **** McNemar test
3 Task2 Task3 0.000874 0.00262 ** McNemar testThe pairwise comparisons show which specific tasks have different success rates.
library(rstatix)
library(ggpubr)
data("taskachievment", package = "datarium")
n <- nrow(taskachievment)
long <- data.frame(
id = factor(rep(taskachievment$participant, 3)),
task = factor(rep(c("Task1", "Task2", "Task3"), each = n)),
outcome = c(taskachievment$Task1, taskachievment$Task2, taskachievment$Task3)
)
prop.df <- aggregate(outcome ~ task, data = long, FUN = mean)
stat.test <- cochran_qtest(long, outcome ~ task | id)
# Post-hoc pairwise McNemar comparisons, placed above the proportion bars
pwc <- pairwise_mcnemar_test(long, outcome ~ task | id)
pwc$y.position <- c(0.78, 0.90, 0.84)
ggbarplot(prop.df, x = "task", y = "outcome", fill = "task", palette = "jco",
ylab = "Proportion of success", xlab = "Task") +
stat_pvalue_manual(pwc, label = "p.adj.signif", tip.length = 0.01) +
labs(subtitle = get_test_label(stat.test, detailed = TRUE),
caption = get_pwc_label(pwc))
A Cochran’s Q test showed that the proportion of success differed significantly across the three tasks, Q(2) = 39, p < 0.001. Pairwise McNemar tests (Bonferroni-adjusted) identified which tasks differed.
Cochran’s Q compares the column (condition) success totals to what equal proportions would give, adjusting for each subject’s overall success rate: \(Q = \dfrac{k(k-1)\sum_j (C_j - \bar{C})^2}{k\sum_i R_i - \sum_i R_i^2}\), where \(C_j\) is the number of successes in condition \(j\), \(R_i\) is subject \(i\)’s total successes, and \(k\) is the number of conditions. \(Q\) follows a chi-square distribution with \(k - 1\) degrees of freedom. With \(k = 2\) it reduces to the (uncorrected) McNemar statistic.
Run Cochran’s Q on the task data, then change the post-hoc step. The sandbox boots on first Run.
Ask Prova “I have a yes/no outcome measured on the same subjects under several conditions — test whether the success rate differs with Cochran’s Q” — it answers with rstatix code you can run on your own data. The runtime is the judge. Ask Prova →
Your data is in wide format. cochran_qtest() needs long format — one row per subject × condition, with an id, the condition factor and the 0/1 outcome. Reshape the wide columns first.
Your outcome isn’t binary. Cochran’s Q is for a dichotomous (success/failure, yes/no) outcome. For a continuous repeated measure use repeated-measures ANOVA or Friedman.
You have only two conditions. Then use McNemar’s test directly — Cochran’s Q gives the same answer but McNemar is the named two-group test.
Cochran’s Q tests whether a dichotomous (yes/no) outcome has the same proportion of “success” across three or more related groups — the same subjects measured under several conditions or time points. It is the extension of McNemar’s test (two groups) to many, and the binary analogue of repeated-measures ANOVA.
McNemar’s test compares two paired proportions; Cochran’s Q compares three or more. With exactly two conditions they give the same statistic (Q equals the uncorrected McNemar chi-square). Use McNemar for two, Cochran’s Q for more.
Pairwise McNemar tests (pairwise_mcnemar_test()), with a multiple-comparison correction such as Bonferroni — each pair of conditions is a paired 2×2 table, so McNemar is the natural pairwise comparison.
Fill the blank with task. For question 2, recall the paired two-group categorical test.
long |> cochran_qtest(outcome ~ task | id) #> Q(2) = 39, p < 0.0001 — rates differ.For question 2: run pairwise McNemar tests (pairwise_mcnemar_test(), Bonferroni-adjusted). Each pair of tasks is two paired proportions on the same subjects, so McNemar is the correct pairwise comparison — an ordinary chi-square would ignore the pairing.
You can now run Cochran’s Q test in R: reshape the repeated binary measurements to long format, run cochran_qtest(outcome ~ condition | id), and follow a significant result with pairwise McNemar tests. It is the test for comparing a yes/no outcome across three or more related conditions — McNemar generalised to repeated measures.
@online{2026,
author = {},
title = {Cochran’s {Q} {Test} in {R:} {Compare} {Three} or {More}
{Paired} {Proportions}},
date = {2026-06-23},
url = {https://www.datanovia.com/learn/biostatistics/categorical/cochran-q-test-in-r},
langid = {en}
}