Comparing two groups on a numeric variable? Use a t-test (or Mann-Whitney if the data are not normal). Three or more groups? ANOVA (or Kruskal-Wallis). Two numeric variables? Correlation or regression. Two categorical variables? Chi-square. The table below covers the details.
The four questions
- What do you want to do? Compare groups, measure a relationship, predict a value, or analyse counts.
- What type is your outcome? Numeric (scores, times, amounts), ordinal (ranks, 1–5 ratings) or categorical (yes/no, brand, region).
- How many groups, and are they independent? Different people in each group (independent), or the same people measured several times (paired, repeated).
- Are the data roughly normal? Check each group with a Shapiro-Wilk test and a box plot. With large groups (30 or more), moderate departures matter less.
The decision table
| Goal | Design | Normal data | Not normal / ordinal |
|---|---|---|---|
| Compare 2 groups | Independent | Welch t-test | Mann-Whitney U |
| Compare 2 measurements | Same people (paired) | Paired t-test | Wilcoxon signed-rank |
| Compare 3+ groups | Independent | One-way ANOVA (Welch ANOVA if variances differ) | Kruskal-Wallis |
| Compare 3+ measurements | Same people (repeated) | Repeated-measures ANOVA | Friedman test |
| Two factors at once | Independent | Two-way ANOVA | (transform, or analyse factors separately) |
| Relationship between 2 numeric variables | One sample | Pearson correlation | Spearman correlation |
| Predict a numeric value | One or more predictors | Linear regression | (check residuals, consider a transformation) |
| Relationship between 2 categorical variables | Counts | Chi-square test; Fisher's exact test for small 2 × 2 tables | |
| Compare 2 conversion rates (A/B test) | Counts | Two-proportion z-test or chi-square | |
| Compare a mean with a target value | One sample | One-sample t-test | Wilcoxon signed-rank (against the target) |
| Check a questionnaire's reliability | Several items | Cronbach's alpha | |
Select your data, click Run: full results table, assumption checks, a plain-language interpretation and an APA line, free in Google Sheets.
Three examples
“Do students taught with three methods get different scores?”
Goal: compare. Outcome: numeric score. Three independent groups. Normal data → one-way ANOVA, followed by a post-hoc test (Tukey) if it is significant.
“Did the workshop improve scores?”
Goal: compare. Same students before and after → paired. Normal differences → paired t-test. If the differences are clearly skewed → Wilcoxon signed-rank.
“Is satisfaction (1–5) related to the store location?”
Outcome: a single 1–5 rating, which is ordinal. If you compare the ratings across several locations, use Kruskal-Wallis. If you treat the rating as categories (satisfied / not satisfied) and count, use a chi-square test.
Common mistakes when choosing
- Running several t-tests instead of an ANOVA. With three groups and three t-tests, the chance of at least one false positive rises from 5% to about 14%.
- Treating paired data as independent. A before/after design analysed with an independent t-test can lose a lot of power.
- Using a chi-square test on percentages or means. It needs raw counts.
- Skipping the assumption checks. A test is only as good as its assumptions.
Let the Assistant choose
ExplainStats has a which-test Assistant in Google Sheets: say what you want to do and select your data. It checks normality, outliers and equal variances, picks the matching test from this table, runs it and explains why it chose it.
Frequently asked questions
What is the difference between parametric and non-parametric tests?
Parametric tests (t-test, ANOVA, Pearson correlation) assume roughly normal data; the t-test and ANOVA compare means. Non-parametric tests (Mann-Whitney, Kruskal-Wallis, Spearman) work on ranks and make fewer assumptions, at the cost of a little power when the data are normal.
Can I use a t-test with a small sample?
Yes, if the data in each group are roughly normal and have no extreme outliers. With fewer than about 15 values per group, check normality first; if it clearly fails, use a non-parametric test.
Is a Likert scale item ordinal or numeric?
A single Likert item (1 to 5) is ordinal, so non-parametric tests are the safer choice. The average of several items forming a scale is often treated as numeric.
What if I have more than one independent variable?
For two categorical factors and a numeric outcome, use a two-way ANOVA. For several predictors of a numeric outcome, use multiple regression.
Select your data, click Run: full results table, assumption checks, a plain-language interpretation and an APA line, free in Google Sheets.