Tukey HSD vs Games-Howell: Which Post-Hoc Test Should You Use?

tukey hsd vs games howell comes up regularly when researchers reach the post-hoc stage of a dissertation. The practical distinction is fairly simple: Tukey HSD is designed for pairwise comparisons when a common variance assumption is reasonable, while Games-Howell is designed for comparisons when equal variances are not assumed. The difficult part is deciding which situation actually applies to your data.

After 12 years of working with dissertation research and statistical analysis, I have seen many researchers choose a post-hoc test because it is the first option they recognise in SPSS. I would not recommend that approach. Your post-hoc procedure should follow the design, variance structure and omnibus test you selected.

Tukey HSD vs Games-Howell: The Quick Answer

Data situation	Recommended approach
Equal or reasonably similar variances	Tukey HSD
Equal variances but unequal group sizes	Tukey-Kramer
Unequal variances	Games-Howell
Unequal variances and unequal group sizes	Games-Howell
Only two independent groups	Usually consider a two-sample or Welch comparison rather than a multi-group post-hoc procedure

The reason for this distinction is that Tukey HSD was developed around a common variance framework, whereas Games-Howell does not assume equal variances. Current SciPy documentation makes the same distinction and applies Tukey-Kramer when group sizes are unequal under the equal-variance setting. (SciPy documentation)

The rule I want dissertation researchers to remember is this:

Equal variances → Tukey HSD or Tukey-Kramer
Unequal variances → Games-Howell

Do not turn the rule into “unequal sample sizes = Games-Howell”. Those are two different issues.

---

Why Do You Need a Post-Hoc Test After ANOVA?

A one-way ANOVA answers an overall question: are all group means equal, or is there evidence that at least one differs?

The F statistic is the test statistic used for this omnibus comparison. A significant F statistic tells you that the group means are not all the same, but it does not identify the specific pairs that differ.

Imagine that I compare dissertation satisfaction scores across three teaching formats:

Group	Mean
Online	68
Hybrid	73
Face-to-face	80

A significant ANOVA does not automatically tell me that Online differs from Hybrid, or that Hybrid differs from face-to-face. I need appropriate pairwise comparisons to answer those questions.

This is why post-hoc procedures exist. They allow multiple pairwise comparisons while controlling the family-wise error rate rather than treating every pair as an isolated test. NIST describes Tukey’s procedure as a simultaneous method for all pairwise mean differences. (NIST/SEMATECH)

That is also why I would not solve a three-group ANOVA by simply running three separate unadjusted t-tests.

---

What Is Tukey HSD?

Tukey HSD, or Tukey’s Honestly Significant Difference test, is a post-hoc procedure for comparing every pair of group means while controlling the family-wise error rate.

It is particularly appropriate when the groups can reasonably be treated as having a common variance and you want all pairwise comparisons.

Tukey’s method is based on the studentized range distribution. NIST describes it as a simultaneous procedure covering the complete set of pairwise comparisons. (NIST/SEMATECH)

Tukey HSD assumptions

The wider ANOVA assumptions still matter. In practice, I would assess:

• independence of observations
• reasonable distributional behaviour within groups
• variance homogeneity where the standard Tukey framework is used
• an appropriate one-way ANOVA design

Current statistical documentation also lists independent observations, within-group normality and equal finite variances among the assumptions associated with the standard equal-variance Tukey implementation. (SciPy documentation)

I would not treat anova assumptions as a checklist where every p-value automatically decides the analysis. Group sizes, distributions, standard deviations and the research design all provide useful context.

When I would choose Tukey HSD

I would normally choose Tukey when I have:

• a one-way ANOVA design
• reasonably similar group variances
• independent observations
• a genuine need to compare all pairs of groups

It is a familiar and defensible choice for many standard dissertation datasets.

---

What Is the Games-Howell Test?

Games-Howell is a post-hoc multiple-comparison procedure for pairwise mean comparisons when equal variances cannot reasonably be assumed.

The method was introduced by Paul Games and John Howell in their 1976 study of pairwise multiple-comparison procedures under unequal sample sizes and/or variances. (SciPy documentation)

Unlike traditional Tukey HSD, Games-Howell does not require a common variance assumption. This makes it useful when one group is much more variable than another, particularly when the group sizes are also different.

IBM’s current SPSS documentation places Games-Howell among the post-hoc procedures that do not assume equal variances. (IBM SPSS documentation)

When I would choose Games-Howell

I would consider Games-Howell when:

• the group variances differ materially
• sample sizes are unequal, especially alongside variance heterogeneity
• I need all pairwise comparisons
• a Welch-type omnibus approach is appropriate

Statistically, the important point is not simply that the sample sizes differ. It is that the equal-variance assumption is not credible.

---

Tukey HSD vs Games-Howell: The Technical Difference

The easiest way to understand the distinction is to look at what each test assumes about variability.

Equal variances

If the groups have reasonably similar variances, Tukey is a natural choice for all pairwise comparisons.

Unequal variances

If the amount of variability differs substantially between groups, Games-Howell is designed for that situation.

Unequal sample sizes

This is where I see a lot of dissertation confusion.

Unequal sample sizes do not automatically make Tukey inappropriate. The Tukey approach can be extended to unequal group sizes through the Tukey-Kramer method. NIST specifically describes Tukey-Kramer as the Tukey procedure for unequal sample sizes. (NIST/SEMATECH)

So this:

Group A = 100 participants
Group B = 80 participants
Group C = 40 participants

does not by itself mean that Games-Howell is required.

If the variances remain reasonably comparable, Tukey-Kramer can be appropriate.

By contrast, if Group C also has a variance several times larger than the others, the case for a heteroscedastic procedure such as Games-Howell becomes much stronger.

---

Tukey HSD vs Tukey-Kramer vs Games-Howell

These three should not be treated as interchangeable labels.

Procedure	Main situation
Tukey HSD	Equal variances and equal group sizes
Tukey-Kramer	Equal variances with unequal group sizes
Games-Howell	Unequal variances, including when group sizes differ

NIST notes that the ordinary Tukey formulation is conservative with unequal sample sizes and describes the unequal-n extension as Tukey-Kramer. (NIST/SEMATECH)

SciPy’s current implementation makes the same distinction directly: equal variance with equal sample sizes uses Tukey HSD, equal variance with unequal sample sizes uses Tukey-Kramer, and unequal variance uses Games-Howell. (SciPy documentation)

That makes the decision much easier:

Ask about variance first. Ask about sample size second.

---

What Does Levene’s Test Have to Do With the Decision?

Levene’s test is commonly used when assessing the homogeneity of variances.

A practical workflow is:

1. Examine your groups and descriptive statistics.
2. Check the variance assumption, including Levene’s test where appropriate.
3. Decide whether a standard ANOVA or Welch’s ANOVA better fits the variance structure.
4. Select the post-hoc procedure that matches that decision.
5. Interpret pairwise differences using adjusted p-values and confidence intervals.

A common shortcut says:

“Levene’s p < .05, therefore use Games-Howell.”

I think that is too mechanical.

A statistically significant Levene test is evidence against equal variances, but I would also look at the actual group standard deviations, sample sizes and distribution of the data. With very large samples, even modest variance differences can produce a small p-value.

So I treat Levene’s test as evidence within the assumption assessment, not as an automatic software switch.

---

Should You Use Standard ANOVA or Welch’s ANOVA?

This is closely connected to the post-hoc decision.

When the equal-variance assumption is reasonable, standard one-way ANOVA followed by Tukey or Tukey-Kramer is a common route.

When substantial variance heterogeneity is present, Welch’s ANOVA provides an alternative omnibus test that does not require the same equal-variance assumption. Games-Howell can then be used for the pairwise comparisons.

So my decision framework is:

Data structure → assumptions → omnibus test → post-hoc procedure → interpretation

Not:

Run ANOVA → select the familiar post-hoc test.

That distinction matters because the omnibus and pairwise stages should tell the same statistical story.

What if standard ANOVA is significant but Welch’s ANOVA is not?

Do not simply report the significant result because it is the one you prefer.

I would go back to the assumptions, sample sizes and magnitude of variance differences. If heterogeneity is substantial, the Welch result deserves serious consideration, and the methodology section should explain why the final approach was selected.

---

A Worked Dissertation Example

Consider an anonymised dissertation-style example examining whether academic satisfaction differs across three teaching formats.

Group	n	Mean	SD
Online	85	68.4	7.1
Hybrid	62	72.8	8.0
Face-to-face	31	79.6	14.8

Notice what stands out.

The group sizes are different, but more importantly, the face-to-face group has a much larger standard deviation.

Suppose the variance assessment also indicates meaningful heterogeneity.

In this situation, I would not select Games-Howell merely because 85 ≠ 62 ≠ 31. I would select it because the variance structure makes the standard equal-variance approach less suitable.

My analysis would therefore look something like:

Descriptive statistics → variance assessment → Welch’s ANOVA → Games-Howell pairwise comparisons

Suppose the Games-Howell output shows:

• Online vs Hybrid: not significant
• Online vs Face-to-face: significant
• Hybrid vs Face-to-face: significant

The conclusion would be that the face-to-face group differs significantly from both other groups, while the online and hybrid groups do not show a statistically significant difference.

That is a much more useful conclusion than saying:

“The ANOVA was significant.”

---

What If Tukey HSD and Games-Howell Give Different Results?

They can.

That is not automatically evidence that one of the calculations is wrong. The procedures make different assumptions and handle pairwise uncertainty differently.

This is also why I strongly disagree with a common dissertation habit: running both tests and choosing the one that gives the more convenient p-value.

Do not do that.

If your data support a heteroscedastic approach, report Games-Howell. If the equal-variance framework is appropriate, report Tukey or Tukey-Kramer.

The statistical method should be selected from the evidence, not from the preferred conclusion.

If an examiner asks why you selected Games-Howell, you should be able to answer in one sentence:

“The pairwise procedure was selected because the equal-variance assumption was not considered appropriate for the group structure.”

That is much stronger than:

“SPSS gave me Games-Howell.”

---

How to Interpret Tukey HSD Results

When I read a Tukey table, I focus on three things.

Mean difference

This tells you how far apart two group means are and gives the direction of the difference.

Adjusted p-value

This tells you whether that pair remains statistically significant after accounting for the multiple comparisons.

Confidence interval

This gives a range of plausible values for the population mean difference.

For a clear anova interpretation, I would report the F statistic, p-value, effect size, and relevant pairwise comparisons rather than reporting statistical significance alone.

For example:

Group C scored significantly higher than Group A, mean difference = 11.2, adjusted p < .01, 95% CI [4.1, 18.3].

That tells your reader much more than:

“Tukey was significant.”

Researchers who want to check the calculation can use Statssy’s Tukey HSD post-hoc calculator.

---

How to Interpret Games-Howell Results

The interpretation follows the same general logic.

Suppose the output contains:

Group A vs Group C, mean difference = -11.2, p = .003.

I would interpret this as:

Group C had a significantly higher mean than Group A, with an estimated mean difference of 11.2 points.

I would then inspect the confidence interval and the other pairwise comparisons.

For dissertation writing, name the procedure explicitly. Do not simply write “post-hoc analysis showed…”.

Write:

“Games-Howell post-hoc comparisons indicated that…”

That makes the analytical decision transparent.

You can also use Statssy’s Games-Howell test calculator as a calculation check.

---

How to Use a One-Way ANOVA Calculator

A one way ANOVA calculator is useful when you want to check an omnibus F statistic or perform a quick analysis.

But I would not use a calculator as a substitute for methodological judgement.

Before accepting its output, ask:

1. How many independent groups do I have?
2. Are the observations independent?
3. Are the group variances reasonably similar?
4. Are the group sizes very different?
5. Should I use standard ANOVA or Welch’s ANOVA?
6. Which post-hoc method follows from that decision?
7. What effect size should I report?

Statssy’s one-way ANOVA calculator can help with the omnibus calculation.

If the normality or variance assumptions are not appropriate for an ordinary ANOVA framework, the answer should not be “the calculator said so”. The analysis needs to match the data.

---

What Effect Size Should You Report?

This is one area where I think many dissertation articles stop too early.

A p-value tells you about statistical evidence. It does not tell you how large or practically meaningful the overall difference is.

For a one-way ANOVA, researchers commonly report an ANOVA effect size such as eta-squared (η²) or omega-squared (ω²), depending on the analytical framework and reporting convention.

For pairwise comparisons, report the mean difference and confidence interval alongside the adjusted p-value.

For example, I would rather see:

F(2, 177) = 8.42, p < .001, η² = .09.

followed by the relevant pairwise differences, than a results section that contains a long list of p-values with no indication of magnitude.

The effect size and confidence interval help the reader understand what the statistical significance actually means.

---

How to Run Tukey HSD and Games-Howell

The exact menu differs by software, but the statistical logic remains the same.

SPSS

IBM’s One-Way ANOVA procedure provides Tukey under equal-variance assumptions and Games-Howell among procedures that do not assume equal variances. In SPSS, the route is:

Analyze → Compare Means → One-Way ANOVA → Post Hoc

IBM’s documentation confirms that Games-Howell is available for unequal-variance situations. (IBM SPSS documentation)

R

R users can perform ANOVA and use appropriate packages or functions for Tukey and Games-Howell comparisons. The key point is to decide the statistical framework first, then select the post-hoc implementation.

Python

SciPy’s current tukey_hsd implementation supports standard Tukey HSD, Tukey-Kramer for unequal group sizes under equal variances, and Games-Howell when equal variances are not assumed. (SciPy documentation)

JASP

JASP can also be used for one-way ANOVA and post-hoc comparisons. Again, software selection is secondary to the statistical decision.

I would not write a dissertation methodology sentence such as:

“Games-Howell was selected because SPSS had it.”

The software performs the calculation. The researcher is responsible for the methodological choice.

---

Common Mistakes I See in Dissertations

After 12 years of working with dissertation analyses, these are the mistakes I see most often.

Choosing Tukey because it is familiar

A familiar test is not automatically the correct test.

Choosing Games-Howell because sample sizes differ

Unequal sample sizes alone do not prove unequal variances.

Treating Levene’s p-value as the entire decision

Look at the actual group variability and study design too.

Running multiple unadjusted t-tests

This can increase the family-wise Type I error rate.

Reporting only p-values

Include the mean difference and confidence interval where appropriate.

Ignoring the omnibus test

Post-hoc results should sit within a coherent ANOVA or Welch ANOVA framework.

Reporting statistical significance without effect size

A very small difference can be statistically significant in a large sample.

Testing several post-hoc procedures and selecting the favourite result

This is one of the practices I would actively discourage in a dissertation.

Choose your procedure based on the methodology, then report the result honestly.

---

How to Report Tukey HSD in a Dissertation

A straightforward example is:

A one-way ANOVA indicated a significant difference in satisfaction across the three groups, F(2, 177) = 8.42, p < .001. Tukey HSD post-hoc comparisons showed that Group C had significantly higher satisfaction than Groups A and B.

Where appropriate, add the pairwise mean differences, adjusted p-values and confidence intervals in the accompanying table.

This is also where the broader quality of your dissertation matters. Statistical output is only useful when it is connected clearly to the research question, methodology and discussion. My guide on organising a dissertation covers the broader structure around that process.

---

How to Report Games-Howell in a Dissertation

For an unequal-variance analysis, a reporting style could be:

Welch’s ANOVA indicated a significant difference in satisfaction across the three groups. Games-Howell post-hoc comparisons showed that Group C scored significantly higher than Groups A and B, while the difference between Groups A and B was not statistically significant.

Replace the example statistics with your own software output.

Do not copy a reporting sentence and insert numbers without checking what the procedure actually tested.

---

What If You Need a Different Post-Hoc Test?

Tukey and Games-Howell are not the only options.

For example, IBM SPSS also provides procedures such as Bonferroni, Scheffé and Dunnett under different comparison settings, while Tamhane’s T2 and Dunnett’s T3 are available for unequal-variance situations. (IBM SPSS documentation)

The correct procedure also depends on the research question.

If you only want to compare several treatment groups against one control group, Dunnett’s test can be more directly aligned with that question than a full all-pairs procedure.

If your outcome or design does not support ordinary ANOVA assumptions, a non-parametric route such as Kruskal-Wallis with an appropriate post-hoc procedure may need consideration.

The question should therefore be:

“What comparison am I actually trying to make, and what assumptions does my data support?”

not:

“Which post-hoc test is most popular?”

---

Tukey HSD vs Games-Howell: My Decision Rule

When I advise a dissertation researcher, I use this simple framework:

Step 1: Decide what the omnibus test should be

Use standard one-way ANOVA when its assumptions are reasonably appropriate. Consider Welch’s ANOVA when variance heterogeneity is substantial.

Step 2: Assess the variance structure

Look at group standard deviations, group sizes and relevant diagnostics. Levene’s test can contribute to this decision.

Step 3: Match the post-hoc procedure

Equal variances + equal group sizes → Tukey HSD

Equal variances + unequal group sizes → Tukey-Kramer

Unequal variances → Games-Howell

Step 4: Report more than significance

Report the omnibus result, relevant effect size, pairwise mean differences, adjusted p-values and confidence intervals.

That is the analysis I would find much easier to defend in a dissertation viva.

Final Takeaway

The tukey hsd vs games howell decision should come from your statistical assumptions, not from the first option you recognise in your software.

My practical rule is:

Equal variances → Tukey HSD or Tukey-Kramer
Unequal variances → Games-Howell
Unequal sample sizes alone → do not automatically switch to Games-Howell

Then go one step further. Report why you selected the test, which omnibus procedure you used, which groups differed, the size of those differences, the adjusted p-values and suitable confidence intervals.

That is what turns a dissertation results section from a collection of software tables into a defensible statistical analysis.

If you are unsure whether your analysis matches your research design, an independent statistical review can be useful before you submit the chapter. Statssy’s statistics tutoring and research support covers this kind of methodological checking.

Author

Siddharth Gupta, Independent Analyst and Researcher
LinkedIn profile

Sources

• NIST/SEMATECH, Tukey’s Method. (NIST/SEMATECH)
• SciPy documentation, scipy.stats.tukey_hsd. (SciPy documentation)
• IBM SPSS Statistics documentation, One-Way ANOVA Post Hoc Tests. (IBM SPSS documentation)
• Games, P. A. & Howell, J. F. (1976), Pairwise Multiple Comparison Procedures with Unequal N’s and/or Variances: A Monte Carlo Study. The original Games-Howell study is referenced in current SciPy documentation. (SciPy documentation)