Putting MBI on a Formal Footing:
a Comment on The Vindication of
Magnitude-Based Inference Daniël Lakens Sportscience 22,
sportsci.org/2018/CommentsOnMBI/dl.htm, 2018 Summary: Magnitude based inference (MBI) has been
successful in moving researchers beyond the limitations of null-hypothesis
significance tests (NHST) but has faced criticism for a lack of error
control. One way forward is to place MBI on a more formal footing by making
either Frequentist error control or the quantification of Bayesian posterior
probabilities central to MBI. Batterham and Hopkins
have introduced sport science to statistical inferences that do not just aim
to reject the null hypothesis, but invite researchers to interpret their data
in relation to meaningful effect sizes. This is an important accomplishment.
However, recent criticisms on magnitude based inferences by Sainani (2018)
have pointed out that when the verbal labels proposed by Batterham and
Hopkins are interpreted dichotomously, MBI has unacceptably high error rates.
To prevent
sport scientists from falling back on NHST in the face of criticism on MBI, I
suggest evaluating MBI in relation to alternative approaches to statistical
inference–either full Bayesian estimation or equivalence testing–that are on
a more solid theoretical footing, and aim to achieve very similar goals.
Importantly, a choice needs to be made whether MBI is at its core a Bayesian
of Frequentist approach to statistical inferences. As pointed out
by Sainani, the use of confidence intervals by Batterham and Hopkins to make
judgments about the probability of true values “requires interpreting
confidence intervals incorrectly, as if they were Bayesian credible
intervals.” As Batterham and Hopkins (2006) wrote: ‘The approach we have
presented here is essentially Bayesian but with a “flat prior”’ One way
forward seems to switch to a formal Bayesian interpretation. This would make
MBI very similar to the ROPE procedure as suggested by Kruschke (2018) where
a region of practical equivalence (ROPE) is specified, and a Bayesian highest
density interval is interpreted based on whether or not it overlaps with the
region of practical equivalence. A second
alternative is to strongly value error control. This requires MBI to be based
on a formal Frequentist footing, where long-run error rates are accurately
controlled at a desired level. This would make MBI very similar to
equivalence testing (Lakens, 2017; Lakens, Isager, & Scheel, 2018), where
a smallest effect size of interest (SESOI) can be rejected whenever a 90%
confidence interval lies between the equivalence bounds based on the SESOI. By specifying the epistemological basis
of MBI, and by framing the answers MBI provides in formally correct terms,
sport scientists can continue to make statistical inferences that take meaningful
effect sizes into account, without being criticized either for unacceptable
error control (the Frequentist interpretation) or for the interpretation of
confidence interval as the likely range of the true magnitude (the Bayesian
interpretation). Batterham,
A. M., & Hopkins, W. G. (2006). Making Meaningful Inferences About
Magnitudes. International Journal of Sports Physiology and Performance, 1(1),
50–57. https://doi.org/10.1123/ijspp.1.1.50 Kruschke,
J. K. (2018). Rejecting or Accepting Parameter Values in Bayesian Estimation.
Advances in Methods and Practices in Psychological Science, 2515245918771304.
https://doi.org/10.1177/2515245918771304 Lakens, D., Scheel, A. M., & Isager, P. M.
(2017). Equivalence
Testing for Psychological Research: A Tutorial. PsyArXiv. https://doi.org/10.17605/OSF.IO/V3ZKT Lakens,
D. (2017). Equivalence Tests: A Practical Primer for t Tests, Correlations,
and Meta-Analyses. Social Psychological and Personality Science, 8(4),
355–362. https://doi.org/10.1177/1948550617697177 Sainani,
K. L. (2018). The Problem with “Magnitude-Based Inference.” Medicine &
Science in Sports & Exercise, Publish Ahead of Print. https://doi.org/10.1249/MSS.0000000000001645 Back to index of comments. Back to The Vindication of Magnitude-Based Inference. First published 3 June
2018. |
