FROM PAPER TO PODIUM: EVALUATION OF THE TRANSLATIONAL POTENTIAL OF PERFORMANCE NUTRITION RELATED RESEARCH

KEY POINTS

• Sport nutrition is one of the fastest growing and evolving disciplines of sport and exercise science with research ranging from the discovery of novel nutrient sensitive cell signaling pathways to the effects of sports drinks on ratings of perceived effort.
• Sport nutritionists often utilize both classic and contemporary literature to help guide their applied practice, ultimately hoping that the research can help to fuel winning performances.
• Step one in the translation of research to practice should always be a well-structured critique of the translational potential of the existing scientific evidence.
• We present an operational framework (the paper to podium matrix) providing a checklist of criteria to prompt the critical evaluation of performance nutrition related research papers.
• It is the combination of boldness of reform (i.e., innovations in research) and quality of execution (i.e., ease of administration of practical solutions) that is most likely to deliver the transition from paper to podium.

INTRODUCTION

The idea of testing nutritional interventions dates back many centuries. However, sports nutrition as both an academic and practical discipline, has stemmed mainly from research performed in the late 1960s, following a series of seminal studies examining the effects of muscle glycogen on exercise capacity and performance (Bergstrom & Hultman, 1966 a,b; Bergstrom et al., 1967; Hermansen et al., 1967). Since these landmark articles, the field of sports nutrition has developed significantly with studies examining the effects of nutrient availability and ergogenic aids for modulation of performance, recovery, training adaptation and body composition (Thomas et al., 2016).

Although the practitioner’s goals should always be the delivery of research-informed practice, there is often a required balance struck between waiting for the peer-reviewed randomized controlled trials (RCTs) and subsequent meta-analysis, and the early adoption of novel methods in an attempt to gain a competitive advantage (Coutts, 2016). The sport nutrition discipline may be viewed as highly confusing and contradictory, with numerous real-world examples where research findings have been misconstrued to inform applied practice. It is imperative for the growth and development of the sports nutrition area that a well-structured critique of available peer-reviewed scientific evidence is conducted prior to use in practice.

The aim of this sports science exchange article is to provide the sports nutritionist with a time-efficient operational framework to critically evaluate the translational potential of applying research to practice (Close et al., 2019). The aim of this framework is not to critique the general methodology of research, but instead appraise the direct and applied applicability of research through the use of our Paper to Podium (P2P) Matrix (Figure 1.)

RESEARCH CONTEXT

As the area of sports science continues to grow, research has moved toward investigation of the underpinning mechanisms behind adaptation to exercise and nutritional interventions using molecular biology techniques (Close et al., 2016). Cell culture and rodent models are often utilized to investigate physiological responses (Gomez-Cabrera et al., 2008; Holland et al., 2016; Owens et al., 2015); however, findings may or may not have application to the exercising human during whole body exercise. While identifying suppression or activation of a specific signaling pathway is likely to yield new insights and research questions, this does not mean that an athlete’s training or nutritional program should immediately change accordingly. It is important to not only consider the in vitro model/technique, but also whether the study was conducted using human cell types as opposed to rodent, as these nonhuman cellular investigations are often designed to examine the potential mechanisms of actions of adaptation and were never intended to be directly translated into practice. It is crucial to assess if the trial has ecological validity (i.e., the methods somewhat replicate the conditions expected in the real-world setting) when assessing the translational ability of the paper. Initial and careful consideration of the research context therefore provides the platform to further evaluate the translational potential to a specific sporting situation.

PARTICIPANT CHARACTERISTICS

Currently, there is often a large amount of ambiguity when describing physiological characteristics and the general training status of participants. Often descriptions such as ‘well-trained’, ‘elite’ and ‘world-class’ are used interchangeably, cited incorrectly, and sometimes provided without any detail regarding habitual training or physiological health and fitness measures. Jeukendrup et al. (2000) initially delivered an objective standards framework for categorization of cyclists depending upon qualitative (e.g., ranking, race days per annum, training frequency) and quantitative (Wattmax, maximal oxygen uptake (VO2max), economy) criteria. The idea of defining habitual and physiological standards such as this may be pertinent in all endurance sports; however, within team sport scenarios, it may be practical to simply classify subjects as amateur, semi-professional or professional depending on the grading of competition they compete in. Authors should describe participants using as much quantifiable data as possible and where this data is not cited, the results must be treated with some degree of caution given the vast differences in response to exercise between untrained and trained participants (Bergman & Brooks, 1999). In the context of performance, it is therefore possible that the performance-enhancing effects of a specific intervention (e.g., beetroot juice) is negated in trained vs. less trained participants (Wilkerson et al., 2012). As such, the efficacy of any nutritional intervention should be investigated in the specific population of which the intervention is intended to be used in practice. In this regard, good examples include studying the efficacy of ketogenic diets in Olympic-standard race walkers (Burke et al., 2017) or work from our group that examined rugby-game glycogen utilization taken from professional players during a competitive fixture (Bradley et al., 2017).

RESEARCH DESIGN

The “gold standard” for research design is usually considered to be the randomized, counterbalanced, repeated measures, crossover design that incorporates a double-blind and placebo-controlled intervention. Typically, such research would also include adequate familiarization trials along with controlling all of the threats to internal validity such as the effects of differing warm-ups, prior exercise, prior nutrition, consistent laboratory conditions, verbal encouragement, etc. For example, in assessing the effects of a novel sports drink on exercise performance, the same participants are tested twice, and the test drink and control drink are taste, color and flavor matched. Assuming appropriate pretrial dietary and exercise controls, a valid and reliable performance test, and adoption of suitable statistical procedures, this design should allow the researchers to ascertain the true effects of the test drink on performance in the absence of researcher/participant bias and placebo effects (Clark et al., 2000, Jones et al., 2016). Many studies, however, may not be able to use this “gold standard” of research design. For example, in the case of testing the effects of “real” foods on exercise performance (e.g., high fat vs. high carbohydrate (CHO) intakes), the research design can lack the double-blind placebo-controlled approach given that both researchers and participants are consciously aware of the food eaten by the subjects (Burke et al., 2017). Similarly, when examining the effects of CHO restriction on training adaptations and performance, a double-blind placebo-controlled design may be lacking (Marquet et al., 2016) unless CHO availability has been manipulated via the provision of taste, color and flavor matched treatments (Morton et al., 2009). Although a consideration must be made as to whether the dietary intervention has truly caused the performance effect, or whether this was caused by a cognitive bias toward any specific dietary approach, we acknowledge the power of “placebo” within the applied practice realm. Within sport nutrition research, it is often difficult to fully avoid cognitive bias (i.e., a belief effect) given that some interventions can be obvious (for example, the effects of caffeine are hard to mask, and most athletes are aware of the performance-enhancing effects of caffeine). In some research scenarios, a counterbalanced, repeated measure, crossover design is simply not possible due to issues that may arise with required “wash-out” times or repeated bout effects, such as in the evaluation of common supplementation and ergogenic aids (e.g., carnitine, creatine, beta-alanine, vitamin D) on muscle and whole body performance (Owens et al., 2014; Sale et al., 2011; Tomcik et al., 2018; Wall et al., 2011). In these instances, investigations may have selected to use a matched group design and, in these situations, it is essential that groups are randomly allocated to treatment groups that are matched on baseline physiological characteristics (e.g., age, stature, body composition, physiological profile, etc.). Like many things in science, the perfect research design never exists. Nonetheless, practitioners must consider the nuances discussed above prior to making any conclusions on the translational potential of the study in question.

DIETARY & EXERCISE CONTROLS

Despite published guidelines for standardization within research (Jeacocke & Burke, 2010), literature often fails to objectively control habitual diet or report any dietary controls employed. Indeed, there are also discrepancies present surrounding the actual delivery of dietary controls, ranging from subject self-reporting, researchers freshly preparing, and/or meal preparation companies preparing and delivering. In addition, it may be useful to also report and standardize exercise during the days leading to the experimental trial, especially within situations that may lead to differences in pre-exercise muscle glycogen availability and thus altered performance. There are, of course, advantages and disadvantages to many of the common dietary standardization methods outlined above, including cost and ease of intervention but also the ecological validity to the research participants. Fasting is often utilized as a control for dietary intervention studies given that feeding around exercise can significantly alter metabolic responses pre-, during, and post-exercise (Bartlett et al., 2013; Horowitz et al., 1997; Lane et al., 2013; Widrick et al., 1993). However, it is unlikely that elite athletes would compete or, indeed, undertake consecutive aerobic and resistance training sessions in the fasted state or without energy intake between sessions; again, caution should be taken when translating research from studies on fasted participants to athletes intending to compete in the fed state. Finally, practitioners should be aware that some studies purposefully restrict the nutrient of interest, e.g., nitrate (Lane et al., 2014) or polyphenols (Bell et al., 2014) in an attempt to ensure a consistent pretrial concentration of the substance of interest. While this approach is scientifically sound, some caution should be exerted when applying the results given that the athlete in the real world may actually consume a diet naturally rich in the compound of interest. Taken together, we therefore recommend that practitioners carefully evaluate research designs and dietary protocols in relation to the nutritional practices, training loads and training organizational practices that are inherent to their specific sport. And potentially, future study designs should consider utilizing conditions considered to be “best nutritional practice” within their real-world context to assess the “true” magnitude of intervention.

VALIDITY & RELIABILITY OF EXERCISE PROTOCOLS & PERFORMANCE TESTS

The ecological validity, reliability and real-world context of exercise models are often overlooked within research, which alters the ability to interpret and translate the data directly into practice. One such example of this is the one-legged knee extensor model (Andersen et al., 1985), which has been used extensively in exercise metabolism research to evaluate local control of muscle metabolism and adaptation to exercise training. From a mechanistic perspective, this model is advantageous, as examination of responses can be conducted in the same individual with the resting leg as a control. However, the single leg modality fails to accurately replicate exercise in vivo, given that these models have a limited muscle mass, low cardiac function and a reduced hormonal response (Helge et al., 2007). It is noteworthy that dose-response studies evaluating the optimal protein dose to stimulate muscle protein synthesis suggest that the absolute protein dose is effectively doubled (i.e., 20 to 40 g post-exercise protein feeding) when using whole body resistance training protocols (Macnaughton et al., 2016) vs. unilateral exercise protocols (Moore et al., 2009; Witard et al., 2014).

In addition to ecological validity, there is also the requirement to carefully consider the reliability of any exercise performance tests as well as the inclusion of any familiarization trials. Within the context of endurance type performance tests, it can also be debated as to whether the participant should have access to any internal (e.g., heart rate) or external (e.g., power output, running velocity) cues during testing (Edwards & McCormick, 2017) as well as the validity of time trial vs. exercise capacity tests (Currell & Jeukendrup, 2008; Karsten et al., 2018). For example, in the case of professional road cycling, it could be argued that the “true” effects of any nutritional intervention or ergogenic aid should always be evaluated with access to external cues, given that riders have continual access to power meters, heart rate data and verbal feedback from accompanying support staff. Additionally, the use of time-trial and exercise capacity tests could both be considered as valid performance measures, given that both situations do occur in the form of designated time-trial stages and the ability to respond to “attacks” on mountain climbs, respectively. We need to remember, the controlled, calm and temperature-controlled laboratory environment is usually never representative of the elite sporting arena. Would the effects of caffeine on motivation be the same in a calm laboratory environment compared with walking out at Twickenham for a Rugby World Cup final in front of 80,000 spectators?

DATA ANALYSES & PRESENTATION

Often, one of the most contradictory components of performance nutrition related research is the way in which data has been analyzed and presented. In the realm of applied performance research, there has been a recent trend to adopt the approach of magnitude-based inferences (Batterham & Hopkins, 2006) as opposed to more traditional probability-based testing and, as such, it is important that applied practitioners are familiar with the advantages and disadvantages of both methods. Importantly, researchers should provide a clear rationale and justification of the chosen sample size (often accompanied by power calculations) and reasons for their choice of statistical analysis. Finally, the approach to data presentation can also greatly influence how the results are evaluated and interpreted by readers. For example, presentation of group means and standard error (as opposed to standard deviation) does not provide a true representation of the variability between subject responses, but researchers often choose to represent variability using standard error (especially in graphical format) for cosmetic reasons (Morton, 2009).

Given that practitioners usually pursue the application of interventions with individual athletes, richer evaluations of data should be made, and individual differences plotted and visualized where possible, especially when the research was conducted on a small sample. For example, in a recent study from our laboratory examining the effect of muscle glycogen availability on endurance capacity, we observed that mean exercise capacity was increased by 60 min with high vs. moderate pre-exercise glycogen concentration (i.e., 600 vs. 300 mmol.kg^-1 dry weight) (Impey et al., 2016). Nonetheless, the individual magnitude of increase in time to exhaustion ranged from 4 min to 113 min. Clearly, evaluation of individual responses can be lost in translation where only group means are presented.

PRACTICAL APPLICATIONS

• Consider the translational potential and context in which the data are collected.
• Investigate the participant characteristics from studies to decide whether the research has direct applicability to your own context.
• Study the research design fully and consider whether it actually was the nutritional intervention that caused an effect.
• Decide whether the research used adequate dietary and/or exercise controls to confidently attribute differences to their intervention of choice.
• Search for any validity and reliability issues within the chosen exercise protocols and performance tests, paying particular attention to the environment in which the study was conducted.
• Reflect on how the data analyses were conducted, how the data were presented, and look for individual differences and standard deviation values.

SUMMARY

Although relatively simple in concept, the translation of research to practice is not always a straightforward process. Indeed, elite sport is dynamic, unpredictable and often chaotic, none of which can be interpreted by a two-way ANOVA or predicted from the controlled laboratory environment. Despite the continual pursuit and often-impatient demands for the latest-winning edge, we consider that the starting point for the research-informed practitioner should always be the critical evaluation of the translational potential of the available scientific evidence. Put simply, we must look beyond the abstract, the 140-character tweet and latest infographic in order to truly evaluate the scientific rigor and translational potential of performance nutrition related research studies. Utilization of the P2P Matrix (Figure 1) may help practitioners to personally evaluate a research paper, increasing their own confidence in the intervention they are about to implement, which may ultimately result in a more enthusiastic consultation with the athlete and increase the chance of an effective intervention. It is readily acknowledged that the content and indices of such a framework are not exhaustive. Rather, it was the deliberate aim to provide a time-efficient evaluation tool that can be readily applied by practitioners who all too often operate under the intense time constraints inherent to elite sport. Subsequent to the evaluation of existing research, we also encourage practitioners to conduct field-based research (e.g., case reports or small sample size studies) with the same degree of scientific rigor and precision of measurement that is requisite of RCTs. Ultimately, it is the combination of boldness of reform (i.e., innovations in research) and quality of execution (i.e., ease of administration of practical solutions) that is most likely to deliver the transition from paper to podium.

REFERENCES

Andersen, P., R.P. Adams, G. Sjogaard, A. Thorboe, and B. Saltin (1985). Dynamic knee extension as model for study of isolated exercising muscle in humans. J. Appl. Physiol. 59:1647-1653.

Bartlett, J.D., J. Louhelainen, Z. Iqbal, A.J. Cochran, M.J. Gibala, W. Gregson, G.L. Close, B. Drust, and J.P. Morton (2013). Reduced carbohydrate availability enhances exercise-induced p53 signaling in human skeletal muscle: implications for mitochondrial biogenesis. Am. J. Physiol. 304:R450-458.

Batterham, A.M., and W.G. Hopkins (2006). Making meaningful inferences about magnitudes. Int. J. Sports Physiol. Perform. 1:50-57.

Bell, P.G., I.H. Walshe, G.W. Davison, E. Stevenson, and G. Howatson (2014). Montmorency cherries reduce the oxidative stress and inflammatory responses to repeated days high-intensity stochastic cycling. Nutrients. 6:829-843.

Bergman, B.C., and G.A. Brooks (1999). Respiratory gas-exchange ratios during graded exercise in fed and fasted trained and untrained men. J. Appl. Physiol. 86:479-487.

Bergstrom, J., L. Hermansen, E. Hultman, and B. Saltin (1967). Diet, muscle glycogen and physical performance. Acta Physiol. Scand. 71:140-150.

Bergstrom, J., and E. Hultman (1966a). The effect of exercise on muscle glycogen and electrolytes in normals. Scand. J. Clin. Lab. Invest. 18:16-20.

Bergstrom, J., and E. Hultman (1966b). Muscle glycogen synthesis after exercise: an enhancing factor localized to the muscle cells in man. Nature. 210: 309-310.

Bradley, W.J., J.C. Morehen, J.D. Haigh, J. Clarke, T.F. Donovan, C. Twist, C. Cotton, S. Shepherd, M. Cocks, A. Sharma, S.G. Impey, J.P. Morton, and G.L. Close (2017). Muscle glycogen utilisation during Rugby League match play: effects of pre-game carbohydrate intake. J. Sci. Med. Sport. 20:878-883.

Burke, L.M., M.L. Ross, L.A. Garvican-Lewis, M. Welvaert, I.A. Heikura, S.G. Forbes, J.G. Mirtschin, L.E. Cato, N. Strobel, A.P. Sharma, and J.A. Hawley (2017). Low carbohydrate, high fat diet impairs exercise economy and negates the performance benefit from intensified training in elite race walkers. J. Physiol. 595:2785-2807.

Clark, V.R., W.G. Hopkins, J.A. Hawley, and L.M. Burke (2000). Placebo effect of carbohydrate feedings during a 40-km cycling time trial. Med. Sci. Sports Exerc. 32:1642-1647.

Close, G.L., D.L. Hamilton, A. Philp, L.M. Burke, and J.P. Morton (2016). New strategies in sport nutrition to increase exercise performance. Free Radic. Biol. Med. 98:144-158.

Close, G.L., A.M. Kasper, and J.P. Morton (2019). From paper to podium: quantifying the translational potential of performance nutrition research. Sports Med. 49 (Suppl 1):S25-S37.

Coutts, A.J. (2016). Working Fast and working slow: the benefits of embedding research in high performance sport. Int. J. Sports Physiol. Perform. 11:1-2.

Currell, K., and A.E. Jeukendrup (2008). Validity, reliability and sensitivity of measures of sporting performance. Sports Med. 38:297-316.

Edwards, A.M., and A. McCormick (2017). Time perception, pacing and exercise intensity: maximal exercise distorts the perception of time. Physiol. Behav. 180:98-102.

Gomez-Cabrera, M.C., E. Domenech, M. Romagnoli, A. Arduini, C. Borras, F.V. Pallardo, J. Sastre, and J. Vina (2008). Oral administration of vitamin C decreases muscle mitochondrial biogenesis and hampers training-induced adaptations in endurance performance. Am. J. Clin. Nutr. 87:142-149.

Helge, J.W., B. Stallknecht, E.A. Richter, H. Galbo, and B. Kiens (2007). Muscle metabolism during graded quadriceps exercise in man. J. Physiol. 581:1247-1258.

Hermansen, L., E. Hultman, and B. Saltin (1967). Muscle glycogen during prolonged severe exercise. Acta Physiol. Scand. 71:129-139.

Holland, A.M., W.C. Kephart, P.W. Mumford, C.B. Mobley, R.P. Lowery, J.J. Shake, R.K. Patel, J.C. Healy, D.J. McCullough, H.A. Kluess, K.W. Huggins, A.N. Kavazis, J.M. Wilson, and M.D. Roberts (2016). Effects of a ketogenic diet on adipose tissue, liver, and serum biomarkers in sedentary rats and rats that exercised via resisted voluntary wheel running. Am. J. Physiol. 311: R337-R351.

Horowitz, J.F., R. Mora-Rodriguez, L.O. Byerley, and E.F. Coyle (1997). Lipolytic suppression following carbohydrate ingestion limits fat oxidation during exercise. Am. J. Physiol. 273:E768-E775.

Impey, S.G., K.M. Hammond, S.O. Shepherd, A.P. Sharples, C. Stewart, M. Limb, K. Smith, A. Philp, S. Jeromson, D.L. Hamilton, G.L. Close, and J.P. Morton (2016). Fuel for the work required: a practical approach to amalgamating train-low paradigms for endurance athletes. Physiol. Rep. 4:e12803.

Jeacocke, N.A., and L.M. Burke (2010). Methods to standardize dietary intake before performance testing. Int. J. Sport Nutr. Exerc. Metab. 20:87-103.

Jeukendrup, A.E., N.P. Craig, and J.A. Hawley (2000). The bioenergetics of World Class Cycling. J. Sci. Med. Sport. 3:414-433.

Jones, H.S., E.L. Williams, D.C. Marchant, S.A. Sparks, C.A. Bridge, A.W. Midgley, and L.R. Mc Naughton (2016). Deception has no acute or residual effect on cycling time trial performance but negatively effects perceptual responses. J. Sci. Med. Sport. 19:771-776.

Karsten, B., J. Baker, F. Naclerio, A. Klose, A. Bianco, and A. Nimmerichter (2018). Time trials versus time to exhaustion tests: effects on critical power, w' and oxygen uptake kinetics. Int. J. Sports Physiol. Perform. 13:183-188.

Lane, S.C., S.R. Bird, L.M. Burke, and J.A. Hawley (2013). Effect of a carbohydrate mouth rinse on simulated cycling time-trial performance commenced in a fed or fasted state. Appl. Physiol. Nutr. Metab. 38:134-139.

Lane, S.C., J.A. Hawley, B. Desbrow, A.M. Jones, J.R. Blackwell, M.L. Ross, A.J. Zemski, and L.M. Burke (2014). Single and combined effects of beetroot juice and caffeine supplementation on cycling time trial performance. Appl. Physiol. Nutr. Metab. 39:1050-1057.

Macnaughton, L.S., S.L. Wardle, O.C. Witard, C. McGlory, D.L. Hamilton, S. Jeromson, C.E. Lawrence, G.A. Wallis, and K.D. Tipton (2016). The response of muscle protein synthesis following whole-body resistance exercise is greater following 40 g than 20 g of ingested whey protein. Physiol Rep. 4:e12893.

Marquet, L.A., J. Brisswalter, J. Louis, E. Tiollier, L.M. Burke, J.A. Hawley, and C. Hausswirth (2016). Enhanced endurance performance by periodization of carbohydrate intake: "sleep low" strategy. Med. Sci. Sports Exerc. 48:663-672.

Moore, D.R., M.J. Robinson, J.L. Fry, J.E. Tang, E.I. Glover, S.B. Wilkinson, T. Prior, M.A. Tarnopolsky, and S.M. Phillips (2009). Ingested protein dose response of muscle and albumin protein synthesis after resistance exercise in young men. Am. J. Clin. Nutr. 89:161-168.

Morton, J.P. (2009). Reviewing scientific manuscripts: how much statistical knowledge should a reviewer really know? Adv. Physiol. Educ. 33:7-9.

Morton, J.P., L. Croft, J.D. Bartlett, D.P. Maclaren, T. Reilly, L. Evans, A. McArdle, and B. Drust (2009). Reduced carbohydrate availability does not modulate training-induced heat shock protein adaptations but does upregulate oxidative enzyme activity in human skeletal muscle. J. Appl. Physiol. 106:1513-1521.

Owens, D.J., A.P. Sharples, I. Polydorou, N. Alwan, T.F. Donovan, J. Tang, R.G. Cooper, W.D. Fraser, J.P. Morton, C. Stewart, and G.L. Close (2015). A systems based investigation into vitamin D and skeletal muscle repair, regeneration and hypertrophy. Am. J. Physiol. 309:E1019-E1031.