Carts, Horses, and Evidence

By Koretz, Ronald L

Sentence first, verdict afterward. -Queen of Hearts in Alice in Wonderland

The Queen of Hearts presumed that the Knave was guilty of stealing her tarts, even though Alice had actually done this, and called for his head to be cut off. When the queen was informed that there had been no trial yet, she demanded that the sentence be executed; the trial verdict could follow. The practice of putting the cart before the horse actually dates back to antiquity.

We have known at least since 1936 that malnourished patients have poorer outcomes than do well-nourished patients carrying the same diagnosis.1 We also know that even well-nourished individuals who are deprived of nutrients for a long-enough period of time will develop morbid, and ultimately mortal, complications from malnutrition.2 Extrapolating these findings, one might presume that any treatment that would improve or prevent malnutrition would result in an improved morbidity, and even mortality. Thus, the use of artificial nutrition, in which putative nutrient solutions are infused into veins (parenteral nutrition, or PN) or the GI tract (enteral nutrition, or EN), has been widely accepted for decades as adjunctive therapy for a variety of underlying disease processes. These techniques have been referred to as “nonvolitional feeding” (because the infusion of the solution does not require the cooperation of the patient). Artificial nutrition is not the same as feeding; using the word “feeding” suggests a process of nurturing and downplays the reality that this is a medical intervention, with defined risks and costs.

If a disease process has an absolutely predictable outcome, it is possible to establish the efficacy of an intervention simply by showing that the outcome was favorably changed. Examples of such clinical situations are cardiopulmonary resuscitation for cardiac arrest or dialysis for end-stage renal disease. For the vast majority of disease processes, the outcome is not so well-defined, so the efficacy of any intervention is best determined by undertaking a randomized controlled trial (RCT) comparing its use to its nonuse.

It is generally accepted that we use evidence as the basis of clinical decision making. The RCT is considered to be the standard of evidence regarding therapeutic interventions. More recently, a carefully done systematic review of RCTs has become accepted as an even higher standard.

Systematic reviews should be differentiated from the typical narrative review. In the latter, an expert is asked to write an article discussing and summarizing the important literature about a given topic. The expert is left to his or her own devices and such efforts often become reflections of that individual’s bias.3 The systematic review is more analogous to a research project. The first step is the creation of the question to which the answer will be sought in the medical literature. A formal protocol is then developed, detailing what type of literature will be sought (typically RCTs for therapeutic questions), how the pertinent articles will be found (search strategy), what the inclusion and exclusion criteria are for any study that is identified, the particular outcomes to be sought from each study, and a decision to combine or not to combine the data from the studies. Data combination (such as meta-analysis) offers advantages (increasing the numbers to reduce the chance of a type II error and more precisely defining any effect that is identified) and disadvantages (combining trials that are inherently different). The search is then undertaken, the identified studies are abstracted for the outcomes of interest, and (with or without combination) conclusions drawn based on the data that are found.

It is the evidence that has created the cart and horse problem for artificial nutrition. RCTs comparing PN or EN to no nutrition support have been conducted since the 1970s. By the 1980s, it was clear that the vast majority of those studies were not able to prove that either form of artificial nutrition favorably altered clinical outcome.4,5 Over the intervening 20 years, a number of other randomized trials became available. My colleagues and I published a systematic review of PN in 2001(6) that indicated that PN not only could not be shown to provide benefit in almost all conditions, but even appeared to cause net harm (more infections). (It should be noted that permanent, or even long-term [months] bowel failure was a condition for which outcome was absolutely predictable, and randomized trials for PN were not needed.) We subsequently undertook a systematic review of EN7 that led us to conclude that bias, rather than a true effect, may have been responsible for the few potential benefits that we could identify. Employing a therapeutic intervention when randomized trials have been conducted but have failed to provide convincing evidence of efficacy is putting the cart before an empty stable.

Although artificial nutrition has not been shown to improve parameters of morbidity or mortality, these interventions do improve nutrition outcomes, especially body weight and nitrogen balance.8 The implications of this observation are that such surrogate outcomes cannot be used to predict clinical efficacy and that the association between malnutrition and poor outcome is not causative. (It is more likely that the association is attributable to the fact that more severe disease causes both more malnutrition and poorer clinical outcomes.)

From this perspective, it is surprising that so many investigators continue to put carts before horses by undertaking comparative trials in which patients are randomized into PN or EN treatment arms. After all, it is of little value to compare one intervention with another if we do not know what the absolute value of either is.

Nonetheless, the RCTs have been done and, in this issue of JPEN,9 Andrew Thomson employs principles of evidence-based medicine to critique 5 systematic reviews of these trials.7,10-13 He particularly expresses concern about a phenomenon that he calls “enteralism,” namely an overly enthusiastic acceptance of EN because these systematic reviews concluded that EN is associated with less morbidity than PN. He believes that defects in these reviews make any conclusions unreliable. Although such methodologic critique is to be encouraged, there are problems with some of his interpretations. The intent of this editorial is to view his commentary from the perspectives of evidence-based medicine, including the background data that we have just considered.

Dr Thomson begins with the premise that artificial nutrition is effective. In the opening paragraph of his Introduction, he states that the provision of artificial nutrition is essentia] if there has been significant suboptimal nutritional intake. (While he states that formal trials demonstrating a benefit of nonvolitional nutrition are lacking, he fails to acknowledge that a large number of such formal trials have been conducted.) In the first statement of his conclusion, he reiterates his belief that nonvolitional nutrition support is a vital part of the care of seriously ill patients. For him, the real controversy revolves around the timing, route, and regime of the artificial nutrition that is employed.

He itemized 9 methodologic problems that at least 1 of these systematic reviews contains. I would note that 3 of them are problems that arise in systematic reviewing in general.

The first relates to the heterogeneity of the included trials. Heterogeneity refers to the fact that no 2 RCTs are ever created and conducted in an entirely equal manner; it is a problem that plagues any effort to combine data from several trials. The decision to combine or not to combine depends on the amount of heterogeneity that does exist. Statistical tests have been proposed to detect significant heterogeneity, but these tests only rely on the numerical results, not on the study design. Furthermore, they are relatively insensitive. Heterogeneity can also be dealt with by doing various subgroup analyses (eg, only considering the subset of trials that contain a common element). Systematic reviewers can also decide that the differences are too great to justify data combination and only provide a qualitative review, considering each trial separately and descriptively. With regard to the comparative studies of EN vs PN, 5 systematic reviews, each using a different combination of the RCTs, all came to the same conclusion regarding differences in morbidity, despite the attendant heterogeneity.

Second, Dr Thomson raises concern about the risk of bias that is present in a randomized trial. Empiric evidence has suggested that both the generation of the randomization scheme and the presence or absence of concealment of that scheme, the use or nonuse of blinding, and the presence or absence of an intent-to-treat analysis all influence an observed effect.14-16 Recause the randomized trials that have compared PN to EN have largely been at high risk of bias, Dr Thomson appropriately raises concern that the benefits ascribed to EN may actually be a reflection of bias.

Third, a of the systematic reviews10 seems to have counted one RCT twice. However, it appears that the removal of the duplicate data would not change the conclusion.

Dr Thomson raises the problem of trying to define and stratify infectious complications. Individual RCTs that use infection as an outcome create definitions for such an event; those definitions are often different from study to study. Thus, when a systematic review is designed, a decision can be made either to prespecify what an infection would be or to simply use whatever the original investigators reported or subsequently provided. Although the latter decision does create heterogeneity, it is not a reason to disregard all of the data. Dr Thomson seems to have misunderstood the term “consensus” as it was used in 4 of the 5 systematic reviews.7,10-12 Systematic reviews require that at least 2 individuals independently abstract the data (including judging whether or not the study meets the criteria for inclusion) to reduce the frequency of abstracting errors. When different decisions are made by the abstractors, they meet, often with a third person, to decide what the correct decision is (the consensus process). The word “consensus” that appears in the text of the 4 systematic reviews in question all refer to this process, not to the original criteria that defined the studies to be included. (Those criteria were established in the original protocol that was written before the systematic review was begun.)

Hyperglycemia is more commonly associated with PN. However, the purpose of the RCT is to see if the benefits outweigh the risks. It would be no more appropriate to require RCTs or systematic reviews to make allowances for hyperglycemia than it would be to require trials of medical vs surgical interventions to so treat postoperative wound problems.

Dr Thomson suggests that EN and PN cannot be easily compared because the former should be started early, and the latter is usually begun later in the course of a disease. He claims that “many observers” would consider waiting 96 hours as being tardy. Although the statement is unreferenced, this opinion may have been derived from the Canadian review of artificial nutrition in critically ill adult patients, which is updated on a regular basis.1′ That review employed 10 trials to reach a conclusion that early EN was beneficial. Unfortunately, that Canadian conclusion is debatable. The trials in general were of low quality; when the risk of bias is high, the treatment effect is usually overestimated.14 Furthermore, the individual studies may not have been randomized,18,19 included surgical patients who were not necessarily in the intensive care unit,20,21 may have included children,22 provided some type of PN to the control group (thereby exposing them to the risk of increased infections),23-25 or used different routes of delivery (small intestinal in the early group, gastric in the delayed group).26 In spite of the presence of a number of factors that would bias the results in favor of the intervention, the differences found in the meta-analyses regarding mortality and infectious complications did not even achieve statistical significance. Thus, the presumption that EN should be started early is not yet clearly proven by the available evidence.

Dr Thomson was critical of systematic reviews that included unpublished data. One of the goals of systematic reviewing is to obtain data from all of the trials that have been undertaken, to get around the problem of publication bias. Including such data is a well-accepted practice. On the other hand, the particular meta- analysis of interest27 assessed data from both unpublished and unpublished studies sponsored by a single company (and the article was not a systematic review). Thus, the real concern regarding that article27 was whether or not it would have been published if the data had not been favorable.

Finally, Dr Thomson was critical of the practice of including trials where more than 10% of the patients dropped out. As we have already noted, failure to perform intent-to-treat analyses is one element that can introduce bias. For this reason, the systematic review by Simpson and Doig,13 which was the only one that found any benefit for PN, seemed to be more favorably critiqued by Dr Thomson. However, the lack of an intent-to-treat analysis is only one of several factors that can introduce bias. Simpson and Doig began with 3 quality elements (blinding and concealment of the randomization in addition to intent-to-treat); only 1 of the 11 RCTs mentioned concealment and none were blinded. Because the included trials were still at high risk of bias, there does not seem to be any reason for considering this systematic review to be more reliable than the other four. There is also no reason to single out one element of quality as being more important than the others.

Curiously, while being critical of the data currently in existence regarding the comparative trials, Dr Thomson has a much less critical view about the use of artificial nutrition in general. He proposes that future programs of artificial nutrition be patient- specific combinations of PN and EN and that this therapy be guided by the effect of the intervention on physiologic processes. Although it is appropriate to call for trials to determine if “strategies rather than fixed menus” are useful, to identify potential candidates by physiologic studies, and/or to use improvements in physiologic endpoints as measures of success, such recommendations cannot become accepted therapy until and unless RCTs comparing these more holistic approaches demonstrate that such treatment is better than doing nothing. These future RCTs will have to employ, as the comparators, untreated patients rather than recipients of some other type of artificial nutrition. As Dr Thomson did observe at the end of his equipoise example, no therapy is a viable option.

Dr Thomson is to be complimented for advocating critical reading and an evidence-based approach in making clinical decisions. However, he and other proponents of artificial nutrition still have to provide the horses before this cart can proceed.

References

1. Studley HO. Percentage of weight loss. A basic indicator of surgical risk in patients with chronic peptic ulcer. JAMA. 1936;106:458-460.

2. Keys A. Caloric deficiency and starvation. In: Jolliffe N, ed. Clinical Nutrition. 2nd ed. New York, NY: Harper and Brothers; 1962:122-136.

3. Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987;106:485-488.

4. Koretz RL, Meyer J. Elemental diets: facts and fantasies. Gastroenterology. 1980;78:393-410.

5. Koretz RL. What supports nutritional support? Dig D/s Sci. 1984:29:577-588.

6. Koretz RL, Lipman TO, Klein S. AGA technical review- parenteral nutrition. Gastroenterology. 2001;121:970-1001.

7. Koretz RL, Avenell A, Lipman TO, Braunschweig C, Milne AC. Does enteral nutrition affect outcome: a systematic review of randomized trials. Am J Gastroenterol. 2007;102:412-429.

8. Koretz RL. Nutrition society symposium on ‘End points in clinical nutrition trials.’ Death, morbidity and economics are the only points for trials. Proc Nutr Soc. 2005;64:277-284.

9. Thomson A. The enteral versus parenteral nutrition debate revisited. JPEN J. Parenter Enteral Nutr. 2008;32:474-481.

10. Braunschwieg C, Levy P, Sheean P, Wan X. Enteral compared with parenteral nutrition: a meta-analysis. Am J Clin Nutr. 2001;74:534-542.

11. Gramlich L, Kichian K, Pinilla J, Rodych NJ, Dhaliwal R, Heyland DK. Does enteral nutrition compared to parenteral nutrition result in better outcomes in critically ill patients? A systematic review of the literature. Nutrition. 2004;20: 843-848.

12. Peter JV, Moran JL, Phillips-Hughes J. A metaanalysis of outcomes of early enteral versus early parenteral nutrition in hospitalized patients. Crit Care Med. 2005;33:213-220.

13. Simpson F, Doig GS. Parenteral vs. enteral nutrition in the cally ill patients: a meta-analysis of trials using the intention to treat principle. Intensive Care Med. 2005;31:12-23.

14. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995:273: 408-412.

15. Moher D, Pham B, Jones A, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352:609-613.

16. Ioannidis JPA, Haidich A-B, Pappa M, et al. Comparison of dence of treatment effects in randomized and nonrandomized studies. JAMA. 2001;286:821-830.

17. Dhaliwal R, Heyland DK. Nutrition and infection in the intensive care unit: what does the evidence show? Curr Opin Crit Care. 2005;11:461-467.

18. Chiarelli A, Enzi G, Casadei A, Baggio B, Valerio A, Mazzoleni G. Very early nutrition supplementation in burned patients. Am J Clin Nutr. 1990;51:1035-1039.

19. Pupelis G, Selga G, Austrums E, Kaminski A. Jejunal feeding, even when instituted late, improves outcomes in patients with severe pancreatitis and peritonitis. Nutrition. 2001;17:91-94.

20. Singh G, Ram RP, Khanna SK. Early post-operative enteral feeding in patients with nontraumatic intestinal perforation and peritonitis. J Am Coll Surg. 1998;187:142-146.

21. Malhotra A, Mathur AK, Gupta S. Early enteral nutrition after surgical treatment of gut perforations: a prospective randomised study. J Postgrad Med. 2004;50:102-106.

22. Peck MD, Kessler M, Cairns BA, Chang YH, Ivanova A, Schooler W. Early enteral nutrition does not decrease hypermetabolism associated with burn injury. J Trauma. 2004;57:1143-1149.

23. Moore EE, Jones TN. Benefits of immediate jejunostomy feeding after major abdominal trauma-a prospective, randomized study. J Trauma. 1986;26:874-881.

24. Chuntrasakul C, Siltharm S, Chinswangwatanakul V, Pongprasobchai T, Chockvivatanavanit S, Bunnak A. Early nutritional support in severe traumatic patients. J Med Assoc Tliai. 1996;79:21- 26.

25. Kompan L, Vidmar G, Spindler-Vesel A, Pecar J. Is early enteral nutrition a risk factor for gastric intolerance and pneumonia? Clin Nutr. 2004;23:527-532.

26. Minard G, Kudsk KA, Melton S, Patton JH, Tolley EA. Early versus delayed feeding with an immune-enhancing diet in patients with severe head injuries. JPEN J Parenter Enteral Nutr. 2000;24:145- 149. 27. Moore FA, Feliciano DV, Andrassy RJ, et al. Early enteral feeding, compared with parenteral, reduces postoperative septic complications. The results of a meta-analysis. Ann Surg. 1992;216:172-183.

Ronald L. Koretz, MD

Financial disclosure: none declared.

From the Department of Medicine, Olive View-UCLA Medical Center, Sylmar, California.

Address correspondence to: Ronald L. Koretz, MD, Olive View-UCLA Medical Center, 14445 Olive View Drive, Sylmar, CA 91342; e-mail: koretz@ladhs.org.

Copyright American Society for Parenteral and Enteral Nutrition Jul/ Aug 2008

(c) 2008 JPEN, Journal of Parenteral and Enteral Nutrition. Provided by ProQuest LLC. All rights Reserved.