Evidence-Based Medicine and Best-Practice Guidelines

Caroline Boyle, BA

Niv Marom, MD

Robert G. Marx, MD

Dr. Marx or an immediate family member has stock or stock options held in Mend and serves as a board member, owner, officer, or committee member of the International Society of Arthroscopy, Knee Surgery, and Orthopaedic Sports Medicine. Neither of the following authors nor any immediate family member has received anything of value from or has stock or stock options held in a commercial company or institution related directly or indirectly to the subject of this chapter: Ms. Boyle and Dr. Marom.

INTRODUCTION

A desire to provide standardized and competent care has driven the medical field to implement evidence-based medicine. The goal of evidence-based medicine is for physicians to use the best available evidence, as well as their clinical experience, to guide their clinical decision making. In order for physicians to determine which research studies provide the best evidence, medical journals classify publications based on their level of evidence. The higher the level of evidence, the less chance there is that the results of the study might be affected by bias or confounding. Systematic reviews and meta-analyses compile available evidence to make it easier for physicians to review all of the publications on a certain topic. Once there is a sufficient amount of evidence on a given topic, healthcare organizations, such as the American Academy of Orthopaedic Surgery, develop best-practice guidelines to inform health care in their field. Best-practice guidelines have important implications both legally (malpractice litigation) and economically (healthcare insurance company coverage); thus, it is very important that people understand, and critically evaluate, how they are developed and modified when new evidence becomes available.

HISTORY OF EVIDENCE-BASED MEDICINE

In 2007, The British Journal of Medicine published a list of the 15 most important medical milestones since the Journal originated in 1840.¹ On that list were vaccinations, the discovery of DNA, as well as the concept of evidence-based medicine (EBM). This concept was originally described by Gordon Guyatt at McMaster University in 1992.² Guyatt explained that with EBM, clinical decisions are made based on the best available evidence and patients’ values, rather than exclusively on medical practitioners experience or intuition. A good practitioner will combine both their experiences and the best available literature to make medical decisions.³ The challenge to handle an increased demand for healthcare services required an effective way to determine the most successful treatment plan for each patient.

Advances in orthopaedics between 1940 and 1970s were made based on case series for treatments such as total joint replacement, internal fixation of fractures, and instrumentation for the spine. However, after these major advances, it became clear different research methods (case-control studies, prospective cohort studies, and randomized controlled trials) were needed to continue to improve patient outcomes.⁴ The Journal of Bone and Joint Surgery began to publish “Evidence-Based Orthopaedics” in 2000 which published randomized controlled trials (RCTs) and provided a commentary to put the evidence presented in the RCT into context. The journal did this to help orthopaedic surgeons use the best evidence available to inform their clinical practice.⁵

LEVELS OF EVIDENCE

Sacket et al first broached the concept of levels of evidence in 1996 when he explained that there are not always RCTs available to inform medical decisions.⁶ In these scenarios, one must be able to evaluate the evidence that is available and determine what is of the most significance. See the pyramid in Figure 1 which depicts the quality of evidence of different research studies. RCTs are considered to be the strongest type of evidence followed by cohort studies, case series, and finally by expert opinions (Figure 1). Systematic reviews are considered the strongest because they combine all available evidence.

The level of evidence table is a tool that was developed to help organize the hierarchy of different types of research studies. In 2003, the Journal of Bone and Joint Surgery published their first level of evidence table and began to assign every article published a level of evidence. This was done in an effort to help readers classify the evidence that they were reading and to encourage authors/researchers to improve their research methodology.⁵ See Table 1 which is the updated version of the Level of Evidence table that is used by the Journal today (Table 1). There are four different types of studies which can then be assigned levels I to V. Therapeutic studies evaluate the effect of a treatment on patient outcomes. Prognostic studies consider the effect of patient characteristics on patient outcomes. Diagnostic studies study if the results of a test are related to the presence of a disease. Lastly, economic studies consider the cost of a certain treatment.⁷

The reason some types of studies are given higher level of evidence over others is because of their ability to limit bias and confounding. Bias is described as systematic error introduced by the design of the study.⁸ This can come in the form of selection bias, in which case the selection of participants results in a group that is different than the study population. For example, if observing complication rates after a procedure, and one only surveys people who are still attending follow-up visits, this might lead to the conclusion that there is a higher complication rate because patients who continue to attend follow-up visits often have increased complaints they want to share with their surgeon. Healthy patients are less likely to attend long-term follow-up visits as they do not have anything to share with their surgeon. Thus, the population of people still attending follow-up visits may have a higher concentration of patients with bad outcomes. Additionally, information bias is defined as a systematic error in the measurement of exposures or outcomes.⁸ For example, if patients are asked to report on the state of their condition, those who have had problems may be more likely to remember more details. This is known as recall bias.

FIGURE 1 Illustration showing the hierarchy of research studies. (Adapted from Murad MH, Asi N, Alsawas M, Alahdab F: New evidence pyramid. Br J Med 2016;21:125-127.)

Confounding is when there is another variable other than the independent or dependent variables that is associated with either the exposure or the outcome. This can lead to either masking a true correlation in the study results or to creating a correlation that is not actually present.⁸ Both bias and confounding can be limited with good study design.

RANDOMIZED CONTROLLED TRIAL

A randomized controlled trial is given the highest level of evidence because it has the best potential to limit bias and confounding. In this type of study, patients are randomly assigned to the intervention group (receive a treatment) or the control group (receive no treatment or receive the standard treatment). As patients are randomly assigned to a treatment group, it is assumed that factors that cause confounding or bias are evenly distributed between the two groups. However, RCTs often have extremely narrow inclusion/exclusion criteria to achieve randomization. This can limit the generalizability of the results to specific patients. One thing that can limit the quality of an RCT is when randomization is not done properly. Additionally, when the outcome assessors are not blinded to the group designation there can be bias with outcome evaluation.

There are many issues specific to RCTs when randomizing to surgical or nonsurgical treatment. For example, differences in surgeon experience level may affect the uniformity of the surgery arm of the RCT. This is a limitation that is not present in drug-related RCTs. Crossover is another issue that is often found in surgical RCTs, and this can introduce patient selection bias. Crossover is when patients assigned to the nonsurgicalerative group crossover and undergo surgery. For example, Frobell et al published a RCT randomizing patients to either rehabilitation or anterior cruciate ligament reconstruction (ACLR), and by 5 years post randomization, 51% of the patients in the nonsurgical group had chosen to undergo ACLR.⁹ This increases the complexity of the analysis for the comparison between groups. Overall, while RCTs are assigned the highest level of evidence, they are often very expensive and not always possible in the surgical environment due to ethical concerns and patient/surgeon preferences.¹⁰

TABLE 1 Levels of Evidence for Primary Research Question^a^,^b

Study Type	Question	Level I	Level II	Level III	Level IV	Level V
Diagnostic—Investigating a diagnostic test	Is this (early detection) test worthwhile?	Randomized controlled trial	Prospective^c cohort^d study	Retrospective^e cohort^d study Case-control^f study	Case series	Mechanism-based reasoning
Diagnostic—Investigating a diagnostic test	Is this diagnostic or monitoring test accurate?	Testing of previously developed diagnostic criteria (consecutive patients with consistently applied reference standard and blinding)	Development of diagnostic criteria (consecutive patients with consistently applied reference standard and blinding)	Nonconsecutive patients No consistently applied reference standard	Poor or nonindependent reference standard	Mechanism-based reasoning
Prognostic—Investigating the effect of a patient characteristic on the outcome of a disease	What is the natural history of the condition?	Inception^c cohort study (all patients enrolled at an early, uniform point in the course of their disease)	Prospective^c cohort^d study (patients enrolled at different points in their disease) Control arm of randomized trial	Retrospective^e cohort^d study Case-control^f study	Case series	Mechanism-based reasoning
Therapeutic—Investigating the results of a treatment	Does this treatment help? What are the harms?^g	Randomized controlled trial	Prospective^c cohort^d study Observational study with dramatic effect	Retrospective^e cohort^d study Case-control^f study	Case series Historically controlled study	Mechanism-based reasoning
Economic	Does the intervention offer good value for dollars spent?	Computer simulation model (Monte Carlo simulation, Markov model) with inputs derived from level-I studies, lifetime time duration, outcomes expressed in dollars per quality-adjusted life years (QALYs) and uncertainty examined using probabilistic sensitivity analyses	Computer simulation model (Monte Carlo simulation, Markov model) with inputs derived from level-II studies, lifetime time duration, outcomes expressed in dollars per QALYs and uncertainty examined using probabilistic sensitivity analyses	Computer simulation model (Markov model) with inputs derived from level-II studies, relevant time horizon, less than lifetime, outcomes expressed in dollars per QALYs and stochastic multilevel sensitivity analyses	Decision tree over the short time horizon with input data from original level-II and III studies and uncertainty is examined by univariate sensitivity analyses	Decision tree over the short time horizon with input data informed by prior economic evaluation and uncertainty is examined by univariate sensitivity analyses
^aThis chart was adapted from OCEBM Levels of Evidence Working Group, “The Oxford 2011 Levels of Evidence,” Oxford Centre for Evidence-Based Medicine, http://www.cebm.net/ocebm-levels-of-evidence/. A glossary of terms can be found here: http://www.cebm.net/glossary/.
^bLevel-I through -IV studies may be graded downward on the basis of study quality, imprecision, indirectness, or inconsistency between studies or because the effect size is very small; these studies may be graded upward if there is a dramatic effect size. For example, a high-quality randomized controlled trial (RCT) should have ‡80% follow-up, blinding, and proper randomization. The level of evidence assigned to systematic reviews reflects the ranking of studies included in the review (ie, a systematic review of level-II studies is level II). A complete assessment of the quality of individual studies requires critical appraisal of all aspects of study design. ^cInvestigators formulated the study question before the first patient was enrolled. ^dIn these studies, “cohort” refers to a nonrandomized comparative study. For therapeutic studies, patients treated one way (eg, cemented hip prosthesis) are compared with those treated differently (eg, noncemented hip prosthesis). ^eInvestigators formulated the study question after the first patient was enrolled. ^fPatients identified for the study on the basis of their outcome (eg, failed total hip arthroplasty), called “cases,” are compared with those who did not have the outcome (eg, successful total hip arthroplasty), called “controls.” ^gSufficient numbers are required to rule out a common harm (affects >20% of participants). For long-term harms, follow-up duration must be sufficient.
Reproduced with permission from Marx RG, Wilson WS, Swiontokowski MF: Updating the assignment levels of evidence. J Bone Joint Surg Am 2015;64(1):1-2; Table 1.