Biostatistics

Linda I. Suleiman, MD

Chris Culvern, MS

Craig J. Della Valle, MD

Dr. Della Valle or an immediate family member has received royalties from Smith & Nephew and Zimmer; serves as a paid consultant to or is an employee of DePuy, A Johnson & Johnson Company, Smith & Nephew, and Zimmer; has stock or stock options held in Parvizi Surgical Innovations; has received research or institutional support from Smith & Nephew, Stryker, and Zimmer; and serves as a board member, owner, officer, or committee member of the American Association of Hip and Knee Surgeons, the Arthritis Foundation, and the Hip Society. Neither of the following authors nor any immediate family member has received anything of value from or has stock or stock options held in a commercial company or institution related directly or indirectly to the subject of this chapter: Dr. Suleiman and Dr. Culvern.

ABSTRACT

Evidence-based medicine should be used by orthopaedic surgeons to inform clinical decisions. However, this requires the ability to critically review research with an emphasis on study design, statistical tests, and understanding of patient-reported outcomes. The purpose of this chapter is to provide a basic overview of biostatistics and research methodology to facilitate the interpretation of the published orthopaedic literature.

Keywords: common statistical tests; evidence-based medicine; patient-reported outcomes; study design

Introduction

Orthopaedic research investigators require knowledge of the published evidence to apply the appropriate interventions and address the healthcare needs of patients. Knowledge and comprehension of scientific research methodology requires a basic understanding of biostatistics. Biostatistics provides the scale in which information is reviewed and ultimately analyzed.¹ After reading this chapter, an orthopaedic surgeon should understand (1) the strengths and limitations of evidence-based medicine (EBM), meta-analysis, and systematic reviews; (2) the five Levels of Evidence by type of study; (3) the different tools used for commonly reported function scores; and (4) common statistical tests that are used and procedures that are required before conducting data analyses.

Evidence-Based Medicine

Evidence-based medicine (EBM) was first defined by David Sackett in the British Medical Journal (BMJ) in 1996 and has drastically changed how orthopaedic surgeons practice medicine today. EBM is the process of utilizing and assessing scientific information and applying them to medical decisions.² This definition was adapted from Sackett’s commonly quoted definition that “Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”³ He emphasized using individual clinical proficiency with the best externally validated research to guide clinical assessments. This emphasis has changed how orthopaedic research has been conducted.

To practice EBM, one must use the five steps (Figure 1) defined by Sackett: (1) formulate an answerable question, (2) gather evidence, (3) appraise the evidence, (4) implement the evidence, and (5) evaluate the process.²^,³ Formulating an answerable question requires the clinician to define the patient population, delineate the possible treatment modalities, and specify the outcomes of interest. After formulating an answerable question, the clinician must gather externally validated research by conducting a thorough review of the literature. Each study should be assessed for the level of evidence by asking the question, “how good is the evidence?” not “is this evidence perfect?”² To assess the evidence, understanding the levels of evidence based on the study design is critical (Table 1).

Although the EBM movement has held clinical studies to the highest level of evidence, it is not without limitations. The notion that clinical trial data have the highest level of reliability over clinical evidence and
mechanistic logic has been criticized.⁴ In practice, as described in a 2016 study, mechanistic reasoning is used to deduce and apply large population studies to individual patients.⁵ Additionally, the overreliance on the validity of clinical trials, underrepresentation of patients due to demographics, therapy, and comorbidities, and reliance on statistical significance over clinical significance have negatively impacted clinical medicine. As EBM continues to evolve with continued aims to demand reliable clinical research with a high level of external validity, systematic reviews and meta-analyses are being used to help formally synthesize research data and provide a conclusion when answering a specific clinical question.

Figure 1 Illustration showing the five steps of evidence-based medicine.

Table 1 Levels of Evidence

Level	Type of Studies
I	Randomized controlled trials Systematic reviews of randomized controlled trials
II	Prospective cohort Randomized controlled trials with less than 80% follow-up
III	Case-control Retrospective cohort
IV	Case series
V	Expert opinion

Systematic Reviews and Meta-Analyses

Systematic reviews are high-level summarizations of primary research used to identify high-quality evidence related to a study question.⁶ They account for individual study biases and combine patient outcomes from several distinct yet comparable clinical trials.⁷^,⁸ To perform a systematic review, the study purpose and a plausible and answerable question must be identified. Inclusion and exclusion criteria are determined based on the generation of a Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) flowchart.⁹ PRISMA enables the researcher to focus on relevant studies, reduces duplication of efforts, and prevents a researcher from conducting an arbitrary review of the literature. Search methodology must include several publically available electronic databases such as PubMED and the Cochrane Library and involve multiple reviewers. Extraction of data from these databases can be completed using a common published checklist provided by Spindler et al that allows the reporting of systematic reviews in orthopaedic surgery.¹⁰ The strength of a systematic review in providing a recommendation relies on the quality of studies. Reviewers must evaluate the quality of the methodology and report the appropriate grading of evidence followed by the strength of the recommendation. Another similar, yet different approach to assessing information from multiple studies is the meta-analysis.

A meta-analysis differs from a systematic review by utilizing statistical methodology to assess collective data from individual studies.⁶ The process used to collect the component studies is the same as a systematic review, and a good meta-analysis will be comprehensive and will explain the criteria for inclusion. While a systematic review may be more qualitative, a meta-analysis is an objective attempt to synthesize a collection of similar research studies. The goal is to combine many findings into one statistical analysis to determine if the overall effect is statistically present. This process involves combining the outcomes in a weighted-average fashion, so better studies, those that have larger samples or smaller variation, receive a higher weight¹¹^,¹² while newer methodologies have incorporated regression analysis.¹³ A meta-analysis is helpful when studies offer conflicting results of varying strengths. When reviewing these types
of meta-analysis, it is important to understand what studies were included and whether they are cohesive and consistent from a statistical standpoint. A growing trend is to incorporate some measure of this consistency in the analysis.¹⁴

While systematic reviews and meta-analysis allow for the assessment of multiple research studies, orthopaedic research investigators can be tasked with conducting their own studies. Planning research always begins with a clinical question and determining the appropriate study design. In orthopaedic literature, there are two main types of studies: analytic and descriptive.¹⁵

Analytic Studies

Analytic studies seek to answer a scientific hypothesis using inferential statistics (the use of sample data to make generalizations about a target population)¹⁵ and fall into three main categories: randomized controlled trial (RCT), cohort, and retrospective case-control.

In the era of EBM, there are amplified demands for orthopaedic research to provide external validity such that the casual relationship discovered can be applied to patient populations. As such, the benchmark for a research study design is an RCT. Prior to 2000, RCTs encompassed approximately 5% of studies in the Journal of Bone and Joint Surgery (JBJS).¹⁶ Since that time, the JBJS now includes 50% Level I, II, and III evidence, thus substantiating the importance of these studies to the field of orthopaedic medicine.¹⁷^,¹⁸

One of the hallmarks of the RCT is the ability to eliminate selection bias and potentially regulating confounding variables, provided the appropriate level of power is achieved.¹⁵ RCTs are ideal for evaluating adverse effects and effectiveness of new interventions. Study participants in an RCT are in a defined population and randomized into a treatment or control (placebo) group. The treatment group could be a new or existing treatment, and the control group could be an existing treatment or no treatment at all. Study participants are followed prospectively and results, by treatment group, are compared. Blinded RCTs provide the highest level of internal validity with randomization, providing balance between measured and unmeasured variables associated with outcome.¹⁹ The primary advantage of an RCT is that they can provide evidence for causality by avoiding selection bias, selection by prognosis, and by ensuring balance between the treatment groups.²⁰

Estimates suggest that over 18,000 RCTs are published each year. However, systematic reviews and clinical guidelines frequently conclude that there is limited evidence to support study findings.¹⁹ While RCTs may provide internal validity, there is a potential for low external validity given limitations of identifying and enrolling enough patients. Additionally, depending upon resource constraints other circumstances, an RCT may not be economically feasible or ethical. As such, the use of other designs may be warranted.

Cohort studies compare groups with comparable demographics or exposure and follow them retrospectively or prospectively. These studies are suitable when approximating incidence, which represents the proportion of new cases of a disease within a specified time period. Typically, cohorts are described with specific risk factors and followed prospectively to observe outcomes. In orthopaedic studies, this design aids in determining prognosis of a given risk factor.

Retrospective case-control studies compare cases (patients with the disease of interest) and controls (patients without the disease) with respect to their level of exposure to a particular risk factor. While cohort studies select on exposure status, case-control studies select on disease status. Exposure differences between cases and controls are helpful in finding risk and protective factors associated with outcome. The most challenging part of this study design is defining the base population and selecting the appropriate, representative controls. This research design has been referred to as the “house red wine” of study design because it is more modest, has less risk, is inexpensive to conduct, and surprisingly good when compared with other designs.²¹ These studies are typically longitudinal and observational. The odds ratio, the primary outcome measurement in case-control studies, is defined as the odds of disease in exposed individuals compared with the odds of disease in unexposed individuals.²² The odds ratio is a good approximation of the relative risk when used in reporting rare diseases or events and can be determined from both case-control and cohort studies. However, the odds ratio should be used with some caution as it can overestimate the effect of events that occurs more than 10% of the time.²³

Descriptive Studies

Descriptive studies include case report, case series, correlational (or ecological) studies, and cross-sectional studies. A case report is generally made by a clinician or group of clinicians on a seminal patient. They are used to document rare medical occurrences and highlight initial clues of an emerging disease or deleterious effect of a specific exposure. A case series is a larger version of the case report and describes a disease process or complication in a group of 10 or more patients.¹⁵ Within the orthopaedic literature, case series (Level IV) evidence remains the most commonly reported research.²⁴

Correlational studies, used to determine the potential relationship between two variables, use correlation data to represent average exposure levels within a particular population rather an actual individual levels. They are relatively easy to conduct (with the appropriate data), are low cost, and can help with hypothesis generation. However, the prevalence of correlation does not imply valid statistical association and relationships observed at the group level may not be applicable to individuals (ecological fallacy). Additionally, correlational studies also do not account for effect of potential confounding variables.²⁵

Cross-sectional studies, used to examine the relationship between health outcomes and other variables, are often regarded as a “snap shot” of the health experience of a population at a given point in time. Because exposure and disease status are determined simultaneously, causality cannot be determined.¹⁵ Cross-sectional studies are typically used to provide information on disease prevalence (proportion of cases at any given time point) and can be used for hypothesis generation about disease/exposure relationship.¹^,²⁶ The advantage of conducting this type of study is that they provide a general description and scope of a problem. Additionally, they are relatively low cost and can be completed within a short period of time. Results from cross-sectional studies are often used for health service evaluation, planning, and resource allocation. An example of a cross-sectional study is reviewing the incidence of prosthetic joint infections after a total knee arthroplasty. In all of the aforementioned study designs, orthopaedic research investigators use a variety of tools to collect important exposure and outcome data about their patients. Patient-reported outcome measures (PROMs) are commonly used in orthopaedic clinical practice and research to track patient response to management and guide clinical decision making.²⁷

Only gold members can continue reading. Log In or Register to continue