1 Evidence-Based Research

10.1055/b-0035-122001

1 Evidence-Based Research

Hamish Simpson

Poor quality research has little or no value; therefore, it is essential that we ensure that any research we carry out is to a high standard. Guidelines have been published by several bodies, such as the Wellcome Trust and the UK Medical Research Council for Good Research Practice; these cover the ethical and data protection aspects of good research. For clinical research, several frameworks have been suggested in order to ensure a high standard of research. These include the CONSORT statement for the reporting of clinical trials and the principles of evidence-based medicine. Many of the requirements in these clinical frameworks can be considered in a preclinical research setting and if followed will ensure that the pre-clinical research is carried out to the highest standards.

1.1 Lessons to Be Learned from Evidence-based Medicine for Preclinical Research

In 1996, David Sackett wrote that “Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”1

Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values. Evidence-based medicine asks questions, finds and appraises the relevant data, and harnesses that information for everyday clinical practice. Evidence-based medicine follows five steps encompassed within the five “A”s:

Ask as answerable question
Find the relevant Articles (the evidence)
Critically Appraise the evidence (validity, impact, applicability)
Apply
Assess

The same steps can be applied to translational research. In particular, it is very important to formulate a clear clinical question from a patient′s problem.

Asking the right question can be difficult, yet it is fundamental to carrying out relevant translational research. One framework that has been suggested to help formulate the question for evidence-based medicine is “PICO.” This framework states that a “well-built” question should include four parts, referred to as PICO, that identify the patient problem or population (P), intervention (I), comparison (C), and outcome(s) (O). Not all translational research can be fitted into this framework, but it does stress the importance of starting with the right question and the necessity of having appropriate control groups.

The next two steps of evidence-based medicine are also entirely relevant to preclinical research, namely finding the relevant previous publications and critically appraising this literature to ensure that the experimental design is optimized. In addition, it is important that the model, whether it is biomechanical, in vitro, in silico, or in vivo, is valid for the question being addressed. For instance, although muscle structure is similar in different mammals, the structure of bone and its propensity for remodeling vary in different mammals, and it is essential this is taken into account in ensuring the model is valid (see Chapter 42).

1.2 Lessons to Be Learned from Clinical Trial Design for Preclinical Experiments

The second framework described for the reporting of clinical trials but also of relevance to preclinical research is the CONSORT statement outlined in Table 1.1. The items of particular relevance to preclinical research are outlined in Table 1.2.

**Table 1.1** **The CONSORT Framework**
Title and abstract
	1a	Identification as a randomized trial in the title
	1b	Structured summary of trial design, methods, results, and conclusions (for specific guidance, see CONSORT for abstracts)
Introduction
Background and objectives	2a	Scientific background and explanation of rationale
	2b	Specific objectives or hypotheses
Methods
Trial design	3a	Description of trial design (such as parallel, factorial) including allocation ratio
	3b	Important changes to methods after trial commencement (such as eligibility criteria), with reasons
Participants	4a	Eligibility criteria for participants
	4b	Settings and locations where the data were collected
Interventions	5	The interventions for each group with sufficient details to allow replication, including how and when they were actually administered
Outcomes	6a	Completely defined prespecified primary and secondary outcome measures, including how and when they were assessed
	6b	Any changes to trial outcomes after the trial commenced, with reasons
Sample size	7a	How sample size was determined
	7b	When applicable, explanation of any interim analyses and stopping guidelines
Randomization
Sequence generation	8a	Method used to generate the random allocation sequence
	8b	Type of randomization; details of any restriction (such as blocking and block size)
Allocation concealment mechanism	9	Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned
Implementation	10	Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions
Blinding	11a	If done, who was blinded after assignment to interventions (e.g., participants, care providers, those assessing outcomes) and how
	11b	If relevant, description of the similarity of interventions
Statistical methods	12a	Statistical methods used to compare groups for primary and secondary outcomes
	12b	Methods for additional analyses, such as subgroup analyses and adjusted analyses
Results
Participant flow (a diagram is strongly recommended)	13a	For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analyzed for the primary outcome
	13b	For each group, losses and exclusions after randomization, together with reasons
Recruitment	14a	Dates defining the periods of recruitment and follow-up
	14b	Why the trial ended or was stopped
Baseline data	15	A table showing baseline demographic and clinical characteristics for each group
Numbers analyzed	16	For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups
Title and abstract
Outcomes and estimation	17a	For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval)
	17b	For binary outcomes, presentation of both absolute and relative effect sizes is recommended
Ancillary analyses	18	Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing prespecified from exploratory
Harms	19	All important harms or unintended effects in each group (for specific guidance, see CONSORT for harms)
Discussion
Limitations	20	Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses
Generalizability	21	Generalizability (external validity, applicability) of the trial findings
Interpretation	22	Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence
Other information
Registration	23	Registration number and name of trial registry
Protocol	24	Where the full trial protocol can be accessed, if available
Funding	25	Sources of funding and other support (such as supply of drugs), role of funders

**Table 1.2** **Components of the CONSORT Framework of Particular Relevance to Preclinical Research**
Abstract	1	Structured summary of trial design, methods, results, and conclusions
Background and objectives	2	Specific objectives or hypotheses
Methods	3	The interventions for each group with sufficient details to allow replication, including how and when they were actually administered
Outcomes	4	Completely defined prespecified primary and secondary outcome measures, including how and when they were assessed
Sample size	5	How sample size was determined
Randomization	6	Type of randomization; details of any restriction (such as blocking and block size)
	7	Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned
Blinding	8	If done, who was blinded after assignment to interventions (e.g., researchers or those assessing outcomes) and how
Statistical methods	9	Statistical methods used to compare groups for primary and secondary outcomes
Results	10	For each group, losses and exclusions after randomization, together with reasons (an experimental flow diagram should be considered)
Harms	11	All important harms or unintended effects in each group
Discussion	12	Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence
Funding	13	Sources of funding and other support (such as supply of drugs), role of funders

Of particular note are the statements about minimizing bias (see also Chapter 54). Frequently, this is not done in preclinical research,2,3 even when it adds little to the complexity of the design. For example, (1) randomization: Ideally the allocation of specimens (for biomechanical or in vitro work) or animals (for in vivo studies) should be randomized in a similar manner to patient randomization for clinical trials. (2) The assessments should be carried out in a blinded manner. For instance, if the number of positive cells on a histological section are being counted, the assessor should be unaware of which group the histological section has come from. (3) Multiple observers should be used if possible.

If the steps outlined for clinical research, which are relevant to preclinical research, are applied, the standard of the preclinical research will rise and with this the degree to which the preclinical studies can be applied clinically will increase.

For in vivo preclinical studies, an excellent fuller reporting guideline has been produced by Kilkenny and co-authors4: The ARRIVE guidelines (Table 1.3).

**Table 1.3** **Animal Research: Reporting in Vivo Experiments: The ARRIVE Guidelines**
Item		Recommendation
Title	1	Provide as accurate and concise a description of the content of the article as possible.
Abstract	2	Provide an accurate summary of the background, research objectives including details of the species or strain of animal used, key methods, principal findings, and conclusions of the study.
Introduction
Background	3	a. Include sufficient scientific background (including relevant references to previous work) to understand the motivation and context for the study, and explain the experimental approach and rationale. b. Explain how and why the animal species and model being used can address the scientific objectives and, where appropriate, the study′s relevance to human biology.
Objectives	4	Clearly describe the primary and any secondary objectives of the study, or specific hypotheses being tested.
Methods
Ethical statement	5	Indicate the nature of the ethical review permissions, relevant licenses (e.g., Animal [Scientific Procedures] Act 1986), and national or institutional guidelines for the care and use of animals, that cover the research.
Study design	6	For each experiment, give brief details of the study design, including: a. The number of experimental and control groups. b. Any steps taken to minimize the effects of subjective bias when allocating animals to treatment (e.g., randomization procedure) and when assessing results (e.g., if done, describe who was blinded and when). c. The experimental unit (e.g. a single animal, group, or cage of animals).
		A timeline diagram or flow chart can be useful to illustrate how complex study designs were carried out.
Experimental procedures	7	For each experiment and each experimental group, including controls, provide precise details of all procedures carried out. For example: a. How (e.g., drug formulation and dose, site and route of administration, anesthesia and analgesia used [including monitoring], surgical procedure, method of euthanasia). Provide details of any specialist equipment used, including supplier(s). b. When (e.g., time of day). c. Where (e.g., home cage, laboratory, water maze). d. Why (e.g., rationale for choice of specific anesthetic, route of administration, drug dose used).
Experimental animals	8	a. Provide details of the animals used, including species, strain, sex, developmental stage (e.g., mean or median age plus age range), and weight (e.g., mean or median weight plus weight range). b. Provide further relevant information such as the source of animals, international strain nomenclature, genetic modification status (e.g., knockout or transgenic), genotype, health/immune status, drug- or test-naïve, previous procedures, etc.
Housing and husbandry	9	Provide details of: a. Housing (e.g., type of facility, specific pathogen free; type of cage or housing; bedding material; number of cage companions; tank shape and material, etc., for fish). b. Husbandry conditions (e.g., breeding program, light/dark cycle, temperature, quality of water, etc., for fish, type of food, access to food and water, environmental enrichment). c. Welfare-related assessments and interventions that were carried out before, during, or after the experiment.
Sample size	10	a. Specify the total number or animals used in each experiment and the number of animals in each experimental group. b. Explain how the number of animals was decided. Provide details of any sample size calculation used. c. Indicate the number of independent replications of each experiment, if relevant.
Allocating animals to experimental groups	11	a. Give full details of how animals were allocated to experimental groups, including randomization or matching if done. b. Describe the order in which the animals in the different experimental groups were treated and assessed.
Experimental outcomes	12	Clearly define the primary and secondary experimental outcomes assessed (e.g., cell death, molecular markers, behavioral changes).
Statistical methods	13	a. Provide details of the statistical methods used for each analysis. b. Specify the unit of analysis for each dataset (e.g., single animal, group of animals, single neuron). c. Describe any methods used to assess whether the data met the assumptions of the statistical approach.
Results
Baseline data	14	For each experimental group, report relevant characteristics and health status of animals (e.g., weight, microbiological status, and drug- or test-naïve) before treatment or testing (this information can often be tabulated).
Numbers analyzed	15	a. Report the number of animals in each group included in each analysis. Report absolute numbers (e.g., 10/20, not 50%). b. If any animals or data were not included in the analysis, explain why.
Outcomes and estimation	16	Report the results for each analysis carried out, with a measure of precision (e.g., standard error or confidence interval).
Adverse events	17	a. Give details of all important adverse events in each experimental group. b. Describe any modifications to the experimental protocols made to reduce adverse events.
Discussion
Interpretation/scientific implications	18	a. Interpret the results, taking into account the study objectives and hypotheses, current theory, and other relevant studies in the literature. b. Comment on the study limitations including any potential sources of bias, any limitations of the animal model, and the imprecision associated with the results. c. Describe any implications of your experimental methods or findings for the replacement, refinement, or reduction (the 3Rs) of the use of animals in research.
Generalizability/translation	19	Comment on whether, and how, the findings of this study are likely to translate to other species or systems, including any relevance to human biology.
Funding	20	List all funding sources (including grant number) and the role of the funder(s) in the study.