The most reliable information of effectiveness of medical interventions can be obtained from RCTs, which create the basis for systematic reviews on effectiveness. However, RCTs often exclude patients who are commonly treated in ordinary clinical practice and, thus, the results may not be generalisable to all patients obtaining the studied intervention. In addition, the units and clinicians providing the interventions have usually very good or even exceptional expertise in treating the patients eligible for the trial. Thus, the efficacy shown by trials and systematic reviews is often better than the efficacy provided by average healthcare units for average patients. Therefore, the extrapolation of the results to ordinary clinical settings is often uncertain, and there are no formal rules about how to do this; rather one has to rely on one’s own clinical expertise (Croft, Malmivaara, & van Tulder, 2011).

The difference between efficacy in RCTs and efficacy in ordinary clinical practice has been demonstrated well in the field of vascular surgery, where clinical documentation for some procedures is most important in order to maintain low-adverse risk events. For example, in two RCTs assessing the perioperative mortality of carotid artery surgery, the mortality was 0.1 % and 0.6 %, respectively. However, clinical follow-up data from real-world circumstances showed that the mortality was much greater compared to that from these two RCTs. In the hospitals participating in the trial, the mortality was 1.4 %, and in other hospitals 1.8 % (Wennberg, Lucas, Birkmeyer, Bredenberg, & Fisher, 1998).

Hypotheses in RCTs are based on a PICO-type research question, which constitutes a clear framework for formulating the aim for these studies. RCT designs enable the most valid assessment of single interventions. However, in the real healthcare clinical practice, the most important question from the patient’s perspective is quite rarely based on one single intervention but for the effectiveness of the whole clinical pathway (episode of care). For example, in an RCT, a drug-eluting stent may be compared with a bare metal stent for acute myocardial infarction, but this procedure represents only one part of the treatment process in the hospital taking care of the acute phase. Furthermore, angioplasty with stenting constitutes only a tiny part of the whole clinical pathway consisting of follow-up, treatment and rehabilitation in the hospital and further in the primary care setting. Thus, in the real world of clinical practice, the effectiveness is usually not determined solely by a single intervention, but by how well the whole clinical pathway works. Furthermore, in some cases, RCTs cannot be used for ethical reasons. For these cases, observational studies are the only feasible option to provide data on efficacy (Häkkinen, 2011). In a supplement issue of the PERFECT project on measuring performance of healthcare episodes (clinical pathways), the following definition has been given: “An episode of care refers to the entire treatment pattern from the beginning of the (e.g. acute stage of the) disease to the end of the treatment over any organisatory boundaries to solve the health problem at hand in a specific time frame” (Peltola et al., 2011). An example of a clinical pathway starting from admission to a hospital (e.g. because of a hip fracture) is shown in Fig. 26.2.

To sum up, the knowledge of the efficacy and costs of a single procedure for a particular disease is not enough to evaluate the outcome dependent on the whole clinical pathway. RCTs and systematic reviews will always be unable to answer the question of what the effectiveness of a particular treatment unit is in comparison with other units treating similar patients. Even for single treatments, the assessment of generalisability of the findings from RCTs to ordinary clinical settings is often difficult, and there is no method enabling a quantitative extrapolation of the efficacy data to real-world circumstances. It is obvious that, besides evidence arising from clinical expertise and clinical science, one needs valid data on what happens in ordinary clinical settings. Information from all the four levels of REM should be used for advancing effectiveness in the ordinary clinical practice for ordinary patients.

The questions to ask to improve the performance at the four levels of REM are presented in Fig. 26.3 and clarified further in the text below.

There is evidence that many patients do not receive appropriate care, as revealed by RCTs and systematic reviews (Grol & Grimshaw, 2003). Implementation of new evidence or clinical guidelines into practice has been shown to be resource demanding and, even then, seems to succeed only partially (Grol & Grimshaw, 2003). Furthermore, due to the limited resources in health care, one must also question the effectiveness in relation to the invested resources—i.e. cost-effectiveness. Translating cost-effectiveness data in a valid way from an RCT to another setting, especially to another country, presents substantial problems. So far, only first studies have been published on determining in which cases this translation is justifiable (Knies, Ament, Evers, & Severens, 2009; Welte, Feenstra, Jager, & Leidl, 2004). For these reasons, it seems evident that there is a need to gather performance data at the ordinary healthcare level.

The third level of REM consists of standardised documentation of the performance of healthcare units and continuous quality improvement measures based on the performance data. The aim of the performance assessment is to increase effectiveness and cost-effectiveness of care in routine clinical practice. Increasing cost-effectiveness means that resources are allocated to those patient groups and interventions which show the best effectiveness, as the resources always have their limits. Kaiser-Permanente in the United States provides a good example of a healthcare provider, which continuously documents the performance of its services (https://​healthy.​kaiserpermanente​.​org). In Canada the state of Saskatchewan has implemented in the year 2011 a clinical pathway for low back pain patients. The spine pathway describes assessment and treatment processes for improving the performance of family physicians and other health providers and to expedite the care and specialist referral for the patients. A standardised form has been developed for primary care practitioners to be filled when referring patients with low back pain to a Spine Pathway Clinic (http://​www.​health.​gov.​sk.​ca/​back-pain). Also national register-based methods have been developed for performance assessment—the PERFECT project will be described in this chapter.

It would be optimal to assess the performance of the whole clinical pathway covering primary, secondary and tertiary care. If feasible, the performance indicators should be those for which there is scientific evidence that a change in the care process leads to improved outcomes, the indicators capture whether the process is indeed provided, the process indicator is sufficiently near the important outcomes, and there is low or no risk of inducing adverse consequences (Chassin, Loeb, Schmaltz, & Wachter, 2010). However, ideal documentation of patients, processes and outcomes is rarely possible. Thus, the documentation of performance should be started with the best available indicators; increasing such quality is not a project but a continuing process.

Information of accessibility of care, patient characteristics, diagnostic procedures and treatments, and treatment outcomes can be used for assessments of the performance of an individual unit by comparing changes in time or by making comparisons to other units that are treating similar patients. However, valid comparisons—especially for outcomes—necessitate that confounding factors at baseline can be adequately adjusted for. If possible, all the relevant outcomes for patients, and also adverse effects, should be documented. Obtaining a quality certification based on fulfilment of established criteria is a mark of quality and increases the transparency of the treatment processes: e.g. a status of a designated stroke centre warrants that internationally defined requirements for stroke care are fulfilled (Xian et al., 2011). Implementation science has produced evidence of the effective ways to promote the uptake of research findings into routine health care, and this evidence should be utilised (Rubenstein & Pugh, 2006).