Depiction of EPAs, competencies, and milestones
Orthopaedic Surgery was one of the initial seven specialties to adopt milestones as a framework for resident assessment. Workgroups, consisting of experts in orthopaedic surgery, identified 16 milestones, based on key surgical procedures in orthopaedic surgery. The milestones were reflective of knowledge and skill in Medical Knowledge and Patient Care . In addition, an additional nine milestones were written for four of the generic core competencies (P, ICS, PBLI, and SBP). However, the orthopaedic surgery milestones have not been outlined for the entire six domains of competence and sub-competencies required for completion of training.
In fact, the orthopaedic surgery milestones that have been developed have met with criticism. Van Heest and Dougherty (2015) argued that some procedural milestones may need to be modified to reflect more common procedures in orthopaedic surgery, and advocated for more research on appropriate assessment methods and tools. In addition, these authors argued that the milestones should undergo further research to establish them as valid metrics at the individual and program levels. Research must continue to establish strong and valid assessment tools for the milestones that are pertinent to surgical practice .
Backward Design: A Method for Writing Competency-Based Curriculum
Conducting a Needs Analysis
The first step in creating a competency-based curriculum is conducting a needs analysis to determine the key concepts and skills that are taught on a specific rotation or educational experience. A needs analysis should incorporate accreditation requirements, current gaps in the curriculum, and best educational practices in the specialty. A needs analysis will determine where a particular rotation or educational experience curriculum may need to be revised. Some programs may have an educational committee that can be charged to do this work. Other programs may need to create one and should include the program director, core clinical faculty from different subspecialties, and residents. Newer program directors may need to consult with or survey their subspecialty peers at the institution. Consulting with program directors from other institutions may also be helpful.
Writing competency-based curriculum for other educational experiences in the program is strongly suggested. These educational experiences may include didactics, simulation, journal clubs, quality improvement curriculum, etc. Using a standardized process, such as the one described, may be helpful. Utilizing the help of others, such as the Educational Committee within the program, key clinical faculty, senior residents or fellows, and program coordinators, may reduce the burden on the program director.
It is important to refer to the ACGME accreditation requirements in order to ensure that the curriculum meets appropriate standards, and to remain aware of ACGME updates to these standards. The ACGME Common Program requirements are a set of requirements common to all training programs, regardless of specialty. These requirements can be found on the ACGME website . The specific curricular requirements for Orthopaedic Surgery Residency and Orthopaedic Surgery Fellowships are found on the ACGME website . Program requirements are categorized as core, outcome, or detail. Core requirements refer to essential educational structures, processes, or resources required across all graduate training programs. Outcome requirements refer to measureable educational outcomes characterized as the knowledge, skills, or attitudes residents or fellows should demonstrate at key points in training. Detail requirements describe specific structures, processes, or resources required to meet the core requirements.
Other important sources to review include:
Current curriculum for each rotation.
A focused discussion, with residents and core clinical faculty, to determine whether there are content gaps, procedural gaps, or opportunities for improvement in the current curriculum: Information gleaned from the ACGME annual review process may also help to identify curricular gaps.
Feedback from Clinical Learning Environment Review (CLER) visits can be useful.
A discussion with the Designated Institutional Official (DIO) to determine if there is a plan to develop institutional curricula, with a surgical focus, for the four competencies: Interpersonal and Communication Skills, Professionalism, Practice-based Learning and Improvement, and Systems-based Practice.
A review of board certification requirements.
Curricula from other institutions.
Writing competency-based curricula for an orthopaedic surgery residency is a time-consuming task, but can be made easier by using an instructional design model to align outcomes (milestones) with objectives and competencies. Backward design is a three-step goal-directed process to align outcomes and curriculum with national standards . Backward design is predicated on the following principle identified by Stephen Covey: “Begin with the end in mind” . It is important to know the end result (educational outcome) and to construct curriculum to meet that result. In medical education, curriculum is generally constructed beginning with content and, then, attempting to link content to outcomes. This method often falls short in identifying specific outcomes, and communicating those outcomes to learners. The backward design model consists of three steps: identify desired results (outcomes such as milestones), determine acceptable evidence (assessment methods or tools), and plan learning experiences and instruction (objectives and teaching strategies). Backward design encourages an educational focus on the outcomes of a particular educational experience, and is an instructional design model directly in alignment with competency-based education. Figure 3.2 describes a step-by-step guide that can be used by program directors and faculty to create competency-based curriculum.
A step-by-step guide to create competency-based curriculum
This guide contains components of backward design principles, which have been blended with ACGME accreditation requirements, providing a systematic process for creating curriculum. Program directors should begin by including a broad description of what the resident or fellow might learn during this rotation. The purpose of this description is to help orient the learner to the overall structure of the rotation. It may also be helpful to include an overall description of the setting, any specific requirements, level of supervision, any educational resources provided, and any other important features of this rotation. Once the description is written, the step-by-step guide can be useful in articulating the goals and objectives and educational outcomes of the rotation or educational experience.
Milestones (Identify Desired Results)
Resident performance on the ACGME Orthopaedic Surgery Milestones is the educational outcome expected at the end of each rotation. The milestones may vary from rotation to rotation, and not all milestones or competencies may be addressed on a particular rotation or educational experience. Program directors should list the specific milestones (outcomes) that are covered by the rotation in their curriculum document. Many orthopaedic surgery rotations may last a few months, and occur multiple times during training. The curricular document should list each iteration (PGY2, 3, etc.) and reflect the milestones for that particular iteration. The milestones described for the rotation may remain fairly consistent over the training period; however, resident or fellow progress on the milestones should increase as they become competent or proficient. Specifying the milestones and level of performance to be reached by trainees helps them develop a road map for learning.
Assessment Methods or Tools (Identify Evidence)
This second step in the backward design model articulates appropriate assessment methods or tools to provide evidence for attainment of the milestones covered on the rotation and link the methods or tool to the milestones. The linking can be done by using codes or abbreviations. Multiple assessment tools, reflective of an overall assessment system, should be used, and aggregated to form a snapshot of resident or fellow performance on particular milestones. Important assessment methods to consider include longitudinal direct observation tools, multisource feedback, and performance audits. These assessment methods capture a picture of a resident’s actual performance. Additional assessment tools might include case logs, in-training examination scores, patient surveys, simulation, and rating scales. The rotation curriculum should list the specific assessment methods or tools so trainees can identify how they will be assessed and the certifying body can see the process for evaluation. The ACGME Milestones Guidebook provides an overview of milestones, assessment, and best practices related to creating an assessment system . Assessment tools should be specified for each time residents or fellows rotate through a particular rotation. It may also be helpful to include statements about when and how feedback occurs. Helping residents incorporate feedback into their own learning plan helps them to refine knowledge and skills.