Lessons in Orthopaedic Leadership: An AOA Podcast
Lessons in Orthopaedic Leadership: An AOA Podcast
Revolutionizing Orthopaedic Residency Training with Dr. J. Lawrence Marsh: From Time-Based to Competency-Based Education
Unlock the future of orthopaedic residency training as Dr. Douglas W. Lundy sits down with Dr. J. Lawrence Marsh, a leading voice in the field and chair of the Department of Orthopedic Surgery at the University of Iowa Hospitals and Clinics. Dr. Marsh shares a treasure trove of insights on the paradigm shift from a time-based to a competency-based education system, already gaining momentum in countries like Canada, Australia, and the UK. Listen to Dr. Marsh's firsthand experience and wisdom on what these changes mean for the next generation of orthopaedic residents in the US and how they are set to revolutionize medical training.
Discover the intriguing challenges of granting more independence to exceptionally skilled surgical residents and how this can widen the competency gap between them and their peers. We explore the logistical and operational hurdles in transitioning to a competency-based training system and why a well-defined curriculum is critical. Organizations like the ACGME and orthopaedic boards are taking significant steps to implement this new framework, and we discuss the delicate balance needed to integrate competency-based attributes within the traditional time-based structure.
Stay ahead of the curve with the latest advancements in residency training, particularly through practice-based assessments and real-time evaluations using mobile technology. These innovations promise to significantly enhance skill acquisition and feedback, generating vast amounts of data to improve training standards. From basic models to cutting-edge VR simulations, we delve into the essential role of technology in preparing residents for complex surgical tasks. Finally, we contemplate the external forces driving rapid transformation in medical education, emphasizing the urgency for change within the next three years. Join us for an enlightening conversation that promises to reshape the future of orthopaedic education.
Welcome to the AOA Future in Orthopedic Surgery podcast series. This AOA podcast series will focus on the future in orthopedic surgery and the impact on leaders in our profession. These podcasts will focus on the vast spectrum of change that will occur as the future reveals itself reveals itself. We will consider changes as they occur in the domains of culture, employment, technology, scope of practice, compensation and other areas. My name is Doug Lundy, host for the podcast series. Joining us today is Dr Larry Marsh.
Speaker 2:Dr Marsh is a tenured professor and chair of the Department of Orthopedic Surgery at the University of Iowa Hospitals and Clinics. He received his bachelor's degree from Colgate University, his medical degree from Upstate Medical Center in Syracuse, new York, and trained in orthopedic surgery at Boston University. After completion of his residency, he served two years as university lecturer in orthopedic surgery at Oxford University in Oxford, england. He was the president of the American Board of Orthopedic Surgery from 2015 to 2016, as well as president of the American Orthopedic Association and the Mid-American Orthopedic Association. He's a member of many professional organizations, including the Orthopedic Trauma Association, american Academy of Orthopedic Surgeons and the National Board of Medical Examiners. He's past chair of the Orthopedic Residency Review Committee of the Accreditation Council of the Graduate Medical Education.
Speaker 2:Dr Marshall's clinical practice is devoted to orthopedic trauma and adult reconstruction, and he's developed techniques of minimally invasive articular fracture surgery. His research has focused on articular fractures and techniques of image analysis to assess the mechanical factors leading to post-traumatic osteoarthritis. His research has been funded by the NIH, the OTA, the Arthritis Foundation, the AO and the NBME. He and his co-authors were recipients of the 2011 OREF Clinical Research Award for their work on post-traumatic arthritis. Today, dr Marsh and I will discuss how the future in orthopedic surgery relates to changes in resident education, and I've got to know Larry well through my time with him on the board of the American Board of Orthopedic Surgery, where he focused a lot on this. So, dr Marsh, welcome to the podcast, sir.
Speaker 3:Doug thanks for having me and thanks for that intro.
Speaker 2:Yes, sir, my pleasure. So while we were on the board together and I was the sole private practice guy that really had no clue to what y'all in the depths of academia were discussing really had no clue to what y'all in the depths of academia were discussing you and a number of other folks were really discussing and talking about the changes in residency education and where this was all headed, and that's very salient to the whole purpose behind this podcast series. So, in a nutshell, your vision. Where do you think orthopedic residency training is going as the future continues to roll on?
Speaker 3:Well, doug, to cut it fairly quick, it isn't just orthopedics. I think all of graduate medical education is moving in the direction of competency-based education. That does not mean in the US, in my opinion, away from time-based education, and I can sort of explain the differences as we get into it. But I do think, led by colleagues across the world, in the English-speaking world, in the Canadian system world, in the English-speaking world, in the Canadian system, the Australian system and the United Kingdom, there have been pretty strong shifts that the quality of an individual resident should be measured by reaching competency benchmarks rather than some arbitrary piece of time or time designation. And you know, as you, you know, we are so embedded in it. We all educated, we educate our current residents and they come in as PGY1s and they expect, and we expect in orthopedics that five years later their educational program is done and we do need to dictate paragraphs to say that they are ready to practice general orthopedics and ready to go to fellowships. But the truth is, the real driver of the end of their education is time, and in orthopedics it is five years. And if you look at the board requirements, it's dissected into how many weeks you need to do each year and and we have looked at that to be able to accommodate various medical leaves or maternity leaves or things like that. But basically it is still all you're talking about how many times, how much time, how many weeks a year, and how many times how much time, how many weeks a year and how many years.
Speaker 3:And again, those of us that in orthopedics we know what it is.
Speaker 3:It's five plus one, it's five years and one year of fellowship, because the vast majority of residents are you should meet and you know somebody needs to decide what those standards are and then you need to decide how you're going to measure whether you've met them or not.
Speaker 3:But you should meet those competency standards to finish training. So that's sort of the big picture and you know that is happening in orthopedics around the pretty predominantly around the English speaking world. It's happening in Canada across all of graduate medical education. I don't know exactly where they stand now, but I think all programs have had to move to competency standards and in this country, in our graduate medical education and in orthopedics, most people interested in education should at least have a grasp on these concepts. We aren't there, we're still in a time-based system, but we are making steps towards thinking about how we can integrate competency and, as you know, in your years on the board, the board and the education committee on the board and the overall board has been very interested in what its role is in making that happen and I can also talk about that if we want to dive into that deeper.
Speaker 2:Sure, now in your time as AOA president, of course you were on carousel so you got to visit and you became friends with all the presidents of the English speaking orthopedic organizations. So to folks who are thinking, is this just you know, the board or the academic folks just trying to shake stuff up and do something different To your point, in your experience this has already been rolled out to a further degree across the English speaking world. Can you talk about that just a little bit and also about what you saw when you were on Carousel in terms of, or your interactions with the other orthopedic organization presidents?
Speaker 3:Yeah, I can. I can go there somewhat. You know, I think the Toronto program in orthopedics was a leader in really totally changing the paradigm and mostly eliminate timing. I don't know if they completely eliminated and changing their large orthopedic educational program to competency. It was probably close to just straddling the first and second decade of the 2000s and they had enough data in 2013 to put it in the JBJS and, as I recall, a lead article and I remember it well because I got asked to write an editorial about it and it was really eye-opening at that time, 10 or 11 years ago, and you sort of looked at it. You said you know this really makes sense and it's really interesting that they are doing it.
Speaker 3:I think in the UK for years it's been modestly time variable. There may be other things in their system other than just competency and assessments that keep trainees in posts and have it be variable time when they move on to be consultants. And during my time on the AOA and in the year or two afterwards, because I went back there the year after I was on the AOA, and in the year or two afterwards, because I went back there the year after I was on the AOA, they were also. It's the Australian AOA. They're both AOAs. They were heavily embraced into trying to move their orthopedic graduate medical education to competency-based and in the time I spent in that country I was very impressed with their efforts towards education and their dedication to try and move it in that direction.
Speaker 2:Now you brought this up a little bit, but to really clarify the issue, what exactly is competency-based training addressing that time-based education is lacking on? You talked about it briefly, but to really just focus in on it, what is the core issue that this is? Fixing that our current system is failing.
Speaker 3:Well, doug, I think we would all have to agree that our residents, when they come, in when they train and when we finish with them are variable.
Speaker 3:They're variable in multiple ways. They're variable in how they interact with faculty. They're variable in many, many things. But they are variable in their operative skills. They're variable in their knowledge. They're variable in their professional behaviors.
Speaker 3:And you know, I do think we're comfortable that most of them get to a good endpoint. Some of them are a struggle. Anybody that's been in education struggles. Some of them get there smoothly and quickly and easily and of course, those are the ones that we love because they can have increasing independence towards the end of their training. And you know one of my major themes, george, just in thinking about this whole thing, that as they get increasing independence because they're really good, what happens to them? They get increasing independence because they're really good, what happens to them? They get better. So to some extent you widen the gap between the gifted trainee and the person that's struggling more because you give the gifted trainee more graduated responsibility, more independence.
Speaker 3:Take it to the operating room. You know who gets to do the most cases. Who do you give the most of your case to? Who do you not scrub and watch them do the case? The resident that's good. Who do you take more of the case away? Who do you always scrub with? The resident that has less skills? So you know, if you think about it that way, there are things in our training paradigm that widens the skill gap or widens the competency gap and I think to anybody that thinks about it those things make sense. And if you put that on it, you know.
Speaker 3:Do you say that that arbitrary amount of time again for us it's five years is the right time to release residents in independent practice, or should the person that was really good have been released a year before?
Speaker 3:You know why do we keep them in the system as a worker? Well, we keep them in because they are partially workers and we pay them to do that and we need that labor. But if you step away from that, you say just in the educational program, why should they stay? And of course that's some of the challenges that we have in our system of really moving fully to competency education. Because they are partially workers and we need them to work and floating them in and out based on competency standards has its challenges. And you know, I can at some point in this I can tell you where I think it's going. But I do think, if you can step out of where we are and say, does that make sense that our graduating residents should have met standards rather than have met some time criteria, it makes sense and that at least for some of our cohort it would be good. They don't need to stay an extra year, or they do need to stay yet one more extra year because they need to catch up and do other things.
Speaker 2:So I could see the operational folks that are suppose you have four residents per year and each of them are on three-month blocks. And Dr Smith, she gets it. She crushed adult recon, she's got it all figured out and she's ready to move on. But her next rotation would be trauma and Dr Jones, who's on trauma, is just struggling to get through. So you can't move him on because there's no place for her to go there, because you're still working with the one that's more difficult operationally. How do you see this going out and how do other successful programs who have done this internationally? How do they make this work?
Speaker 3:Well, I can tell you what I think it will mean for us. I can't see in my career or the truth is, in a career of somebody 10 years behind me that we will eliminate time-based training and I think for sort of the example you gave but you can even talk it at a higher level Again who's going to pay for residents that need to do extra years, right, how is the work system going to deal with residents that you've? You know, say you have four, as you said, and two of them are great. You're just going to let them go in our system. That isn't going to work, or at least not with that 10-year horizon. I don't think it's going to work.
Speaker 3:On the other hand, I think there are important attributes of the competency-based system and I can tell you what I think about those that can be layered onto a time-based system that will improve our education, and I think those are relatively easily within reach and, as you know, our board has sort of started to push some of those forward, of started to push some of those forward and, in my opinion, things like the board has done has to come from one of the organizations that can require things, and there's two organizations One is the ACGME and for orthopedics, the orthopedic RRC, and the second is the board, because nobody is going to just open their arms and say we're going to just start to do this, or at least relatively few. They have to be required and our board has made steps in that direction.
Speaker 2:So what are these attributes that you were talking about that are present within competency-based, that could be layered over time-based?
Speaker 3:So if you want to say that you should train to competency, you need to ask the question competent in what? Right? And that means you have to have a curriculum. My opinion, and orthopedics in a lot of ways over the years that you and I have been in it, have resisted that concept a little bit. You know we've got a part one board exam. You should immerse yourself in all the areas of orthopedics. You don't want to confine your knowledge acquisition. And curriculum was almost a bad word. But the truth is if you say I want my resident to train until they're competent, you have to say okay, competent in what? What do they need to do? What skills do they need to have? What knowledge do they need to have? And I again, I think pushes in the direction that there should be some sort of curriculum of what knowledge and skills they need to acquire to be a good general orthopedist is a good direction. And, as you know, the board has focused its blueprint of the part one exam and made it more available and is wrestling with an assessment-based skills curriculum and is trying to make steps in that direction. And the RRC has broadened its procedural minimums, so now it's gone from just 15 to I don't know the exact number they're going to and about to roll out, and again, to me those things start to approach being a curriculum and it makes sense to me. And then, once you have a curriculum, you say, okay, you have to be competent in these things. How do you know you're competent? Well, you have to have assessments and ideally, you know you have some good workplace-based assessments, so assessments of the residents actually doing the activities they need to do, whether it's surgical cases or whether it's indicating a patient, or whether it's in the emergency room or a variety of things. You have good assessments and then, ideally, you start to move the assessment of their knowledge into residency training rather than just at the end of residency training. You know why should you wait till they're gone from training and say now your knowledge is good, or the four or five percent of you is not? Well, what then? You know you're, you're out into your training. So I think both of those things that are fundamental to competency based training that you have to have a curriculum competent in what? And you have to have assessments that you have achieved those competence in that curriculum are good things and I think those can be layered on to what remains a fundamentally time-based program and if I understand what Toronto is really doing, so again, that's the a dozen years or more blazing a trail in this in a large orthopedic program, blazing a trail in this in a large orthopedic program. I think they actually have a fair amount of time that is still in their competency-based program that's 12 or 15 years mature.
Speaker 3:I don't want to speak too much for it because I'm not an expert, but I think I've heard Mark who say that that, yeah, most of ours are in whatever I think they're also five years, but they have a few that do this or that. I mean in our system. Again, for the example you already gave, floating in and out is not going to be very easy, other than in unusual circumstances, unusual circumstances. I also think a better curriculum and assessment of that curriculum will help programs identify residents that are struggling earlier and better and then hopefully with the goal of remediation and success at the end. And again it just makes sense that we should be able to do these things better than what we do.
Speaker 3:So that's sort of my vision of where it is going. I think it was a pretty shared vision across a lot of the board and since you and I have stepped out of it. You know that at least part of that vision the board has embraced and has started to require. It has and again we can talk about that a little bit if you want and has a number of programs that have stepped into some of these assessments and has now required that for the graduating class next year and for the incoming class that they would be required to participate. So it is starting to happen and in this country it is making orthopedics at least approaching being a leader in this sort of steps toward competency and I think there is a fair amount of looking with admiration at what orthopedics is trying to do through our board initiatives.
Speaker 2:Yeah, so you brought up some of the things that the ABOS board was directing on that and, as you said, both of us have termed off of the board and you started it. I remember you and Ann Van Heese and some of the other folks really dug in and got this thing going. What do you think the ABOS's current requirements and activities are regarding this?
Speaker 3:Well. So their skills assessment program is probably the most mature. They call it KSB. I'll give Terry Peabody credit for those terms knowledge, skills and behavior and Terry should be credited with also pushing this initiative along with Anne, and it's sort of I mean my look at those terms. It takes the six competencies that we live under in ACGME and Boyle puts them together such that it's a package of three, which is a little bit easier. So you should acquire knowledge, you should acquire skills and you should have good communication skills and professional behavior. So that's knowledge, skills and behaviors.
Speaker 3:The board, during our time, worked on initial initiatives in all three of those areas and all three of those initiatives still exist and have gone forward. The skills one is the one that is really making a big step. So skills is practice-based assessments, mobile phone, optimized resident requests and evaluation of a real-time case that they're doing. Of a real-time case that they're doing. We have an OP scoring form that then comes to the faculty on their phone and it literally is a 30-second job to fill out. Again, as close to real-time assessment of performance as you can get. You can also dictate or type in formative feedback in a type inbox and to me this is all nothing but good. And as the residents acquire multiple assessments, it starts to be the level of pilot, leading to publication in JBGS, leading to more and more promotion at meetings, leading to, I don't know, roughly half of the programs or maybe a little bit less, embracing this.
Speaker 3:And the board, I believe at its last fall meeting, has required that they were going to mandate this. So it's going to go from elective embracement to it will be mandated. And again, neither you or I are on the latest requirements, but I think, knowing it and my residency, that a year from now the PGY-5s in that year will need to accumulate a couple of assessments a week for their year in the PGY-5, adding up to 80 years or some assessments like that. And the PGY-1s will need to start as interns, interns getting a small number of assessments on their orthopedic rotations and then, as they go to two, three, four and five. So it will be required that you have to get some assessments and that you will not be permitted to sit, for your part, one aboard if you haven't done that. And then my vision and I haven't heard the inner working of the board, but I have talked to John Harris is as it becomes required in all programs around the country.
Speaker 3:There will start to be lots and lots of data produced. And as lots of data is produced, you'll then be able to dissect things out like which procedures and how many procedures, which year, and what does the average PGY-3 get to in ankle fracture, for instance, and what does the bottom 10% of PGY-3s get to an ankle fracture, how many? And then what assessment level are they getting? And once you start to have that data in thousands and thousands and thousands, you could start to raise your requirement higher. For instance, you could pick ankle fracture and you could say you know as a PGY-1, nothing's required.
Speaker 3:As a 2, you should say you know as a PGY1, nothing's required. As a two, you should have you know who knows four to six exposures to ankle fracture. And as a three, you should start to reach some level. You know, maybe not taking the independent practice, able to be on his or her own box, but that you should start to reach some level. And then you could also decide well, what else should it be in trauma other than ankle fracture? That isn't enough. So you could pick six or eight things and if you have a lot of data you could start to do those with a feeling that what you would further require is based on good information and data. So how long it'll take to go there I don't know, but once it's required, a lot of data will come quickly.
Speaker 2:There's also been a lot of work on skill simulation and assessment through simulation, which much more reasonably standardizes the actual activity across different platforms. Right, I think you were extensively involved in that too. Where do you see that going in the future?
Speaker 3:Well, we just did a symposium at Cord AOA last Saturday morning on that particular topic. So the board when I first came on the board, I was fortunate enough to be involved with that and the board and the RRC got pretty deep into simulation consultancies. With general surgery, we were felt to be modestly behind or quite behind in orthopedics and the board and the RRC stepped up to the 2013 program requirements that required a laboratory-based skills training program for PGY-1. Having been on the podium some at that time there was a modest amount of pushback typical unfunded mandate and that sort of thing but I think most, looking back on it, would say that it was the right thing to do. Since that time, the board has continued to look at it. It's continued to sponsor think tanks and symposium. It has actually funded three relatively large grants. It's brought the grant recipients together to look at what they're doing and I think the board has just slightly hesitated on making another requirement step in simulation. So, while simulations have advanced over the last 10 years, there's some pretty slick things. Now there are programs that are using a number of these slick things. There's a lot of computers involved. So it isn't just sawbones and cadaveric dissections. Vr is coming. You know exactly where it stands in the hierarchy, but it gets better all the time and it is coming.
Speaker 3:You know, it's my belief that to take another big step to nationally moving it forward it has to be required Again because it costs money and it takes time. Our residents are so busy in the operating room. But I do think back to you the origin of your question how does this advance the whole cause of competency-based training? We would all agree we have a highly technically skilled profession. We would all agree that it isn't any easier to let junior residents be operating on our patients than what it used to be. Pressures for time and billing and moving things along and all of those types of things, as well as quality and safety, and therefore advancing junior level learners on relatively simple, inexpensive simulations in the lab rather than having them be retractor holders as second assistants in the operating room makes a ton of sense.
Speaker 3:And then when they come to you as a PGY-3, they are more ready to go. They are already more skilled. You know those are all aspirational things of where we would like to be, I think, other than simulations advanced in some programs. For instance, on the symposium was the Penn State group, robert Gallo, and you know, and they've done a lot of simulated patient-based simulations. So to move out of the technical skills and move into, how do you get your residents to be good at informed consent and communicate well and deal with challenging interactions and all those kind of things that you kind of learn on the job and some learn it better than others. So, anyway, lots of interesting things going on in pockets. But to really widely change things they have to be required and I think at some point the board will step up to that.
Speaker 2:Are there other parts of technology that you see impacting resident education in the future?
Speaker 3:Oh, I don't know. You know, doug, there's all sorts of AI things and stuff that are all over the news, and you know, ai being able to score in the 65th percentile and OITE exams, and you know, aren't these tremendous values for education? I don't know if I have a real square vision. Do it Exactly how those things will advance to education? I don't know. I mean, to some extent you've got to put the knowledge and skills in your brain. Ai can't do it for you, so I don't know. That's sort of a quick thought.
Speaker 2:But to summarize what you were saying, within the next 15 to 20 years we'll move closer to competency-based education, having to figure out the whole timing basis of it and how that affects fellowships and jobs and everything else afterwards, or lengthening folks that get there, the skills assessment and, of course, the simulation moving along as well as other technology. There's a lot going on in that area.
Speaker 3:Yeah, and I'll just mention one other, doug, that steps directly out of the area of education but could impact it big time is money. Oh yeah, so if the government pulled money out of resident education because of course, as we know, the government mostly supports the resident salaries not exclusively, but mostly if they pulled it back or got more involved in some way or another and, for instance, asked the question do you really need to train orthopedic residents for five plus one? That would change things in a hurry. Oh yeah. So that would you know. For instance, if you were going to have to turn five plus one into three plus two. Or you know you're training a totally subspecialized workforce, why do they have to do five years of general orthopedics? And that would lead to tons of arguments.
Speaker 3:But if you pulled money out of it, we would figure out how to change it in a hurry and it would force a curriculum big time. Okay, but only got three years. What do they need to accomplish in three years? And then, have they really accomplished it? And then get them off for two years into their subspecialty work. I mean, I think those things could happen. I don't think we as educators will make them happen, but if things happen with the government, either from money and training or from maldistribution of workforce and some of the healthcare challenges that we face, these things could happen.
Speaker 2:That is very well said because that to your point and you said this multiple times is that it's not just the innovation within the sector, but it's the external forces saying you have to move in this direction. That will really impact change more quickly.
Speaker 3:Totally agree.
Speaker 2:Well, this has been absolutely enlightening for me. Great to talk again with you, larry, and I really appreciate your insight on this. As I said, I got to sit in the seat and listen to you and Terry and Ann and as well as others April and others talk about these things when I was still trying to learn a lot about it, and I'm glad that we were able to share this with the listeners to the AOA podcast series. So, dr Marsh, thank you for being on the podcast, sir.
Speaker 3:Doug, thanks for having me. I enjoy chatting about these things. They're really interesting.
Speaker 2:Yes, sir, and I look forward to other speakers and engaging with y'all again on the AOA Futures in Orthopedic Surgery Podcast Series.