Pay for performance and hip fracture outcomes

Listen to Mr Andrew Duckworth interviewing Prof. Matt Costa and Mr David Metcalfe about their paper "Pay for performance and hip fracture outcomes: an interrupted time series and difference-in-differences analysis in England and Scotland", published in the August 2019 issue of The Bone and Joint Journal.

Click here to read the article

Transcript

[00:00:00] Welcome everyone to one of this month's BJJ Podcasts. I am Andrew Duckworth and a warm welcome to one of two podcasts we're doing for the month of August from your team here at The Bone & Joint Journal. 

I'd like to thank all of our readers and listeners for the comments and support we've received so far for our podcast series, as well as to our authors and guest interviewers who have taken part so far. We really do appreciate all of their efforts. 

So far this year, we've covered a range of topics, including the role of robotic unicompartmental on knee replacement with our editor-in-chief here at the journal, Professor Fares Haddad. We had a fascinating dialogue between Ian Murray and Dr. Scott Rodeo on cell therapies in orthopaedic surgery, and more recently a series of podcasts to accompany our supplements from the American Hip and Knee Society closed meetings. We do hope these podcasts are improving the accessibility and visibility of the studies we published here at the journal to both you as our readers, as well as for our many authors.

As many of you know, over the next 20 minutes or so, we'll cover a range of topics from the chosen study, emphasizing the important points of how the work has been designed, the key findings from the study, how it potentially fits into your [00:01:00] day-to-day practices, with this month's discussion, I suspect, being relevant to many of our listeners. We also hope to give you a behind the scenes insight into how the authors have developed the study and give them an opportunity to put forward the key findings of their work.

 Today I have the pleasure of being joined by both Professor Matt Costa and Mr. David Metcalfe from Oxford, who in collaboration with Cheryl's team from Yale, have produced their paper, Pay for performance and hip fracture outcomes, an interrupted time series and difference in differences analysis in England and Scotland, which we published in the August edition of the BJJ. 

Welcome Matt and Dave, and a big thank you to both for taking the time to join us today. So, Matt, if I could start with you, as it is stated nicely in your paper, hip fractures are associated with high morbidity, high mortality, and obviously healthcare costs, with the figures quoted in your paper of approximately 70,000 cases annually and a cost of almost 2 billion pounds per year. And one strategy we know for improving outcomes is to incentivize hospitals to provide better quality care. So Matt, could you just give us a brief background regarding pay for performance initiatives in England for those listeners who are not familiar with the [00:02:00] system? 

Oh yeah. Sure thanks Andrew. So, it's not really just about England. So a lot of healthcare systems around the world have spent a lot of time and effort, a lot of clinicians involved in creating evidence to improve care for patients, to provide better faster care and more effectively. And there's a lot written about that in the journals, but sometimes translating that evidence of best practice into real change into practice in every hospital around the country or around the world around the healthcare system is difficult. And that's sometimes because the clinicians don't believe the evidence or believe it could be better, but often it's to do with processes and systems and pathways. It just make it difficult to deliver these best practice care. So healthcare systems everywhere,  in the US, in Australia, basically all over the world are finding ways to incentivize systems rather than necessarily individual clinicians to improve care for patients. And one way to do that is obviously with financial [00:03:00] incentives. 

So in the UK, in the last 10 years in particular, there's been over 20 areas of clinical activity that have been subject to topple payments related to performance.

So in a particular area, you will set some standards of what you believe is best quality care. And then, if hospitals systems processes achieve those targets, those quality standards, then you will provide them with more money on the basis of that. And that's really, it took off around 2000... late early two thousands, and then in 2010 was extended and to many areas around the country, including hip fracture in the UK. 

Yeah. So obviously that's very relevant to your study. So obviously the National Hip Fracture database is obviously linked with something called the best practice tariff. Could you give us a bit of detail about how that sort of developed.

Yeah. So the National Database is, I guess, one of the great big success stories of NHS recently. It started in 2007, based on a lot of [00:04:00] work in Scandinavia and in Scotland first, but it's become the biggest and probably the best known Hip Fracture Registry in  the world now. There's, I think, around 600,000 patients now logged on the database.

And since it started in 2007, each year there's been a reduction in mortality, but some of the markers of best practice that might improve mortality and reduce mortality for patients, were proving a little difficult to enact. So for instance, improving time to theater from injury to get to the operating theater, to have the surgery to fix or replace the hip was quite difficult to change.

So actually a colleague of David and mine, Keith Willard, who was involved in NHS England, still is involved in NHS England, trying to introduce a best practice tarrif, linked to quality standards of hip fracture. And this was activated in 2010. And then since then there's been improvements in time to surgery and so on and improvements in mortality. And the question was, were those [00:05:00] changes due to the best practice tariff or would those changes have happened anyway? And this is where David's work comes in to try and answer that question. 

That's brilliant. And that's a really nice background for the listeners. So David, if I can come to you next, obviously the aim of your study as Matt said  was to sort of quantify the effect of the best practice tariff on hip fracture outcomes in England. And you've also used control data from my area in Scotland and obviously estimating the effects of potentially introducing that at the North of the border. So the study relied on the data from the two national databases in the two countries. So can you give us a brief overview of these and what they contain and collect routinely? 

Yeah, thank you, Andrew. So as you said, the study relied on the fact that the National Hip Fracture database and the best practice tariff were implemented in England, but not North of the border in Scotland. So we wanted to select a dataset from [00:06:00] England and a separate data set from Scotland. 

We didn't feel as if we could use The National Hip Fracture Database  in England, which would be the obvious source of research data for this in this population. And that's because, as Matt said, the best practice  tariff is paid based on data that hospitals submit to the national hip fracture database. So there would have been all sorts of problems with using that resource. So instead we used administrative hospital data. So we used hospital episode statistics, which is a data set in England, and we used a very similar dataset, the Scottish morbidity record, which is used in Scotland. And these are both administrative data sets. So their principle purpose is to quantify hospital activity for administrators, but they hold a range of data about individual patients, including demographic characteristics, [00:07:00] diagnoses, and details about their hospital admission. But importantly, for us, both data sets are linked to civil registration, mortality data as well.

So whenever a patient dies anywhere in the United Kingdom, as soon as that death is registered, the civil registration mortality databases are updated, and that allowed us to see if a patient died anywhere in the United Kingdom. We weren't limited by geography. So that was very helpful for us in terms of tracking patients over time. Very important for our study. 

Excellent. That's great, David. So if we get onto sort of the meat of the methodology, so which patients did you include from the databases and what were your primary and your secondary outcome measures that you used in your analysis? So we extracted data on all patients aged 60 or older if they were treated with a hip fracture, obviously in England or [00:08:00] Scotland with an admission date between January, 2000 and December, 2016. So we had quite a long period of time. The primary outcome was death at 30 days from admission to hospital, so 30-day mortality, but we also looked at death at 60, 90 and 365 days. We looked at readmission to hospital. We looked at, the length of time it took for a patient to receive an operation after presenting at hospital and we also  looked at hospital length-of-stay. 

Excellent. And so in terms of how the analyses were formed, obviously I've read the paper myself and they're very nicely laid out in the manuscript, very clear, but just for our listeners, there's some complexity there and probably a few terms that a few of us have not heard before. Can you just give a simple, concise overview of the analysis performed for the listeners. 

Yeah, of course. So as we said  earlier, the study relied upon the fact that the national hip fracture [00:09:00] database and the best practice tarrif were implemented in England, but not in Scotland. And one of the problems with looking at patient outcomes over a long period of time is that they often trend in one direction or the other for all sorts of reasons, for example, we know that hip fracture mortality has trended gently downwards in most countries over the last couple of decades. And this could be for any number of reasons and probably multiple reasons. It might be related to better primary care or better nutrition or better social care. But it does mean that there's a methodological risk of simply comparing outcomes before and after any given date. Because if you just select any date, you'll find there'll be a statistically significant difference because there's a preexisting trend towards improved mortality. 

So in our study, we wanted to examine trends in England while accounting for changes that would have occurred, even if the national hip fracture database and the best practice tariff had not been [00:10:00] implemented. And so we used data from Scotland to model these so-called secular changes. Things that would have happened anyway. Even if the intervention hadn't been introduced. 

Excellent. That's really helpful, David. Thanks for that. It's very clear, concise overview of how it was done and the importance of the control data, should we say, from Scotland. So if we move on to, so obviously the results of the study, you looked at just over a million patients I think over the age of 60, that had a hip fracture over that 17 year period you mentioned in England and just over 116,000 in Scotland. The demographics are very much as you expect, particularly in using big data like this, over 75%  were female and the vast majority sort of occurred in the ninth decade of life. 

So David, if I could ask you, if you could just detail for our listeners the key findings in relation to your primary outcome measure, which is obviously, as you said, the 30-day mortality rate. 

Yeah. So,we found that 30 days was broadly the same in England and Scotland, before the [00:11:00] hip fracture database and the BPT were introduced. And before those interventions, it was slowly decreasing in both countries, but at the same rate. Things didn't change very much when the national hip fracture database was implemented in 2007. But from 2010, when the BPT was implemented, mortality in England began to fall at a much higher rate.

Unsurprisingly the outcomes in Scotland, they continued to improve, but along that slow pre-existing trends towards gradual improvement. And overall this study suggested that there was a statistically significant reduction in mortality of 1.6 percentage points in England, which might sound small, but it represents 7,600 fewer deaths in England between 2010 and 2016 than would have been expected without the BPT.

Yeah. I mean, there's a quite stark phase. I think that just looking at one of the tables in your paper, there is [00:12:00] quite a marked decline in England from 2010 to 2016. So what about mortality rates at the other timepoints, Dave? 

Yeah. So we wanted to reassure ourselves that the BPT wasn't just delaying, however reassuringly, we found exactly the same patterns when looking at mortality at 60, 90 and 365 days. So there seems to be mortality gains for patients, at least as far out as a year. 

Yeah. Cause obviously you look again looking at that table, the reduction in that final six, seven year period, is maintained throughout, isn't it?

Absolutely. 

Yeah. So looking at the other outcomes, so you mentioned time to operation, length of stay and readmission rates, what did you find for those? 

So in England, before the best practice tariff was implemented, there was a trend towards fewer patients being operated within 36 hours of arriving at hospital. So we were  heading in very much the wrong direction. [00:13:00] There was also a trend every year towards increasing numbers of hip fracture patients requiring readmission to hospital. 

One of the interesting things about this study is that once the best practice tariff was introduced both of those negative trends reversed. And we started to see improvements and hospital length of stay also fell in England once the BPT was introduced, which may well have implications for saving costs as well.

I mean, yeah, that's brilliant, David. I think it's very clear the trends across all the outcome measures were very positive. And so if we sort of move on to the implications of the study, which I think are quite profound, given its findings. It's clearly provided strong evidence for the pay for performance program that was instigated and proving the hip fracture outcomes in particular mortality. And, you know, it's an extensive study, really clear, there's a large number of patients included, use of big data and comprehensive national cohort databases. [00:14:00] And the very, very robust analysis performed using the control data from Scotland. 

So Matt, maybe if I came back to you, what do you feel are the key findings of the work and the implications of it, considering any potential, maybe limitations or caveats to the data? 

Yeah, thanks, Andrew. So, I mean, you've really alluded to the key findings, because David has provided the really first or the first really hard evidence to support these financial incentive schemes around hip fracture in the UK. And one of the first really conclusive studies to come out anywhere around the world really. So I think it is quite profound.

 I mean, the use of the best practice targeted in various areas in the national health service is contentious. It costs money to administer these schemes and unless there's proven benefit for the patients is that money being well spent? And this is something that is under hot debate in NHS, England, you know, as we speak. So these findings are really pretty important for those considerations and hopefully will lead to a continuation of this scheme that seems to be [00:15:00] improving outcomes for patients from reducing mortality. As ever there are limitations with this. David's alluded to some of them already. The main ones really are around residual confounding so what do I mean by that? Well, really all of the other things that happen, either patient factors or the hospital fancies that we can't measure within routinely collected datasets that might also have affected the results.

However, the key advantage we have of this system in the United Kingdom is that we have a single unified healthcare provider or our national health  service. And our patient populations, although there are some differences, are actually remarkably similar. So we have a beautiful control in a way really that actually most of the system factors, the delivery of care, processes, the referral pathway, are the same in the two countries. So the comparison becomes much easier. 

So although you can never completely rule out other factors that  will have influenced these results, in this [00:16:00] particular setting with this particular data set that David had access to, then I think we can be reasonably confident that these are findings we can rely on to inform future practice.

Yeah, no, no, I totally, I totally agree. And David, maybe just as Matt alluded to it, I will just come back to you briefly, how do these results compare to any of the previous data on this topic in literature? 

So there's been a lot of studies looking at pay for performance initiatives around the world and in all sorts of different patient populations and on the whole, those studies have not been hugely encouraging, but there does seem to be something a little bit different about the way the hip fracture care has been transformed which appears to have been partly driven by the best practice tariff in England. And there have, over the years been a number of single centre studies from English hospitals showing that quality processes [00:17:00] did improve over time when the national hip fracture database and the best practice tarrif was introduced. And these are clearly consistent with our findings, using data from across England and Scotland. 

So Matt, maybe if I could come to you finally, not really a difficult question, but more a controversial one. So what do you feel the implications of this are moving forward? Maybe including any cost implications, maybe not just for England, but for Scotland as well, and maybe even further afield with regards to this data. 

Yeah. I mean, this is a huge area Andrew really. So, I mean, it costs money to implement best practice tariffs to collect the data and to then set up the payment systems. So that money  has gotta be proven to have benefits. David's, you know, push towards that. Given that we spend around 2 billion pounds every year on looking after hip fracture patients, that's the NHS and the social  care cost, and almost certainly some other recent work that we've done [00:18:00] suggests that that underestimates the costs of informal  care. So other people taking time off work and away from other activities to look after patients with hip fracture. So almost certainly an underestimate to what it costs. Then potential implications financially here are massive. 

So, it's not just about getting people back to their own homes quicker, which saves money from hospitalization, but if we can improve their quality of life and allow them to live more independently. The social care implications in terms of the cost of that, if nothing else, are just enormous and that that's replicated around the world. 

So we've had a big demographic change in the UK over the last 30 years or so, an older, more active population, which is great, but also a lot of frailer patients within our communities who need a lot more care, including those with with hip fracture, but around the world that demographic change is only just starting and the proportions are just absolutely staggering. So [00:19:00] in Southeast Asia, the portion of elderly patients who will be having hip fractures is just going to go up and up every year, and healthcare systems around the world are not necessarily as well setup as we are in the UK to deal with that demographic sort of change. So we're talking trillions of pounds in costs. So even small incremental benefits in outcome for patients to reduce costs will have massive implications around the world. 

Yeah, no, that's a really nice way to put it, Matt. And I think, you know, around the world, these small changes, but actually end up with massive, massive outcome changes for the patient and potentially cost savings as well. 

Well, I think David and Matt thats all we have time for, but thanks so much for joining us for the podcast and congratulations to you both in a really excellent study. I really do think it's a real game changer in the area, and I think it's provided some essential evidence to  support the changes that have happened in England. And I am sure it's given many of our listeners, much food for thoughts. [00:20:00] And to our listeners we do hope you've enjoyed joining us and we encourage you to share your thoughts and comments through Twitter, Facebook, and a like, and feel free to post or tweet about anything we've discussed today. And thanks again for joining us.