Diet Decoded: Navigating the Wahls Protocol and Unraveling Autoimmune Mysteries

Explore the role of diet in autoimmune diseases as we welcome back Heather's Wiki for an intriguing exploration of the Wahls protocol in managing Multiple Sclerosis (MS). Together, we analyze a a small MS diet study, offering a reality check on results versus discussion, separating scientific substance from noise. This episode sharpens your critical thinking skills, providing knowledge to navigate the complex realm of nutritional science.

Join us in addressing practical challenges in dietary intervention studies, such as small sample sizes and recruitment issues. In a lively discussion with Heather, we uncover biases in weight-list controls and adherence problems that can obscure research outcomes. This episode isn't just about effective strategies; it's a journey of discovery, highlighting the rigorous process ensuring scientific validity.

In this episode we will empower your critical thinking and understanding of research findings, while our candid analysis will guide you through interpreting these findings and understanding their clinical implications. 

Lee JE, Titcomb TJ, Bisht B, Rubenstein LM, Louison R, Wahls TL. A Modified MCT-Based Ketogenic Diet Increases Plasma β-Hydroxybutyrate but Has Less Effect on Fatigue and Quality of Life in People with Multiple Sclerosis Compared to a Modified Paleolithic Diet: A Waitlist-Controlled, Randomized Pilot Study. J Am Coll Nutr. 2021 Jan;40(1):13-25. doi: 10.1080/07315724.2020.1734988. Epub 2020 Mar 26. PMID: 32213121.

Learn more and become a member at www.DrJournalClub.com

Check out our complete offerings of NANCEAC-approved Continuing Education Courses.

Transcript

Introducer: 0:02

Welcome to the Dr Journal Club podcast, the show that goes on to the hood of evidence-based integrative medicine. We review recent research articles, interview evidence-based medicine thought leaders and discuss the challenges and opportunities of integrating evidence-based and integrative medicine. Continue your learning after the show at www.drjournalclub.com.

Josh: 0:31

Please bear in mind that this is for educational and entertainment purposes. Only Talk to your doctor before making any medical decisions, changes etc. Everything we're talking about that's to teach you guys stuff and have fun. We are not your doctors. Also, we would love to answer your specific questions. On www.drjournalclub.com you can post questions and comments for specific videos, but go ahead and email us directly at josh at drjournalclub.com. That's josh at drjournalclub.com. Send us your listener questions and we will discuss it on our pod. 

Hello and welcome to the Dr Journal Club podcast with your hosts, Josh and Adam, and today we have a returning guest, a fan favorite, the amazing Heather's Wiki. We were trying to go back and I think you are on either the fourth or fifth one of our starting episodes and maybe our first or second guest actually, so it's fun to have you back almost exactly a year later.

Heather: 1:36

It's fun to be back, and calling me a fan favorite was buttering me up, huh.

Josh: 1:40

Exactly. Well, that's part of the plan. I want you to come back for a third time and then some. So last time we had a lot of fun talking about like the NIH and integrative medicine and sort of history there, and today we have a more specific down in the trenches work around a specific paper. But do you want to set us up as to why you chose this paper, for us to go over it, how you use it in teaching or in general, and then we can go in from there?

Heather: 2:10

Yeah, absolutely so. There's been a lot of interest on whether or not diet can reverse autoimmune disease and when people are starting to look at diet and autoimmune disease. One of the more popular books that is out there is by a woman named Terry Walls, and Terry was an MD who developed MS Multiple Sclerosis herself and dove into the literature the diet literature and treated herself with diet and was successful. But she had some very specific things that she did. Once she treated herself and had success, she felt like she needed to share that success with the world. So she wrote a book and the book outlined again in great detail all of these very interesting ways to approach diet. So for MS she was really focused on things like organ meats and very high fat diet and then made those recommendations to all of these people through a book without having ever done any research on whether or not that diet was appropriate for anyone other than herself. This is the follow-up research study where they've actually now started to look at the specific diet in different individuals who have multiple sclerosis, and I thought it was really an interesting paper when it came out, both because I teach autoimmunity and I teach nutritional immunology, so it kind of spans multiple things, but it also, I feel like, is a great example of why we do research, because Terry did not get the result that she thought she was going to get with this study, and I think that's an important aspect. And yet if you read the results of the study and then read the discussion, the discussion and the results don't match. So I thought that was another really interesting piece about this particular paper. So I use it for teaching for those reasons, because it crosses multiple different disciplines. It gives an actual clinical trial of something that has already had a book written about it, and millions of people are following a diet that now we have the evidence that the diet may not be the best diet for them to be following. So those are the reasons that I really like this paper.

Josh: 4:37

Awesome.

Adam: 4:38

I think it's interesting that you also highlighted the fact that the discussion section and the results section oftentimes or at least in this paper don't meet eye to eye, and it's also a reason why I usually just skip the discussion section altogether if I'm reading a paper, because it's kind of a nice spot for authors to put in magic pixie dust and sort of.

Josh: 4:59

Spin zone.

Adam: 5:00

It's a spin zone. Spin things around to make negative results sound good or positive results sound bad or really whatever you want honestly.

Heather: 5:11

Well, no, it is so true, and what I find with a lot of students is that they'll read the introduction and the discussion and they skip the data. And so this is a great paper where I can tell whether they read the introduction and discussion and skip the data, or whether they actually looked at the data in this paper. So it's a great tool for educators as well.

Josh: 5:33

Love it. Well, just for the sake of the listener who may not have read it, let's maybe go through the basics sort of design of the study first, and then we can jump into some of those issues. I think, although I have to say, one cool thing is when I think it is really neat when you've got so usually clinical practice precedes research, right? Or in this case, maybe it's the opposite. She was reading the research literature, designed her own form of it, right, so it was informed by research, and now research is following the clinical sort of cutting edge, if you will, and I think that's kind of neat. And so you know, in our space of integrative medicine, like how many of these guru providers even do that, right? Like I think that's kind of cool to actually, you know, put a proper research study together. Was there like a backstory at all? Or she was just like interested in doing research on it and Well, it was her MS.

Heather: 6:28

So yeah, the backstory was that it was her diagnosis.

Josh: 6:31

No, but for doing like a proper study, as opposed to just the book?

Heather: 6:37

She was asked to do the study. Actually, the MS Society is the one that funded this, but NIH has now come to her and said hey, there's enough people doing this that you really need to do the research to demonstrate that this is working.

Josh: 6:51

So the NIH approached her. That's really cool.

Heather: 6:55

I know that has happened a second time. I don't know if you guys have reviewed the Dale Bredesen Alzheimer's papers. I don't know if you've done it on Dr Journal Club those papers same thing. Dale Bredesen wrote these great papers on reversing Alzheimer's and reversing cognitive decline and has this whole lifestyle intervention, nutrition, take on how to do that. But had never done a clinical trial. It was literally just a case series and then he wrote a book and then came back and people were like no, you got to do the clinical trials. And so now he's actually funded to do the clinical trials on that for cognitive decline as well.

Josh: 7:38

Okay, oh cool. That's really neat. He's being responsive to what's out there and what patients are doing anyway, so that's cool. All right, so let's describe the study a bit to set it up, and then we can talk more big picture stuff. All right, so there are well who wants to go through the? Should we divvy it up? Who wants to go through the study design? Anybody want to raise their hand? Nobody's raising their hand. All right, I'll do it. Fine, fine, fine, okay, so that's it so. I'll stop. They talk about some background research supporting, I guess, diet and MS in general. So sort of just like broad stuff basically, but setting up an argument for why diet might be impactful for MS and why we should be looking at it.

Heather: 8:29

And there had been a couple of pilot studies showing that paleolytic diet was something that seemed to be benefiting some of the symptoms associated with MS, like quality of life and fatigue and cognition, but they were small studies so nobody was really saying, well, we should put everybody with MS on a paleolytic diet at the point that she started this research.

Josh: 8:56

Gotcha, okay, cool, and it seems like her diet intervention for this. So there were three arms. So there was a control diet which was like the weightless control so people's Regular diet would they would start out with that, so you'd be randomized to possibly that or to seem like two forms of her diet, like it just like a straight keto and then like a paleo keto. Is that? Is that a fair explanation there?

Heather: 9:22

I think that's a fair explanation. It's it's like a paleo or an, a paleo plus fat.

Adam: 9:27

And on that, on a ketogenic diet, they emphasized MCT oil supplementation, so that's also another thing to take into consideration.

Josh: 9:37

Okay, okay, got you right. So there's sort of MCT oil of heavy keto and then paleo, and so I don't know, are these like two versions of her diet that she talks about a lot, or is it? One of them was meant to be like an active control, or were they're both like part of her protocol?

Heather: 9:56

So if you read her book, her book is based on the MCT based ketogenic diet.

Adam: 10:01

Okay so it is based on the high fat diet, but because I think there were some other evidence suggesting that paleo was gonna be a good approach, she decided she needed that as a control right and also in the in the methods, within the study diet subsection it does have sort of like her Walls paleo diet and then the walls paleo plus and that that paleo plus is the MCT based ketogenic diet that's studied here, and then the Paleolithic diet, or her, her version of the paleolithic diet. Is was one of the other intervention groups, and then the, the, the weightless control, was just the usual diet.

Josh: 10:41

Gotcha. So her it seems like the, the true active. There was the MCT keto and then sort of a modified paleo, based perhaps on previous research, and then so I guess we could think of that as like an active control, if that's not the standard Form of her diet, I suppose, and then just the weightless control, of course, would be a Proper, proper control. I'm also noting that she did have a Registration of the study, which makes me happy, and so anyone, anyone want to see while I'm looking up the registration, anyone want to speak to why, in particular in the situation like this, it would be good to have this sort of thing registered.

Adam: 11:27

Well, I mean, you want to understand that or not understand, but you want to know. You know, make sure that they're not changing the protocol you know extensively, to kind of you know, put a Around, peg into a square hole type of thing where everyone has access to what they're going to be looking at, basically what the protocol is, what their outcomes of interest are and how they're going to analyze the data before they even run everything. Because then if a lot of things are changing, you know, do they, do they alter the study to sort of Make the results look nicer than what they actually are?

Josh: 12:04

Yeah, so yeah, and exactly and especially with okay, I guess we didn't mention this yet, but the senior author is walls, the person who designed this and has the business in the books based off of it. And so I think in a particular case here where there's obviously a you know obvious and they're very explicit about it, this an obvious conflict of interest. There you just want to be really careful about like p-hacking and Selective outcome reporting and stuff like that. So, especially with how they ended up and I know I'm getting ahead of myself here but how they analyzed some of these results. I'm just kind of looking in the protocol now so for the listener, the, when it says clinical trials gov and it gives a registration number and CT and then a number. You could just plug that into Google and or any register, any registrar, put in the number and then get the protocol that they Said that they were interested in and it should have a time stamp to see if it was change. And in particular, I'm looking for how they plan to analyze pre-post versus you cross groups and I don't actually See that in here, which is interesting. There's no like Analysis plan. That's kind of curious. Why would you say that Analysis plan is kind of curious. Why would there be no analysis plan? Why is that? Why must be missing something? They have to have an analysis plan.

Adam: 13:31

I think, in small text that wrote just trust us.

Heather: 13:36

Well, you don't have to have an analysis plan if it's in a feasibility study. And I think, because this is such a small Study, it's considered more feasible. And if you look at their outcomes, they say at the very end that this what well, it's not in the title, it is. At the end. They say it is feasible to feed the ketogenic diet to people with MS. So I do think that they were Pitching it as a feasibility study initially.

Josh: 14:02

So that's how they're getting away with it, Okay which there's things to talk about there.

Adam: 14:09

One is then recognizing that this is a feasibility study and it's not really about the efficacy of it. It's about can we actually implement this within a research setting or within General population? Right, and then I would also argue and we can talk about this as we, as we kind of go through the paper that it's actually not really feasible.

Josh: 14:33

Okay, yeah, so let's, let's talk about that for sure. So it's interesting, right, because if, if you look at the, the outcomes here, that the, the primary outcome that they talked about is perceived fatigue, and they talk about the, the measures, they're, they're going to use the fatigue scale, but they don't say that they're gonna do baseline to endpoint or just the endpoints across groups. So why is this relevant? Why am I harping on this? So when you look at the, when you look at the results of the study, so okay, so we're getting ahead of ourselves. So so we have these three arms of the study. You're randomized to these three different groups weightless control, the active ketogenic walls protocol and then this paleoversion of the walls protocol. There were 15 people randomized as a small study, so randomized about About. So we have four in one group, six and another and four in another group, so randomized across those three groups. So you're not having a lot of people per group. So I guess the first thing I would say is I wouldn't be surprised if they didn't see much difference between Groups, because there just weren't that many people. This is be a lot of variability there as well.

Heather: 15:39

I'd agree with that. I would add that they had trouble with recruitment, mm-hmm. So if you look at the differences in the groups at baseline, they are significant. They have different types of MS relapsing, remitting MS, progressive relapsing, remitting MS, progressive MS and there's different numbers of those patients in each group and they say in the text that that is due to low initial enrollment. So they change the criteria so that they could include all types of MS, and anybody who works with people with MS knows that you can't really compare those diagnoses like if you are in a large clinical trial. That would be a Killer like you would not want to be doing that. So so as we're looking at these three groups, I think it's important to understand that the groups were not similar at baseline.

Josh: 16:36

Right and also, Yeah, go ahead. 

Adam: 16:37

For those who, just for those who are unaware, would you be able to Kind of expand upon what a weightless control is?

Heather: 16:46

Yeah, so, um, often we do weightless controls and studies so that nobody is left without getting a therapy. So instead of having a group that gets a placebo, you can't really do a placebo diet. So instead, what they do is they say okay, we're gonna have you do a standard diet or your your current diet, and then, after one group finishes the clinical diet in this case, either paleo or keto will actually give you the paleo or the keto diet. So during the study you're the control group, but after the study is over, then you can You're weight-listed to actually be in one of these other groups.

Adam: 17:32

And so what's the bias behind that? What, why would we can be concerned by that versus just that, more of like a traditional control group.

Heather: 17:38

Well, one of the things that often happens in a weightless control is, if they know what they're headed towards, they start changing their diet on their own Because they know they're headed towards a paleolithic or a ketogenic diet. So are they a true control? It you have to and measure their adherence to the control diet, which they didn't do in this case, like there, at least they didn't report Adherence to the control diet and the control diet, by the way, if you, if you again, you have to read the paper to figure out what the control diet is. But they require consuming at least two daily servings of gluten, dairy or eggs in the control diet, and so that's significant. If somebody started to change that right if somebody it could mean that the control starts to look better than it is.

Josh: 18:34

The control starts to look better. Wait, what do you mean by that? 

Heather: 18:40

So nutrition studies are complex, right? Anybody who's done a nutrition study knows that nobody wants to be randomized to what they eat. Everybody has specific things about what they want to eat and what they don't want to eat and, and especially Nutrition studies that are 12 weeks long like you can randomize people to eat a certain way for about a week, but once it's going two weeks and then a month and then 12 weeks. Now you've crossed over holidays and birthdays and all these things where people want to be able to eat what they would eat normally, like celebrating with friends or whatever. And so If you're, if you've got a control diet where you're specifically wanting people to eat these specific foods, so you've got a good comparator and now they know that they're gonna get to do the paleo diet or the keto diet. There's enough Literature on the internet that you can research that diet yourself and you can start adding little things. Oh, interesting and so now you're depending on people's honesty that they have remained on the control diet, unless you're truly measuring adherence to the control diet, meaning you're providing the food, you're weighing back what they ate, and now you're making sure you know that they Didn't run out and buy coconut oil in the middle of the night.

Josh: 19:57

So we would call that that's. That's, that's contamination, right, like we would call that contamination Okay, meaning like you gave them one thing they were supposed to do but the intervention got quote-unquote Contaminated with the actual intervention you're studying. But if that's the case, then, just to play that out, then the difference between the control group and the intervention group should be smaller. That's right, because they're approximating. So you should, so it would. So this is a good example, I think, of where the bias actually would make the effect smaller, not larger. Normally we think about a risk of bias is making the effect more Rosie. In a situation like you're describing, it would make the effect less Rosie.

Heather: 20:36

I would, I would suppose so it's and we see that for lots of C I H intervention. Right, we see a lot of integrative health interventions that when people get randomized to not getting to do yoga, then they're like, well, I'll just do it on my own right. So so it happens. Weightless controls happen a lot with C I H interventions because they're accessible. Weightless controls don't have the same issues with a lot of pharmaceuticals because those pharmaceuticals aren't otherwise accessible.

Josh: 21:10

Look, the thing is we don't do this for money. This is pro-bodo and, quite honestly, the mother ships kind of eeks it out every month or so. Right, so we do this because we care about this, we think it's important, we think that integrating evidence-based medicine and integrative medicine is essential and there just aren't other resources out there the moment we find something that does it better, we'll probably drop it. We're busy folks, but right now this is what's out there. Unfortunately, that's it, and so we're going to keep on fighting that good fight. And if you believe in that, if you believe in intellectual honesty and the profession and integrative medicine and being an integrative provider and bringing that into the integrative space, please help us, and you can help us by becoming a member on Dr Journal Club. If you're in need of continuing education credits, take our Nanceac-approved courses. We have ethics courses, pharmacy courses, general courses, interactions with us on social media, listen to the podcast, rate our podcast, tell your friends. There are all ways that you can sort of help support the cause. Would you agree that if there was evidence of contamination in this study and there was an effect, that we would be less concerned? Because whatever effect we see is maybe even larger if there wasn't that contamination. Is that a fair way to thinking about it?

Heather: 22:34

That is a fair way of thinking about it. However, one of the issues with a very small study like this is that if really what you're looking at is feasibility, you've only had to convince four people, or five people, to change their diet here. When it now comes into implementing it on a larger scale, can you convince 20,000 people to change their diet? That's where true feasibility comes in right.

Josh: 23:06

Yeah, very true. We have these small numbers of people in each group. We talked about the issues of weightless control, a weightless control diet. That's awesome Good point, adam. Bring that up. They go through these diets for 12 weeks and then they're looking at all sorts of different outcomes. Presumably they only care about feasibility Can we do this moving forward, which is how they get away with a small study.

Adam: 23:32

Can we also talk about the feasibility issue that I have? Sure, yeah, yeah, okay, go ahead, jump in. There's a couple that I have. One is part of the exclusion criteria. Was they actually excluded participants if they were unable to afford a potential 30% increase in grocery costs? I think that's going to be a huge issue for a large number of patients, especially within a primary care setting, especially if you're working with underserved communities or at risk safety net type of clinics where that's going to be a huge barrier. Then also, they excluded people who are unable to follow the study diet. It takes out the external validity of that where how commonly do we ask people to what we perceive as perhaps simpler diet changes than something such as a Paleolithic or a specific MCT-based diet of just saying, hey, can you increase your fruits and veggies? How much we struggle with that To take out people who would otherwise or not include people in the study who would otherwise not be able to do it. There is a little bit of bias there. Then they also provided these patients with a Vitamix Blunder.

Josh: 24:50

When I saw that, I was like I'd sign up. I want a Vitamix.

Adam: 24:53

That's only $600. How many clinics and hospitals have you been at where we hand out Vitamix to people?

Josh: 25:07

No, that's true. Okay, so you're arguing. Okay, let me push back, because I agree with you, but I want to push back as a devil's advocate. I think this is a big debate between internal and external validity, often, Right, so you're arguing like well, they didn't include people that couldn't afford an increase in groceries. It is important. But also, if this diet actually costs this much, are people that can't afford the increased cost going to do the diet to begin with? So, if you are going to measure the effect of this diet, maybe that's not so crazy. However, I do think it would be fair to say this type of diet is not going to work for folks that don't have the money to pay for it because it's going to be more expensive. So I don't know when you're doing all these exclusions. I think often it can speak to both internal and external validity, but I don't know that sort of thing. I'm like, well, let's be honest. Well, the Vitamix might be a bit of a strong incentive, but if it's like, if you're not going to do the diet, then okay. So here's the other thing. Your other point was if they're not going to be compliant with the diet, then they're excluded. Now that, I think is more worrisome because I think that was in the run-in period. They had a run-in period and then okay, so you have to be super suspicious of run-in. So, Heather or Adam, do you want to do the quick background on what run-ins are in trials and why we care?

Adam: 26:30

Yeah, basically, if you're doing trials, they're going to be expensive and so you want to make sure you're having people who are actually doing the intervention that you're interested in. This is what you can have, as accurate as possible of data with respect to what the question is. So you do see it oftentimes in medication trials, especially if there is private funding, because the companies are paying for the trial and they don't want to spend billions of dollars or millions of dollars on a trial that has no findings or what we consider to be a negative result, because people just aren't taking the medication. So it's not necessary that the medication is not working or the supplement is not working, it's that people aren't taking it or not taking it as prescribed, and so then we're not getting the treatment effects. And then, similarly, from a dietary standpoint, if people aren't going to adhere to a diet in its diet intervention, then of course you're going to have sort of these null findings.

Heather: 27:25

So what they do is they'll take two weeks and they'll say all right, we're going to give you guys this diet for two weeks and we're going to see how compliant you are to the diet and if you're not compliant, we're tossing you out. You don't get to be one of our study participants, and if you are compliant then we'll let you be in the rest of the study. So that's what a run-in is going to do for you.

Adam: 27:44

And similarly with medications too, like a two-week run-in of like. Can you be adherent to the medication? If not, then you're not going to be part of the longer study period.

Heather: 27:56

Now, one of the other things we do with run-ins that I think is important is we also figure out who our placebo responders are. So sometimes we'll give a placebo run-in where for two weeks we'll give people a placebo and the ones who are responding when it's only placebo they get tossed out. So now we know the true effect of the medication for the remainder of the people.

Josh: 28:18

Okay, so that I'm way more concerned with. So walk me through why that's kosher. So like if you're excluding people who can't follow the diet. I don't love that. But at least it's like you're measuring people who are actually are going to stick to a diet and that's reasonable. But it's external validity issue, right, Because not everyone that starts a diet is going to follow it. And if you want to know what the average effect in the population is, you want to know what they're going to do to the pool of results. But if you want to know what actually following the diet does, okay, that's fine. But if you're throwing out placebo responders, then you're essentially making your oh, but you're doing that before randomization. Okay, You're doing that before randomization.

Heather: 29:01

Yeah, no, it's actually. It works really well, because then you know that.

Josh: 29:06

Interesting.

Heather: 29:07

The placebo responders should be gone, so now every response that you get should be due to the medication.

Josh: 29:15

Okay, okay, but you're still okay. So you're not biasing across groups, but you are overestimating the difference that you would expect, right, because there's going to be people out there that are just going to respond on their own, but they're not even part of the pool in this.

Introducer: 29:30

And I think it kind of.

Adam: 29:31

so you're only yeah, go ahead. It kind of depends on what your question is. If the question is, does it work? And that would be adequate, because you just want to see is there a response versus what is the magnitude of effects, Then that becomes an issue.

Josh: 29:48

Right, okay, so like efficacy versus effectiveness, almost like big picture, is this drug really work? Efficacy across the population, including placebo responders. What is the effect type of thing.

Adam: 30:01

Okay, interesting, and I think another thing we could even kind of debate is does a run in period matter in dietary trials, when we know that over the long term, adherence to specific dietary patterns tends to be an issue? And I think, josh, this comes up all the time and I think, based on your meta analysis last year, I want to say with the ketogenic diet for diabetes, I think it was by what by 12 months, most people seem to have adherence issues.

Josh: 30:33

Yeah, and that's when you see the loss of the effect of the diets, and that's pretty standard across all diets is my understanding, and I think you're right. I think that's just because people just burn out on diets, which is like a whole, a whole. Another issue, okay, interesting, so we're kind of having debate on okay. So, heather, so you think it could be kosher to throughout these placebo responders, as long as you're aware and you really just care about Okay, but then also, okay, hold on, because then also you're limiting your variability, right, sure, by doing that right, and so you're more likely to have a nice little p-value because you've got your control groups nice and neat, without a lot of variation, and I feel like that just doesn't feel kosher either. I don't know.

Heather: 31:21

I feel like we did a placebo run-in for a Bacopa trial that we did Okay. So here's what you do. You say, all right, for two weeks everybody's taking placebo. They don't know that they're taking placebo. And then you say, okay, if you are on the medication, well, bananas are going to start tasting bad to you. So people would come to us during the placebo run-in and they say, oh, I must be on the medication because all of a sudden I can't eat bananas.

Josh: 31:53

Oh, really Wow.

Heather: 31:54

And the reality is everybody's on placebo. But we know at that point that those people are placebo responders because they are the ones who are creating a symptom. It's not real that they're going to not be able to eat bananas once they're taking this medication, right? So those people are able to heal themselves with their minds because they are placebo responders. That's what it is right. We don't know the effects. It's nonspecific. So if you want to get rid of those, to know that the Bacopa responders are real and that they're not just placebo responders, so you do it, especially in small studies. You do it so that you can see the magnitude of effect. Now again, this is a feasibility study. Do they need to do a placebo run-in or a diet run-in for a feasibility study? No, but they chose to, probably because they wanted to see the magnitude of effect of people who would be compliant to the diets.

Josh: 32:55

Well, and if you had four people and two of them just aren't putting their numbers down in their thing like that just ruins everything too. Yes, okay, interesting, okay, so nice little. I like this tangent about the run-ins. I think this is great. Adam's probably like stop with your methods tangents, but that's really fun to me.

Adam: 33:12

I just think it's funny that of our remaining two listeners and thanks mom and dad this is what the majority of our conversations are like, and just how nerdy we get with things.

Josh: 33:25

Okay. So I'm hearing Adam in my head saying Josh, keep it clinical. Okay. So going back to clinical, from the efficacy perspective, one thing that they did find in this study was that the ketogenic diet was successful in creating nutritional ketosis and that basically they could show that biochemically, and so they basically just showed that a ketogenic diet is ketogenic, which wasn't too surprising, but I think that's one thing that they were able to kind of like from an efficacy perspective. Right, that's part of it, heather. Right, it's not just like will they follow it, but it's also will the diet do what it intends to do?

Heather: 33:59

Right. Did they get physiological change? And they did.

Josh: 34:02

Okay, so they were able to show that these diets are creating physiologic change. And then you know sort of what they claim is not important, but we're all interested in which is the actual outcomes. This is the thing that. So they look at fatigue. They looked at, I think, generalized markers of MS in general and quality of life and muscle strength. They seem like reasonable outcomes to look at for MS. It seemed Very common outcomes for MS. Yep, okay, so these are common. Okay, so you had no problem with the outcomes. That's good. No, yeah, but, but go ahead.

Heather: 34:41

So they did see a difference between the paleo group and the control group. However, again, if you go to clinical and you look at who's on medication, the paleo group is on medication and only one person in the keto group is on medication and nobody in the control group is on medication, and that strikes me as significant. So is it that the diet's doing the difference or is it the diet plus medication is doing the effects that we're seeing here?

Josh: 35:14

Yeah, yeah, and you wouldn't be able to tell, and that, in theory, should have been the magic of randomization, but we just don't have large enough numbers to balance those out. Okay, that's, you would think that when you do some of these randomizations, like block randomizations and things you can control for that sort of thing, to make sure that there's the same number of, like, males and females in each group, and right, or is it just?

Heather: 35:38

Right and the same numbers of progressing, relapsing, remitting MS.

Josh: 35:41

Right.

Heather: 35:42

Versus relapsing, remitting MS versus progressive MS.

Josh: 35:45

Right.

Heather: 35:46

And there's not the same number of those in each group either, and so when you have a super small study like this, those are some of the defects in design.

Josh: 35:56

Right, you just can't balance those prognostic factors.

Heather: 35:59

Okay, and in fact, you might have actually done better to do it as a case control study to get better balance in this situation than randomization.

Josh: 36:11

That's interesting, and so we can't right, so we can't say it's really this diet, because it could just be the meds, could be the severity of the disease or the type of disease. So a lot of this is coming down to their challenges. It sounds like in recruitment, like they had intended to recruit a more homogenous body of people but that it was apparently very challenging, and then they had to kind of open it up, and now we have these disparities that make the interpretation difficult. Okay, so that makes a lot of sense. The other thing that I wanted to highlight for the results is and maybe this is just me being annoying about this, but you know, when you have this baseline, you know baseline and endpoint Results across groups. So there's like different ways of looking at that. Right, you can say, okay, well, you know, they started at level 10 severity in the Paleolithic group and they ended at level 7 and that difference from baseline to endpoint was statistically significant. Right, you could look at it that way. Or you could say, okay, at the end of the study across all three groups, let's look at the difference in severity and see if those are differences are meaningful, right, and they seem to do like to mostly do baseline to endpoint For across groups, but I don't I didn't see anything in the protocol that said that's how they plan to analyze that and to me that's just a little suspicious for peahacking but maybe, like in something like this, that's the standard way of looking at it. No, especially with small populations.

Heather: 37:36

I mean, anytime you do randomization, your comparison has to be between Experimental group and a control group. You can't be looking baseline to yeah endpoint right so there you go. The statistics are incorrect, that is. I'm glad you caught that.

Josh: 37:54

Okay, so, so. So then this really should have been like looking, comparing the endpoints across groups, which they don't talk about at all. And it isn't clear, at least to me, in the protocol that they published, that they that's what they plan to do, right, and so there's the fear that they kind of did baseline to endpoint because it looks better, okay, awesome. So that is important. And that's another thing and I'm late, I'm, I could be accused of this. You know I look for registration and I usually would just say, oh, check, they registered it, so that's, that's good. But you really should be looking at the registration and Seeing what they said and making sure that they reported what they said they reported, or they they even Said how they were going to report it. And again, I don't see any indication of what their Statistical analysis plan there was, which makes it a little bit. A little bit dubious to me. Cool, okay, what other things did you guys want to raise about the study? Anything else that? Well, and what's your take home, heather, from from the overall paper, would you recommend this, would you not? What's your overall?

Heather: 38:52

good shawl so a couple of things, that that I should also think about this paper and and I think that this happens a lot when you have Someone who has a personal investment Clearly the diet was what was most important to them and that is what they have focused on in the paper. They spend a lot of time defining the diets what people could eat and couldn't eat, and they they didn't bring in the expertise to do the statistics correctly, they didn't bring in the expertise to do the feasibility correctly. All of those other pieces really could have benefited from people who do nutrition trials and Instead the focus was the clinical piece, which is important. No question that the diet is important, but but there's some, there's some novice mistakes that To me are, you know, avoidable, and so that that is a concern, would I recommend the diet? I think this is another small study that shows the Paleolithic diet is actually effective for people with MS. Now, that being said, the Paleolithic group was the group that had the most medication as well, but they also had the most severity starting. So if you look at the baseline characteristics, they still had the most severity of fatigue. They had the most severity in other Symptoms, quality of life, etc. So while they were on the medication, they also had greater severity than the control group. The control group actually had the least severity. So their their baseline tells you right away that the groups aren't matched Mm-hmm. But the fact that they still got something with paleo is great. Now, if you read the discussion, the discussion is all about the keto. I know that was so surprising. And the keto. They didn't see any difference and so it's like well, why are they discussing all this keto stuff, other than the first diet that that dr Wall's Trademarked was the keto diet for MS, and then she later trademarked her paleo for MS.

Josh: 41:04

So after the study after this study or I don't know the date that she did it interesting. This is pretty recent, I guess, mm-hmm.

Heather: 41:12

Yeah, so this is 2021.

Adam: 41:13

So so it's in 2021, but I think they were recruiting much earlier than that.

Josh: 41:20

I think like yeah, like 2019, 2014 even. Oh, wow, okay, so they really had a hard time recruiting.

Adam: 41:28

Yeah, I. Like wow, okay, I Think they even had the dates. If I want to say I can try to keep talking, I'll look for it.

Josh: 41:37

Well, well, while you're, while you're looking for it, heather, the other thing I'd say is, like, methodologically, there's a potential issue there with the severity issue, in that Interventions tend to be more impressive for people that have a higher severity. Right, because there's like a floor effect that happens if you're already doing pretty good, yeah, and so it's really curious, right, so that you could take these results and say the paleolithic diet works here, yeah. Or you could say Medication works here, right, because there are more people on medication. Or you could say, if paleo works, it only really works in any meaningful way with people that have severe, more severe disease, right, because you're just seeing the larger effect there, too, yeah, and it's just like, and then was? Did you say there was also a difference in relapsing, remitting, or was this the severity that you were Highlighting?

Heather: 42:21

I was a severity that I was paying attention to. Yeah Well, so the other pieces. I saw Dr Wells do a talk um maybe a dozen years ago, and she really emphasized organ meats and fat and like if you weren't willing to do organ meats and fat, you might as well not bother. Right, and this did not support her worldview, like this study does not support what she said. So it was also just interesting to me that Her particular bias was not supported by the study, although the discussion supports her bias. So so I thought that was interesting.

Adam: 43:04

Yeah, yeah, the the recruitment says due to low initial enrollment on 11 13, 13, inclusion criteria were changed and then enrollment started on 2, 16, 13 and the final study visit occurred on 11 26 14 oh so so they published it seven years later seven years later and wonder what took him so late, Right.

Josh: 43:29

So they recruited enough people within a couple years, but it took seven years to publish the paper. That doesn't make sense.

Heather: 43:34

Well, it's a can which is a.

Adam: 43:38

This is really really interesting.

Josh: 43:41

That's a bit of a flag.

Heather: 43:43

Uh-huh.

Josh: 43:43

I mean so they, they finished recruitment of everybody and the study period and had presumably all the data, and and then seven years later it's published.

Heather: 43:54

Well, so I can tell you what can happen, because we've had some experiences like this before, where the researcher leaves the institution and Goes on to do something else, and then you're trying to get the study team together and and then you're waiting for the one person to sign their Conflict of interest form, and then you know, blah, blah, blah, like it can take a while.

Josh: 44:17

Okay.

Heather: 44:17

That's a long while, um, but it can take a while.

Josh: 44:22

Oh, I think so.

Heather: 44:23

Okay, so you don't view that necessarily as a flag per se well, but the other piece you have here is you have trademarked diets by somebody who's written a book and is making book money from these diets, and so if you're gonna publish a Study that contradicts the book that's out there, you might want to delay that a little bit, right?

Josh: 44:47

Right, that's a good point too, and I'm always like impressed when industry does like put up money for studies, if they're selling stuff, well, right, because it's like you know they've got a good thing going on and they're taking a little bit of a gamble by doing a Study right, and so, yeah, you can always try to spin it, but like, if it's done in a good manner, like this just speaks to a lot of Confidence. I guess they have in the, in the study results, but cool, okay, I think that's all the stuff that I highlighted that I wanted to talk about. Oh yeah, just one last thing. With such small numbers, like, does it make any sense to have three groups like that? No, that just seemed nuts. Like I feel like you should just have two at the most, obviously, and then maximize yeah. So that seemed like very odd to have three.

Heather: 45:36

Again, anybody who had worked with a clinical trialist would have a clinical trialist would have said do not do three groups when you've got this small With 15 people. Yeah.

Josh: 45:45

Yeah, okay, interesting. So it looks like the take home is clinically it's spot on. I love all the tables describing and the outcome seem. Choice of outcome seem appropriate and the diets well described and there was an RD that helped walk people through the diet and that all seemed really good. So from the clinical perspective it seemed really excellent. A lot of that external validity stuff that you don't always see in trials. But then the actual execution design, the internal validity piece I think really was, really was missing.

Heather: 46:16

Yeah, yeah. And so then, what do you say to your patients? Do you say, well, do Paleolithic. Do you say, oh, try ketogenic. Or do you say, well, we still don't know anything about diet.

Josh: 46:29

I'd probably be a bit of a latter. I don't know, and I guess it depends on the tradeoff right, like if it's going to be this versus a med that has some you know proof of it I don't know what the field. Well, but if there is a good alternative like this does not move the needle for me at all. If there's nothing and this is an additive thing and it's not going to cost an arm and a leg, I don't know, like I'd be like sure, why wouldn't you try it?

Heather: 46:53

I guess, the way I look at it is, the best result they got was for fatigue, and so if I've got an MS patient who is on medication, who still has significant fatigue, yeah, but Heather it's open label.

Josh: 47:04

That's the thing. Like fatigue is such a subjectively reported outcome and it's an open label study and everyone knows this is the active intervention, Like that's just like there, I don't know, that just seems like very unimpressive to me. I don't know, Maybe I'm being too harsh.

Heather: 47:18

Well, yes and no. I mean, fatigue is complicated, right, but we know that, okay, so I'm going to come at it from the basic science lens for this moment. We know that one of the big things that affects fatigue is microbiome, and we know, if you change your diet, you're going to have an effect on your microbiome. So, if you are moving from a diet that is high in gluten and eggs to a diet that is not high in gluten, and you know and I'm not saying that there's anything evil in eggs, except most people aren't eating organic eggs and then there's pesticides and that you know we're funded by the egg industry, right?

Josh: 47:55

Thanks a lot for that. They're a sponsor.

Heather: 47:59

So can I imagine that there's a difference in fatigue? I can imagine that there is a difference in fatigue due to what I know about the physiology of the gut microbiome. Is that going to happen with every person? I mean, they specifically would not take people if they had already been avoiding gluten.

Josh: 48:15

Right.

Heather: 48:16

So it is possible that it's just a gluten-free diet that is actually working for these people.

Josh: 48:22

That's true, right, and or just you know, you ask them if they're less tired and they're excited, and they say, yeah, we're less tired. But you're right, there is a mechanism. I guess it's just like I don't know. We know that with open-label trials, like subjectively reported outcomes are so suspicious and yeah, it would have been very powerful if we had some objective outcome that like actually moved the needle. But yeah, good point, fair enough. Okay, cool, the beta-hydroxid butyrate, for example, right, so like that's the thing. It's like all that we've really showed with confidence is that a ketogenic diet creates ketosis, and I'm like, well, whoop-de-doo.

Heather: 49:01

Well, okay, so whoop-de-doo except. And so let's talk about that physiological pathway for just a second, because we're science nerds, right.

Josh: 49:09

All right.

Heather: 49:10

So there's a researcher at UCSF who's been actually studying in animals not in humans, but he's been studying beta-hydroxid butyrate and reducing IL-17, which is the driving cytokine for autoimmunity. So physiologically I would expect that a ketogenic diet would reduce autoimmune picture because it's reducing IL-17. The fact that they didn't see a reduction in the ketogenic diet for me was really interesting, because all the animal studies suggest that you get beta-hydroxid butyrate up and your IL-17 is going to take a nosedive and your autoimmunity is going to go away. So there is a movement to now do ketogenic diets in autoimmune patients. I don't know if this is because it's too small. I don't know if this is because it was too poorly designed, Like. Why they didn't see a shift in autoimmune outcomes is somewhat interesting. Or maybe it's because these were not the most severe patients. The most severe patients were in the paleo group.

Josh: 50:14

Did they measure any of those sort of? We don't have that biochemistry right. We don't have those labs. That would be neat.

Heather: 50:21

That would be neat. And no, we don't have those labs. Well, we have beta-hydroxid butyrate.

Josh: 50:27

comparatively, yeah, but not the autoimmune markers, but not IL-17.

Heather: 50:33

No, Again. Nobody consulted me. I would have told them Okay, so if you're listening.

Josh: 50:38

Dr Walls, if you don't hate us already, which she probably does there is available for consult. She's quite good. We work with her. Awesome, okay, so excellent. So this was great. I know it was a little bit haphazard and all over the place. What happens when you get three nerds together to talk about a paper? So hopefully the one listener who's left actually there will be more than one, because Heather and I are going to make this a signed listening for our class that we're doing. So there we go. That'll juice our numbers a little bit, so awesome. So thank you to your listeners for hanging with us and Heather, thank you for joining us for another episode. We'll have to have you back at third time.

Heather: 51:14

I would love to come back.

Josh: 51:16

All right, Take care everyone. If you enjoyed this podcast, chances are that one of your colleagues and friends probably would as well. Please do us a favor and let them know about the podcast and, if you have a little bit of extra time, even just a few seconds, if you could rate us and review us on Apple podcast or any other distributor, it would be greatly appreciated. It would mean a lot to us and help get the word out to other people that would really enjoy our content. Thank you, hey y'all. This is Josh. You know we talked about some really interesting stuff today. I think one of the things we're going to do that's relevant. There is a course we have on Dr Journal Club called the EBM Boot Camp. That's really meant for clinicians, to sort of help them understand how to critically evaluate the literature, et cetera, et cetera Some of the things that we've been talking about today. Go ahead and check out the show notes link. We're going to link to it directly. I think it might be of interest. Don't forget to follow us on social and interact with us on social media at Dr Journal Club, Dr Journal Club on Twitter. We're on Facebook, we're on LinkedIn, et cetera, et cetera. So please reach out to us. We always love to talk to our fans and our listeners. If you have any specific questions you'd like to ask us about research, evidence, being a clinician, et cetera, don't hesitate to ask. And then, of course, if you have any topics that you'd like us to cover on the pod, please let us know as well.

Introducer: 52:37

Thank you for listening to the Dr Journal Club podcast, the show that goes under the hood of evidence-based integrative medicine. We review recent research articles, interview evidence-based medicine thought leaders and discuss the challenges and opportunities of integrating evidence-based and integrative medicine. Be sure to visit www.drjournalclub.com to learn more.