Episode 3 - The good, the bad, & the in between

In Episode 3 of "Fact Check Your Health," dive deeper with Katie and Sydney as they discuss the different types of health research studies. This episode is a treasure trove for anyone looking to understand the different types of studies and their impact on the health information we consume daily. 

Katie and Sydney break down the scientific jargon, making sense of randomized controlled trials (RCTs), observational studies, systematic reviews, and meta-analyses using everyday language and relatable examples. They emphasize the importance of distinguishing between causation and correlation, explain the placebo effect and confounding factors, and provide practical tips for evaluating research findings. This episode equips listeners with the knowledge to better understand research claims and make informed health decisions.

Podcast Outline:
0:55 - Not all research is created equal
2:10 - Overview of experimental & observational studies
5:26 - What is a Randomized Controlled Trial (RCTs) and why they are the gold standard for research
6:30 - Placebo effect
8:35 - Confounding factors
9:40 - Example of a confounding factor
10:30 - How to determine if a study is an experimental study or observational study
11:05 - Systematic reviews and meta-analyses
12:40 - Episode recap

For more information and additional resources check out the Fact Check Your Health website at https://factcheckyourhealth.squarespace.com

Disclaimer: The information provided is for educational and entertainment purposes and is not intended as medical advice. For medical advice contact a licensed medical provider.

Transcript

Katie: 0:03

All right, welcome back to the third episode of Fact Check your Health, the podcast where we break down all the health research methods to make them easy to understand. I'm your host, katie, and joining me today is my co-host, sydney, so let's go ahead and div e in.

Sydney: 0:15

So before we start today's podcast, we just want to do a quick recap on what we covered in the first two episodes.

Katie: 0:21

So in the first episode we covered ways to spot unreliable information and some ways to find reliable information, and in the second episode we went through what you'll see when you look at an academic article and what all of that information means. Now in this episode we're going to go into detail about how you can figure out which information you should believe, which information you should be skeptical about, and give you all the tools and resources you need to be able to make that evaluation on your own Right.

Sydney: 0:48

So today we're going to be explaining the different types of research studies, because it's very important for you to know that not all research studies are created equal. Just because somebody said, oh, a research study showed blah, blah, blah, first off, even if they are interpreting it correctly, the research study itself might not be a very strong study or it could be conflicting other studies that have a stronger methodology. I love talking about this because you've all seen these crazy documentaries that make these claims based off research studies, and today we're going to explain to you why those claims might not be accurate and how to find a type of studies that can actually answer the question that you're looking for.

Katie: 1:26

And, like you said, it could be really confusing to know what to believe. So what we're going to do in this episode is break down all the different types of studies and tell you the pros and cons of each study type, and give you information on what the best type of study is and what type of studies you should be looking for when you want to find health information.

Sydney: 1:43

Yeah. So in my personal opinion, this might even be the most important episode that we have, because this is going to help you really differentiate between all of the BS that's out there and all of the conflicting studies. There's so many research studies that conflict each other and it can make you feel like research is unreliable, and that's not true at all. A well done study is very reliable. In fact, if you know how to just find the best type of study, then you can find that reliable information and you can make the best informed decision possible. Generally speaking, there are two main categories that most research studies fall under, and these are experimental studies and observational studies.

Katie: 2:18

So the first umbrella is experimental studies. So, put simply, an experimental study is typically going to be a study that uses a control group and an experiment group to determine cause and effect. So let's say, for example, that you're trying to determine whether a new drug works. You're going to compare a group of people that receive that drug with a group of people that received a placebo drug or a fake drug, and then you're going to look to see if there was a difference between the two groups Exactly.

Sydney: 2:45

Now let's talk about another kind of study called an observational study. So, unlike experimental studies, observational studies instead look at existing information to find patterns and connections between different variables or concepts. So, in simple words, observational studies don't involve directly changing things or separating groups, like experimental studies do. Instead, researchers gather data from real life situations and see how different factors might be linked. For example, let's say we want to know if drinking coffee is connected to heart disease. Instead of conducting experiments and randomly assigning participants to drink coffee or not, like researchers would do in an experimental study, in an observational study they would instead collect data from a bunch of people and then see if those who drink more coffee also have heart problems.

Katie: 3:33

Yeah, exactly. So these observational studies can also come in different forms so they could follow a group of people over time, or they could compare people that have a condition with people who don't have a condition, or they could look at just specific moments in time Exactly, and observational studies are important because they help researchers find patterns between different types of variables or things of interest.

Sydney: 3:57

So, while experimental studies focus on cause and effect, observational studies look at how things are connected in the real world. We need them because a lot of times they're done when conducting experimental studies would be challenging or not possible. So there are many times in health research specifically where it's difficult or impractical to do an experimental study. So, like the example with coffee and heart outcomes, if you did an experimental study you would have to follow those people for 40 years to see if coffee impacted their heart outcomes. But that's not really possible. We can't actually follow people in a study in a randomized experimental way for 40 years. So in those scenarios, observational studies are a valuable alternative to gaining important information and insights.

Katie: 4:39

And another example of why you can't always do an experimental study is because sometimes it might be unethical to require someone to do something. So let's say, for example, that we're trying to figure out the long-term effects of smoking on human health. It would be pretty impractical and unethical to assign individuals randomly to either smoke cigarettes or not smoke cigarettes for an extended period of time, since asking them to smoke a cigarette potentially exposes them to significant health risk. So in this case, what you would do instead is you would do an observational study, and that way you could track people over time and see if there's association between smoking and human health.

Sydney: 5:16

So now, that we've talked you through the two main areas of studies. Now it's important for us to explain the strongest, or in other words the gold standard, type of study. So in the world of health research, randomized controlled trials, also referred to as RCTs, are going to be considered the gold standard RCTs are super powerful because they allow us to determine cause and effect by using a control group.

Katie: 5:40

So to just give a brief overview for someone who might not be super familiar with the phrase randomized controlled trial essentially what a randomized control trial is is an experiment where you're going to randomly assign people to two different groups. So one group is going to receive the treatment or the drug or whatever it is that you're testing, and then the other group is going to get a fake treatment or a placebo, and then people are going to be randomly assigned to one of those two groups. So by randomly picking who gets which treatment, we can make sure that the groups are similar to start off with. That way, at the end of the study, if we do see any differences between the two groups, we can say that that difference is caused by the treatment or the drug or whatever it is that we're testing and not some other factor.

Sydney: 6:23

So, like Katie said, in randomized controlled trials there will be a placebo group who gets a fake treatment. This is important because you may have heard of this before in passing, but there's this thing called the placebo effect and basically, if a person thinks that they're getting any sort of treatment, their symptoms, or whatever the outcome is, will improve just off the fact that they think that they're getting a treatment. This is the power of the human brain If you think you're going to get better, you will get better.

Katie: 6:50

So that's why we have to have the placebo group and, to put it in an example that I'm sure we can all relate to caffeine and coffee.

Katie: 6:58

Several studies have been done that look to see if caffeine actually has an effect on people or whether sometimes the placebo effect plays in there.

Katie: 7:06

So if you're anything like me or Sydney and you need that coffee in the morning to give you a little pick me up, usually even just after the first one to two sips you already start to feel more alive and that could be a case of the placebo effect.

Katie: 7:18

So it might be that the caffeine actually isn't having that effect, but it's more that by drinking the coffee that placebo effect is kicking in and you're feeling the effects of something, even if it's not actually making a difference. So that's why a randomized controlled trial is important, because they're going to be testing a placebo compared to the actual drug so that they can make sure that any differences they see they can actually attribute to that drug and not just the placebo effect. Ultimately, rcts are super important because they give a strong evidence about what works and what doesn't, and while experimental studies are super powerful, conducting long term experiments can be really challenging, like Sydney mentioned earlier. That's why most studies examining long term health outcomes are observational in nature, because imagine trying to assign people to alcohol or non alcohol groups for 10 years. That would be a very difficult thing to do.

Sydney: 8:12

Yeah. So because of this, nutrition studies are often observational and this leads to ongoing debates about the best diets for long term human health, and this is why you probably have heard a plethora of mixed information. You know vegetarian diets are best for health, keto diets are best for health, and they all seem to support their claims. This is because we can pretty much only do observational studies with long term outcomes for nutrition studies and following that another downfall of observational studies can be this thing called confounding factors.

Katie: 8:41

So let's say, for example, the studies that were done forever ago that showed that drinking more wine led to increased heart health. But the thing is that in that study, whenever it was done in that period of time, drinking more wine was usually also correlated with higher income. So it might have been that it wasn't actually alcohol that was leading to those improvements in health, but it could have been other factors that were related to that person's socioeconomic status that was contributing to that instead of the alcohol.

Sydney: 9:10

Right. So in the case of those studies they were confounding factors like income and socioeconomic status. So basically, just to define confounding factors, these are other variables that can influence the relationship between a predictor. So the thing that we're studying and the outcome, confounding factors are typically things like age, gender, education and income. So to break that down, into a relatable example.

Katie: 9:33

Let's say that you have a study investigating whether being a Taylor Swift fan is related to having fewer wrinkles. So let's say that the study found that Taylor Swift fans on average have fewer wrinkles than non-Taylor Swift fans.

Sydney: 9:47

But unfortunately, what's probably going on in this study is that liking Taylor Swift doesn't directly lead to fewer wrinkles, but there's a confounding variable of age. So people who like Taylor Swift tend to be younger and since younger individuals have fewer wrinkles, the observed difference is due to age and not actually liking Taylor Swift, Though I do believe Taylor Swift will keep me young forever, but yeah.

Katie: 10:10

I definitely wish it was true that being a Swiftie would lead to less wrinkles, but ultimately, that's why RCTs, or those randomized controlled trials, are considered the best research method, because by recruiting participants who are similar in terms of age, income and other factors, researchers can keep those potential confounding factors as similar as possible. So now that we've talked about the different studies, whenever I'm reading a research abstract, how should I actually determine what type of study it is? That's a good question.

Sydney: 10:39

So, as a rule of thumb, if the abstract doesn't explicitly mention that it's an experimental study with a control group, or it doesn't say that it's an RCT. It's likely an observational study. Observational studies might be referred to in an abstract as either an observational study or a more specific type of observational study, like a case control study, a cohort study or a cross-sectional study, and there might be a few exceptions to this rule.

Katie: 11:04

So in addition to experimental studies or observational studies, there's also papers that are systematic reviews or meta-analyses. So in a nutshell, a systematic review is basically like a supercharged study that kind of brings together a lot of other studies to give us a clear understanding of a specific question or topic. So basically in a systematic review, the researcher is going to go and find all of the good quality studies that have been done on a topic and then put all of that information together so we can get a more accurate and reliable answer to that question. Meta-analyses are kind of similar but a little bit different, so I'll let Sydney describe what a meta-analysis is Right.

Sydney: 11:41

So, in a similar process, meta-analyses take it just a step further by then actually combining data from these multiple studies and Doing a statistical analysis on all of these data together. So here's a simple explanation of how it works. Imagine if you have a bunch of studies that have investigated the same topic or research question. Each study might have different findings or results, but a meta-analysis combines data from multiple studies to get a more reliable estimate of the overall effect. It helps identify patterns, evaluate the strength of evidence and draw more confident conclusions about a particular research question or topic. However, the strength depends on the quality of studies included. So a meta-analysis that includes strong studies like RCTs are a more reliable source of information than a meta-analysis that includes studies like observational studies.

Katie: 12:29

Whenever you're looking through PubMed or Google Scholar, if you see a meta-analysis or a systematic review, that could be a really good resource for you to go to to get a summary of all the information that's out there on the topic. So now just to recap everything that we've covered in today's episode Experimental studies are going to be best for telling us if whatever we're studying actually causes something else, whereas observational studies observe behaviors in the wild and establish whether or not there's patterns. So, if available, we're going to want to rely on a randomized control trials, because they're the best source of information. But if those sources aren't available, observational studies could also be great and, similarly, systematic reviews and meta-analyses can also be a really good and reliable resource.

Sydney: 13:12

So that concludes today's episode of fact. Check your health.

Katie: 13:15

Join us next time as we teach you why the headline that you're reading might be misleading.