We talked about different interventions to tackle disinformation, the challenges in data gaps, existing and forthcoming research on fact-checking, media literacy, and platform measures, and areas where additional investment is promising to generate impact at speed and scale. We even promised to have another conversation in 2025 after this year's wave of elections to do a retrospect on our collective efforts to counter disinformation.
Tune in to hear more about Jon's and Dean's research and a thought-provoking conversation, including regulatory attempts around digital platforms in Europe and the lack thereof in the United States.
Jon Bateman is a senior fellow in the Technology and International Affairs Program at the Carnegie Endowment for International Peace. His research areas include disinformation, cyber operations, artificial intelligence, and techno-nationalism. Bateman previously was special assistant to Chairman of the Joint Chiefs of Staff General Joseph F. Dunford, Jr., serving as a speechwriter and the lead strategic analyst in the chairman’s internal think tank. He has also helped craft policy for military cyber operations in the Office of the Secretary of Defense and was a senior intelligence analyst at the Defense Intelligence Agency, where he led teams responsible for assessing Iran’s internal stability, senior-level decision-making, and cyber activities. Bateman is a graduate of Harvard Law School and Johns Hopkins University.
Dean Jackson is the principal of Public Circle Research & Consulting and a specialist in democracy, media, and technology. In 2023, he was named an inaugural Tech Policy Press reporting fellow and an affiliate fellow with the Propaganda Research Lab at the University of Texas at Austin. Previously, he was an investigative analyst with the Select Committee to Investigate the January 6th Attack on the U.S. Capitol and project manager of the Influence Operations Researchers’ Guild at the Carnegie Endowment for International Peace. From 2013 to 2021, Jackson managed research and program coordination activities related to media and technology at the National Endowment for Democracy. He holds an MA in international relations from the University of Chicago and a BA in political science from Wright State University in Dayton, OH.
Listen to previous episodes at speedandscalepodcast.com and engage with the host at baybarsorsek.com or on LinkedIn to join the conversation.
We talked about different interventions to tackle disinformation, the challenges in data gaps, existing and forthcoming research on fact-checking, media literacy, and platform measures, and areas where additional investment is promising to generate impact at speed and scale. We even promised to have another conversation in 2025 after this year's wave of elections to do a retrospect on our collective efforts to counter disinformation.
Tune in to hear more about Jon's and Dean's research and a thought-provoking conversation, including regulatory attempts around digital platforms in Europe and the lack thereof in the United States.
Jon Bateman is a senior fellow in the Technology and International Affairs Program at the Carnegie Endowment for International Peace. His research areas include disinformation, cyber operations, artificial intelligence, and techno-nationalism. Bateman previously was special assistant to Chairman of the Joint Chiefs of Staff General Joseph F. Dunford, Jr., serving as a speechwriter and the lead strategic analyst in the chairman’s internal think tank. He has also helped craft policy for military cyber operations in the Office of the Secretary of Defense and was a senior intelligence analyst at the Defense Intelligence Agency, where he led teams responsible for assessing Iran’s internal stability, senior-level decision-making, and cyber activities. Bateman is a graduate of Harvard Law School and Johns Hopkins University.
Dean Jackson is the principal of Public Circle Research & Consulting and a specialist in democracy, media, and technology. In 2023, he was named an inaugural Tech Policy Press reporting fellow and an affiliate fellow with the Propaganda Research Lab at the University of Texas at Austin. Previously, he was an investigative analyst with the Select Committee to Investigate the January 6th Attack on the U.S. Capitol and project manager of the Influence Operations Researchers’ Guild at the Carnegie Endowment for International Peace. From 2013 to 2021, Jackson managed research and program coordination activities related to media and technology at the National Endowment for Democracy. He holds an MA in international relations from the University of Chicago and a BA in political science from Wright State University in Dayton, OH.
Listen to previous episodes at speedandscalepodcast.com and engage with the host at baybarsorsek.com or on LinkedIn to join the conversation.
Hi Bill, welcome to the show. It's such a pleasure and privilege to have you on the first ever episode of Speed and Scale podcast. I look forward to diving deep into your recent piece at Neyman Lab, but before we give our listeners a spoiler alert, I just want to make sure that you give us a chance to walk us through your journalism and fact checking. Maybe the best place to do that is to take you back to 2007. and ask you about the early days when you had the idea of the PolterFact, the largest political fact checking outlet in the United States right now, if not in the world, and let us know what made you to start PolterFact and introduce us with this concept, which inspired a lot of people like myself all around the world. Sure, well first thanks for having me, Barbarz. I'm honored to be the guest on your first episode. So the origin story of PolitiFact is about guilt. It's about my guilt when I was the Washington bureau chief of the St. Petersburg Times, the newspaper that is now the Tampa Bay Times. And I felt guilty that I was not doing enough to fact check. the politicians that I covered in Congress and the White House. And so I went to my editors in 2007 and I said, hey, instead of covering the 2008 presidential campaign in the traditional way that we normally do, let's start a fact checking website. And they were supportive of that. I was just talking to one of them a few minutes ago who was here in our hallway, Paul Tash. And he reminded me that. one of them. keystones to our idea was that we keep it light, that we not be too serious, that we not be boring. And that appealed to me because I'm not too serious. And I think one of the things I like about journalism is to simplify complex things. And so fact checking was perfect for that. So we launched in August of 2007 going strong today. That's great. I mean, I remember the United Facts of America conference, I guess, like two years ago when you were sitting on a virtual panel with some of my most favorite people in the world, like Angie, like Neil, talking about the origins of Poltifact and how things have changed in the last 15 years. Well, right now, I guess, like in the 17 years since the launch of Poltifact. And our listeners cannot see this right now, but you have a Duke Report to staff who are there on your right now. Which kind of like, you know, makes me interested in asking this question around like the journey that you see, you know, in fact checking, like how the technology has changed or even like transformed the practice. Where do you think we were back in like 2007 with Poltifact, FactCheck.org, Snopes, early pioneers of the practice and how the practice has transformed the last 17 years? particularly with the introduction of the social media, online misinformation, and where do you see we are right now before we take a look in the future. Sure. So fact checking is different than other kinds of journalism because it has archival value that lasts a little bit longer. So if you think about a typical news story, it's valuable now. You know, it's the news. We know what's happening now. But a fact check... is valuable for a longer period of time because people want to know is that true? Is that thing that I keep hearing true? So a fact check that we do today still has value in a week or two weeks and it also has value in a different way and this was one of PolitiFact's innovations. It has value to tell you how is a politician doing, how often does a politician get the facts right and how often does a politician get the facts wrong? And so that is valuable. And from the start, I think we realized at PolitiFact that the content should be structured like data. And this was, I think, part of the innovation of PolitiFact. partly my idea, but also the brilliance of a guy named Matt Waite, who was a data journalist for the St. Petersburg Times, now is a faculty member of the University of Nebraska. And so that now today is what a lot of fact checkers are doing thanks to something that we've done here at Duke, which is share that concept. through a data standard called claim review. And the idea is to structure fact checking so it can be found more easily in databases, whether it's through a search engine like Bing or Google, or by anyone who just wants to take the data and create a database and look at it in different ways. And so I think fact checkers from the beginning, have been particularly sophisticated at using technology. I mean, that's striking because I've been in the field for about 10 years. Obviously, I'm not one of the few who were in the early days, but I think it's fair to say that I had the chance to witness the growth of the global fact-checking community, especially since early 2010s and with a lot of organizations in the Balkans, in Eastern Europe, then in India, APAC. I think that helped us to get to the point where now we can talk about a global fact checking community. And there's like a lot of tools and resources out there right now. But on the other hand, Duke Report is that also surveys the fact-checking organizations around the world. And now there is at least a plateau of the growth when it comes to the global fact-checking moment. Do you think that has? a lot to do in terms of like, you know, this is getting a bit like, you know, widespread practice now there's not a lot of room for innovation or do you think there is this tipping point where fact checking is becoming much more part of the mainstream media and there's not that necessarily amount of new organizations or launching their fact checking projects as a standalone organization. How do you see that? Like, you know, pays with the growth of fact checking around the world. Well, I think it can be several different things. I think there's more fact checking that's being done by traditional news organizations inside their news articles. So, and this is, I think, one of the trends of the late teens, like 2016, 2017, when you started seeing news articles that would say, you know, this politician falsely claimed whatever. And that kind of thing never would have happened 10 years earlier, you know. political reporters were scared to death of doing anything like that because they didn't want to they didn't want to offend their sources so that's a good trend and that happens even more today because reporters are like hey I can say that and not get in trouble you know because I think everybody realizes these politicians are just saying a bunch of false things and it's okay to say that it's false so that's good. The other thing that's happening is I think you're seeing lots of different things going on with technology whether it's trying to push back on The different ways that politicians are spreading misinformation on social media And that's gotten getting much more complex whether it's detecting fake images whether it is now trying to in some cases use our own generative AI to create fact checks. That's something we're beginning to experiment with here at Duke is can we use generative AI to basically create more fact checks every day to help fact checkers so that we it's like a force multiplier. because politicians say the same things in Arizona that they're saying, in Virginia that they're saying, in Florida. So if it's been fact-checked in one of those states, we can use generative AI to clone that fact-check. adapt it to the new claim and then through publishing agreements publish those fact checks. So I think that's the great hope with technology. It won't happen without humans. I do think you've got to put a human in the loop to make sure that those fact checks are reviewed. We have a product we're calling Half Baked Pizza. And the concept being that you need a chef to review the half-baked pizza before you put it in the oven and Likewise, you need an editor to review the half written Fact check that's been written by generative AI. So you make sure it doesn't contain any hallucinations Which is sort of the equivalent of a bad pepperoni That's true. I mean, nobody wants a bad pepperoni pizza. I'm just curious and, you know, please see this question as like a prologue to my next one, which will be about your piece for the Neyman Lab. Are you worried about the increase of fact checking around online misinformation and the, you know, the decline of political fact checking? Does that something I mean, is that something that keeps you Worry about the future of fact checking when it comes to holding the powers from the account. worried about the decline meaning The number of fact checking projects spending less resources on political misinformation while they spend. Yeah. I mean, you know, I don't think there's any scenario for combating misinformation in a serious way that does not involve more fact checking. And we, I mean, we just have to realize that the raw materials of combating misinformation are fact checks written by humans, you know, like AI is fine. but it's not dependable enough yet to be able to combat misinformation. And so we need more people writing more fact checks. Now that, you know, that's like a friend used to say, you know, of course, Bill's going to say that when you're a hammer, everything looks like a nail, you know, but, but really, like, I do think we need more. fact checkers doing more fact checks. We did a report this past year here at Duke called Fact Deserts and it looked at the likelihood that an American politician other than, so forget like presidential candidates, of course they're getting fact checked a lot, but how often is a state legislator fact checked? How often is a member of the House of Representatives in Congress fact-checked? Answer, hardly ever. And so, governors, hardly ever fact-checked. So, you know, these guys can say anything they want and no one holds them accountable. Well, we need journalists to do that. And so our report showed the frequency of this, which was frightening. So we need more journalists to be holding politicians accountable for what they say. I mean, this is great because I think it will be this perfect segue to the alarm bell, basically, you're the drink at the end of the year in your piece for the Neyman Lab for your end of year predictions. You rightfully call fact checkers to revisit how they communicate their work with those who need it the most. And to quote you... You say the old way of publishing fact checks, putting them on websites and promoting them through social media isn't getting them to the people who need them. And it's time to reimagine how fact checkers publish and broadcast their work. How do we do that as fact checking organizations or outlets where they're part of larger media outlets or standalone organizations in a world where the primary funders of fact checkers are either platforms. or different philanthropy organizations around the world, especially in the case of the US. I'm just interested in your thought on like, what is the way out and tackling those fact deserts in the light of this challenge that we have as a community. Sure. So, you know. We really are still publishing the way that we published fact checks 20 years ago. The way that FactCheck.org published 20 years ago. Putting stuff on a website and waiting for Crazy Uncle Bob to come to the website to read it. Well, hey, news flash. Crazy Uncle Bob is not coming to Fact Checkers websites. to find that he's wrong. So we've got to find out a way to get to crazy Uncle Bob. And so I think we need to be much more willing to work with behavioral scientists and look at how people are consuming information and consider different kinds of partnerships and creative ways to package fact checks. Here's a wacky one we've talked about. And this is probably way out there, but it's the kind of thing that takes the fact check to the person. We've mused about, hey, why don't we put fact checks on video screens at gas pumps? That's a screen. You could do a short fact check segment, put it there. No, you know, that's a place where everybody fills up their tank, who doesn't have an electric car. And that's a way that you could target people who are not seeing fact checks otherwise, other ways. And I think we need to think and maybe that's too extreme. And maybe that's too in your face. But I think we need to think about creative ideas like that. are that go beyond waiting for Uncle Bob who, you know, has some wacky COVID theory or whatever to come to a fact-checking website because it's not happening. And so, and we know from just following people's habits, the people who need the corrections are not going to seek out fact checks that tell them they're wrong. And so they're not coming to the fact checking websites. And so it's wonderful that we're putting it there as a archive, but it's not getting to the people who need it. I mean, that's a great point, especially when it comes to ways in which we need to be more creative about and how we do things with fact checks. I think we have a lot of way to go and there's a tremendous amount of best practices around the world in which fact-checking organizations collaborate with local nonprofits or different interest groups or people with certain community leadership. They do an amazing job. getting those fact checks out there. But I'm equally interested in your take on like, how we can get creative in terms of like, publishing the fact checks or producing the fact checks. Cause the one part of the problem is about distribution. The other problem is like, basically the creation of the work. And you keep saying that we don't have enough fact checks. I think I've heard this from you million times already and you always, I mean. Yeah, especially someone who has spent years in exploring automated fact checking. What is your take on automated fact checking and what we should understand when we talk about automated fact checking in 2024, especially in the age of generative AI? Well, I think that's changed. You know, if you asked me about automated effect checking five years ago, it would be more like an appendix to content. It would be more like, hey, by the way, that thing that he just said, you know, that may not be true. And now I think we need to. I guess I'm more focused on using it to generate a greater mass of fact checks and get it to a. broader audience and to create more and more fact checks. But think about it, you can use generative AI to do lots and lots of things. You can use it and write in different voices. You can generate customized newsletters. You can write in different kinds of tones. So you could theoretically just say, okay, here's a fact check. for this kind of person and try to make it more appealing. That would be along the lines of what. the American Press Institute did about five years ago with an interesting experiment where they showed that the way that journalism is written can change its appeal to different kinds of audiences. So we could try that. There's just been no experimentation in this area to speak of. And I think we need to just try different kinds of formats. and just lots and lots of experimentation. I mean, that's like, you know, just reminding me the conversation. I think you are referring to the Tom study that he did when he was at the API. Roslind Dale talking about this at a global fact in 2019, I guess. And it was, I guess, like inspirational for a lot of fact checking, um, practitioners at the time, because I've seen many efforts to, uh, in the, like, you know, tailor the specific messaging and, uh, you know, positioning around like how they communicate. the fact checks, just to shift lanes and probably rev it up the conversation on a brighter note, I'm very... I think it's fair to say that I'm a cautiously optimist when it comes to the future of fact checking. And the reason why I'm optimist is that the amount of thought process that has higher and higher, it's just getting bigger, and more people are getting interested in the field. Now there is definitely not a shortage of organizations and entities interested in fact checking. How do you think the fact checkers, on the other hand, could play a larger role in the broader trust and safety space? Because one thing that I've been witnessing lately is that the platforms, particularly working with fact checkers, In every limited scope where it is more like a B2B relationship, they provide you with a queue or dashboard, you do fact checks for them, and they make their content decision, content moderation decisions. But there's not a lot beyond that fact checkers are being interested with, in their, with their expertise by the platforms. Do you think fact checkers have a bigger role to play, especially this year where we have many elections? taking place. Yeah, I think. I think it's too easy for the fact checkers to just sort of wait for the platforms to say, okay, well, here's what we're going to pay you. Sign on the dotted line. I think the fact checkers should get more involved with universities in testing different kinds of things. But I think there needs to be a shift in their mentality. Fact checkers are journalists and I think they have for too long looked upon their role as well we're just gonna put the information out there and People will come to it and you know, maybe they'll be swayed by it. Maybe they won't but we're just Information providers. Well, that's not good enough we We need you know, our goal shouldn't just be to be this Passive transmission belts of information our goal should be to get people the facts so they can Know what's true? And so I think we've and this is a change for me because I used to answer that question When people would ask it in the interviews, I used to ask it by saying oh, you know, we just put the facts out there It's up to you know, it's up to mable to decide how she wants to vote And i'm not telling mable how to vote at all But I do believe that we need to be more, I think we need to connect dots more than we have. You know, like on some of these bigger issues, I think we've been very clinical and sort of said, well, this thing's wrong and that thing's wrong. And we haven't said, hey, put all these things together and what's going on? Yeah. The narratives, I guess, like, you know, really matters here to help the audience and even not to call them readers anymore, because to your point, we need to think more holistically and see our audience in a bit more multimodal way, if possible. So the last question I'm really interested in your take is, and I'm going to be asking this question pretty much to all of our guests this year. It's the year of elections. We have elections in the US. Prior to that, we have elections in India, in the largest democracy in the world, then very likely elections in the UK before the summer, and then the European Union elections. How do you think fact checking is gonna survive after this tide of elections this year, and what is gonna be our takeaway in terms of how we deal with political actors? I know that you spend a lot of time thinking about this question in a much more intellectual space. to see the relationship between fact-checking, political actors, and how politicians at large see fact-checking, and if there are any differences between different political parties. Can you speak just a bit about the role of fact-checking in elections, and what it's gonna look like in 2025 in all those elections, and how fact-checking is gonna survive, and even like, you know, just grow as, you know, experience. is here. So I, we used to have a joke at PolitiFact that, and I would often say this in election years, but not just in election years, I would say it's the year of the fact checker. And you know, like in, after a while, people were like, didn't you say that last year? And didn't you say that the year before? It was like, yeah, but it really is this year. And so, you know. This year, it's going to be the year of the fact checker. And I think that I'm right, actually, because misinformation just seems to find new ways to spread faster, sneakier. And politicians seem to be much more willing to spread falsehoods than they have in the past. So. So I do think when we look back on this year, I think we will say, boy, there was an awful lot of really important fact checking. One key to that, fact checkers have to do a better job telling their story. Journalists tend to be modest and they don't tend to do a good job telling their story. And I think this is something that we really need to do better. I think the IFCN needs to take the lead in doing this and individual fact checkers need to do better. But I think when it's all done, I think we'll look back on this year and we'll say 2024. It was the year of the fact checker. then let's do this again in 2025 and reflect back on this year. Bill, I appreciate it. People don't call you to do no fact checking for nothing. I had the privilege to call you on stage, call you to stage with that title. And you are such an important North Star to all of us, but in the fact checking community. And I think a lot of our listeners. Since this is the first step, I think I'm going to be within the factoring community. And I'm pretty sure they agree with me when I said that we owe you a lot in giving us inspiration and the courage to do this and can thank you enough for everything that you have done for the field and just look forward to what you have more to teach and show us. Thank you so much, Five Arts. It was great to be on the show. Thank you so much, Phil. Bye. so long.