Spry Fox: David Edery

Today on The Art of LiveOps podcast, we are joined by David Edery co-founder and CEO of Spry Fox.  Listen as he talks about the mindset shift to LiveOps in #gamedevelopment and how the nature of marketing has changed and what this means for games.

Transcript

James Gwertzman (00:05):

Hello. I'm James Gwertzman.

Crystin Cox (00:06):

I'm Crystin Cox. Welcome to the Art of LiveOps podcast.

James Gwertzman (00:15):

Who's our guest today?

Crystin Cox (00:17):

Today, we are interviewing David Edery. David Edery is the CEO of Spry Fox.

James Gwertzman (00:21):

I know David.

Crystin Cox (00:24):

Spry Fox has been around for a while now. They make a bunch of really great mobile games. They made Alphabear. They made Triple Town.

David Edery (00:34):

Hi. I'm David Edery. I'm the CEO and co-founder of Spry Fox. That was the first time we had this idea that ... Almost think of it as a live TV show, something that there's constantly stuff being injected into it by the producers, and there's a reason to tune back in all the time.

Crystin Cox (00:46):

They made a lot of really, really well-designed mobile games that have been very successful. They've been on a journey, I think, over the whole course of Spry Fox from really more traditional game development, even though it was on mobile, to actually incorporating more live game elements.

James Gwertzman (01:02):

They're aggressively indie. Part of them, Spry Fox are great is they're really committed to doing innovative work. Never resting on their laurels or copying the same mechanic. They're also super distributed. I don't think they had an actual office.

Crystin Cox (01:13):

That is true.

James Gwertzman (01:14):

All of their people work remotely. They've got interesting lessons about distributed life development as well.

Crystin Cox (01:19):

Let's dive right in and talk to David. I've been making games for 16 years as a designer and a game director focused on Live Ops.

James Gwertzman (01:31):

I founded PlayFab five years ago, because I saw a huge gap in the kinds of access to Live Ops technologies game studios needed to be successful. We've put this podcast together, because a lot of the information on how to do Live Ops effectively just isn't out there. There's way more information about how to optimize your graphics pipeline or how to put together effective game design than there is on how to do effective Live Ops.

Crystin Cox (01:51):

Since it's pretty tough to go around yourself and just find everyone that does Live Ops and ask them questions, we did it for you.

James Gwertzman (01:57):

We've picked together a list of some of the top practitioners in the industry, folks who we think are really pushing the boundaries and doing some cutting edge work.

Crystin Cox (02:03):

We interview a different one every week and ask them about their experiences running live games, doing Live Ops and even having Live Ops disasters.

James Gwertzman (02:10):

Train wrecks are the best. That's where you really learn how to do things effectively.

Crystin Cox (02:14):

Tune in. We have another interview for you today.

David Edery (02:22):

We didn't take Live Ops really seriously until basically Realm of the Mad God, which we co-developed with another studio called Wild Shadow. I think we were working at 2012, 2013 or something. That was the first time we had this idea that, "Look, we're going to be adding new content to this game on a really regular basis. We're going to be having things that feel like events in the game," although we never actually ended up getting to that in a meaningful way back when we were working on Realm.

David Edery (02:45):

We've done more of that in Alphabear 2, but we're going to try to make this thing feel like ... Initially, we started off in the free-to-play space with a bunch of very, very simple games, like Steambirds and a game called Bunny, the first Triple Town. They were all free-to-play, but none of them actually had anything dynamic, for lack of a word, built into them. You came, you played the game, that was that. It never really changed. Six months might go by, we might add a new mode to the game or something like that, but that's not Live Ops, obviously.

James Gwertzman (03:16):

Cool. Realm was the first game where you really started to add that in. What were some of the learnings you made during that process? What were some of the mistakes you made maybe, or what were some of the things that came out of that?

David Edery (03:30):

The main thing that we learned is that it is possible if you want to. We were operating at a hyper fast pace, but it is conceivably possible to be literally injecting interesting new stuff into the game every single week. We were iterating that rapidly. We were releasing at times, daily builds, which was nuts. I don't recommend doing that, but there were times where we were doing builds literally daily. Most of them, obviously would just be bug fixes and tweaks, but we were adding potentially a new character here or a new enemy there or changing an entire new dungeon in some way, shape or form.

David Edery (04:01):

The community loved it. It made them super excited to check in every single day and see what had changed and how it had changed and frankly, to give feedback on the changes, which is from a developer perspective, an incredible thing. You can choose to look at it as a negative, like, "I have to answer to these people," or you can see it as like, "Thank God, I'm no longer working in a black box. I know what I should do, because they're telling me what they want." They don't always tell you ... Sometimes what they say they want isn't actually what they want. It's what they think they want. There's some expertise that comes in, teasing those things apart.

David Edery (04:32):

In general, having that clear signal of, "Here's what they're saying they want," is enormously valuable. I never want to go back to the days of developing where it's like, "We have to have the game perfect at launch, and then that's that," and it'll succeed or fail based on that. That's terrible.

Crystin Cox (04:47):

That shift in focus, how did that change your company's culture? How did you have to approach game development differently in order to get to that place?

David Edery (04:55):

The good news is that we were so small back then. We were already very dedicated to the idea of rapid iteration, just in terms of prototyping and stuff, how we concepted games in the first place, that it was a natural evolution for us. We just rolled from, "We're going to iterate early and often on our prototypes," to "We're just going to keep doing that. That process isn't going to stop." That worked out really well for us. There's certain hard lessons we learned in terms of ...

David Edery (05:26):

For example, I think we naively expected that meta-game would be something that would be easy to iterate on after launch. It turns out, no. If you make certain fundamental mistakes when you first design meta-game, you find yourself with one hand tied behind your back and you can't really easily fix that. For example, with Triple Town, which is probably our most famous example of both a successful game and a game where we screwed this up in a major way, Triple Town launched really without any of the bones to keep people engaged for a long period of time beyond the core game itself. Which is still to this day, remarkably sticky.

David Edery (06:02):

Interestingly enough, Triple Town, even though it came out back in 2011 or whatever has a higher DAU count than any mobile game we released more recently, because people love the core game so much that they keep playing it. Which is sad, because it makes me think, "God, if we had actually gotten the meta-game right, imagine what we could have done," but anyway. Triple Town, it has the core game, which is delightful and it has a couple of aesthetic things that you can unlock, but they don't really impact the game in any significant way.

David Edery (06:30):

It's a single player game, so it doesn't matter anyway. No one can see the aesthetic things you've unlocked, so they're not particularly powerful. Later when we were like, "Okay, well, how do we make more money? How do we improve this game's retention? How do we, how do we, how do we," the answer was, "I don't know, because there's nothing that people care about aside from playing the core game again and getting a slightly higher score or building a slightly better building in that stat. That's all that really matters to them."

David Edery (06:58):

One of the things we learned from that is you have to be thinking from the very beginning, "What are the progression systems? What are the social systems?" Actually, the ladder is something we're particularly interested in personally. The, "What are the progression systems and social systems and other systems that will make people care for a long time?" Then, "Okay, how are you going to take advantage of the systems? Are you going to plug into that in the future?" You don't have to have it all planned out in the beginning.

David Edery (07:21):

You don't have to have it all built out in the beginning, but you have to have a vague idea of, "Because I have this, people are going to care." In Alphabear 2, for example, we learned from a lot of these mistakes. We have hundreds of bears you can collect, and the bears are really meaningful and have interesting powers and can do different things. A, people just like collecting cute bears, so that's great. B, these bears actually have a material impact on whether you can succeed in certain modes of the game. So, "Okay, good. There's a functional benefit as well."

David Edery (07:47):

Now that I have these hundreds of things that you can collect, there's all kinds of stuff I can do that. I can use that to motivate you to win tournaments. I can use that to motivate you to win special events. I can use that to make you want to get further in the campaign, because as you get further in the campaign, you unlock more of these bears and on and on and on. There's all of these things that you can do. It all ties back in. Then the bears themselves, because they have powers ...

David Edery (08:07):

We haven't actually done this yet, but theoretically, I can have the ability to evolve those powers. Then, there's a whole new collection meta-game around evolving the powers and on and on and on. That's an example. If triple town had something like that, it would have been radically easier to both do Live Ops for it and just make it a more successful game in general, but it was missing that. It wasn't obvious how to retroactively shoehorn something like that into it, so it never really reached the level of success that it could have fully.

David Edery (08:35):

It got down a little bit, like 13 million people or something. Fine, but I truly actually believe it could have been a hundred million people had we not made these key mistakes.

Crystin Cox (08:44):

Right. I think that's a key thing you're hitting on there, which is levers. If you don't give yourself room in a game from the initial design, some levers that can be pulled and some knobs that can be turned, it can be difficult to make meaningful changes to a game over time. Having learned that lesson and hearing some of the ways you've applied it to Alphabear 2 now, what do you think the future is for you guys? What are you excited about as far as Live Ops for the future games?

David Edery (09:13):

For us, and part of this just comes to our personal goals, we want to bring people together. We want to help people develop meaningful friendships and have good relationships. In an ideal world, we'd even help figure out how to bring people on the left and right sides of the political spectrum together, but that's a big one. We'll see-

Crystin Cox (09:28):

Good luck.

David Edery (09:29):

Exactly. We'll see if-

James Gwertzman (09:30):

We're rooting for you.

David Edery (09:31):

We'll see if I make any impact on that in my lifetime, but given that those are our strong personal goals, and we also feel that the games that successfully do that, that bring people together and help them develop meaningful relationships, will just simply be more viral. We'll make more money. They'll just naturally be more successful. It's nice when a thing that you care about personally is actually a thing that also makes business sense. Anyway, that's what we're focused on.

David Edery (09:56):

We're thinking, "Going forward, can we make games that have such vibrant communities and do such a good job of bringing people together in pro-social ways that, A, the games are super easy to do Live Ops for, because you're just feeding off this existing community that's already excited to do things together. B, you can make money in ways that you feel good about. It's not really a coincidence that you look at League of Legends and Fortnite and games like that. Nobody complains about the modernization in those games. There's a good reason.

Crystin Cox (10:25):

That's not true.

David Edery (10:26):

Oh, really?

Crystin Cox (10:27):

I just think it's hilarious, because I'm sure at some point we'll have someone for Riot in, and they'll be like, "Let me tell you, I have a catalog of every complaint anyone's ever made-"

David Edery (10:36):

Fair enough.

Crystin Cox (10:37):

"... about the modernization," but the point is-

David Edery (10:38):

There's always someone who complains.

Crystin Cox (10:39):

... there isn't an angry mob.

James Gwertzman (10:41):

Not within the early games. It really felt like they were taking advantage of their players.

David Edery (10:44):

If you look at almost any ... There are certainly good exceptions, but the vast majority of single player free-to-play mobile games have monetization schemes that are like, "Hey." Certainly someone who's really into League of Legends or Fortnite would probably not be thrilled with that kind of monetization. They're more aggressive. They're much more loot box based, et cetera, et cetera. They're very pay-to-win in many cases. You don't have to do that in a game like Fortnite or League of Legends.

David Edery (11:12):

Obviously, those games are aberrations, so you shouldn't focus on them too much. Anytime you look at something that is at that scale of success, there's usually some weird externalities that you can't ever replicate. Nevertheless, the point is, because those games are so driven by their social components, that's part of the reason they can get away with that. Again, going back to what we were talking about like aesthetics. You can sell skins in a multiplayer game and people are excited to buy them, because other people will see them. Selling it in a single player game, you don't get nearly as much leverage.

David Edery (11:44):

We're very focused going forward on, "How do we build these vibrant communities, build games where people are really excited working together, and then leverage that in Live Ops in ways that are scalable?" So it's like, "Hey, today only, if you and your Guild accomplish X, you'll get Y," or whatever. There's a million different ways you can skin this cat. They don't actually require much effort from a development perspective. Literally in many cases, it's just flip a few switches on some system that you've designed. Insert new skin that your artists made in four hours. Good to go. You've got a whole new event that keeps people busy for a week.

David Edery (12:16):

That's the kind of thing you're looking for. You can do that when you have a game that was built from the ground up with these ideas in mind, and in particular, like I said, with a game that has a big vibrant community that's talking, keeping each other amped up, et cetera, et cetera.

James Gwertzman (12:32):

I want to jump on this community theme, because that's actually a trend that we're really seeing very closely here at PlayFab, which is we think that we had games product good, we had game as service. I think people are still really hung up on this notion of games as services, but we're really starting to see now this transition to games as communities, where the games are community platforms. You mentioned Fortnite. I think everyone's talking about the Marshmello concert in Fortnite. To me, it's not a jump the shark moment. It's a jump the ... Something happened that day.

James Gwertzman (12:58):

That was really an apocal moment in the gaming world we look back on when they had the world's most widely attended concert ever in a game. That was something special about that. I think we really are in this whole new era now.

David Edery (13:09):

I agree a hundred percent. There's so many benefits we're not even talking about that are intangibles, but are really meaningful. It's very clear when you go ... We're working on an MMO called Steambirds Alliance, and you jump into the Discord for Steambirds, which is still quite small. We're just doing closed betas. The people in there are doing so many good things for us. Any new user jumps in and has questions, they automatically answer those questions for them. They generally have been pretty cool about it. They're not like, "Screw, you're new." They're actually excited to have a new person that they can communicate this to.

David Edery (13:41):

They're absolutely our best evangelists. They're the ones out there telling people, "Hey, this game is awesome. Here's why you should play it," and trying to get all their friends to play. There's the traditional ... Obviously, this is an aspect of games marketing that we've been talking about for decades. It's not really a surprise that I'm saying any of this, but that's all still there and still just as important as it ever was, if not more important as things like Discord become more and more significant. Then on top of that, like I said, you have this dialogue going with them where you can literally just say, "Hey guys, what would make you excited? If we did X and Y in next month's event, would that be cool?"

David Edery (14:18):

"Oh, yeah. I'd love that." "Oh, no. I'd hate that," or whatever. It's a really meaningful relationship that results in a much, much more compelling longterm experience.

Crystin Cox (14:28):

I think of you guys as a company that cares a lot about community. Realm of the Mad God had a great reputation for having a very dedicated community. It was very well maintained as a community. As people who are so focused on community as a core part of your game design and a core part of your business model, how do you approach building a community on platforms that honestly I think we've struggled with as game developers to figure out, 'How do we make a mobile game have a great community?'" We usually don't have forums. We usually don't have these traditional things that we think of as building game communities.

David Edery (15:05):

I honestly wish I could give you a great answer for that. We've struggled that with ourselves, particularly because our mobile games are all single player games and it's hard to build a community around a single ... You can certainly do it, but single player plus mobile is in general a tough thing. I think driving people as proactively as you can to a place like Discord is actually a pretty good way to do it. We haven't done that. Recently, we tried. We've had forums. "Please go check out the forums." That hasn't really worked for us very well. We've tried the newsletter thing. That's obviously not [crosstalk 00:15:38] interactive.

Crystin Cox (15:38):

It's not as interactive.

David Edery (15:39):

Yeah, exactly. That hasn't really got ... We've actually really struggled there. To be perfectly honest with you, that's one of the reasons why we're less excited about mobile today than we have been in the past, because we feel so strongly that building a meaningful, interactive community is important. Making the kinds of games we've typically made on mobile, it's just really hard. Now, multiplayer game on mobile, sure. Maybe it will be just as easy as it is in the PC, but we've never done a multiplayer game on mobile, so I can't tell you from experience.

David Edery (16:09):

That's why a lot of our efforts have been transitioning towards things like the Steambirds, which is PC/console.

Crystin Cox (16:15):

Then, leveraging things like Discord more heavily there, because-

David Edery (16:17):

Much more.

Crystin Cox (16:18):

... that's where the player base is already.

David Edery (16:22):

They told us to go there, and not vice versa. You know what I mean?

Crystin Cox (16:25):

Yeah, which is great.

David Edery (16:26):

This is an interesting thing. Maybe we'll come to regret this someday. I don't know, but we don't own our Discord channel. We don't control it. It already existed. There were some hyper fans who were moderating it, and we just showed up one day and said, "Hey, this is great. Mind if we hang out?" Obviously, they were happy to see us there, and they still control it. I don't have the ability to kick users. They do.

Crystin Cox (16:46):

I honestly think that's better. When we were running MapleStory, we did have our own forums, but our fan-ran forums were honestly probably a bigger part of our community and a bigger part of our community experience. I think that's healthy actually to have a space for your fans to self-organize, even if you're there a lot.

David Edery (17:05):

I agree. You can certainly imagine worst case scenarios where this could blow up in our face.

Crystin Cox (17:09):

Sure.

David Edery (17:10):

But it seems to be working really well now. I like the fact that at the end of the day, I can come in and encourage them, but it's their space, and they're going to do what they need to, to make ... Hopefully their ownership of it will make them care more about growing it, if that makes sense. You know what I mean?

Crystin Cox (17:28):

Yeah.

David Edery (17:28):

We'll see.

James Gwertzman (17:29):

Which actually brings me to another topic. I'm very interested in your views on how you think the nature of marketing is changing. Because I think when I first got into gaming, almost 20 years ago, marketing had one purpose and one purpose only, which is drive you to go to a store and buy the game. It was very advertising focused. A lot of money is from the big ad budgets. I think marketing today is completely inverted from that. I'm curious to hear your thoughts on where you think marketing is and where it's gone.

David Edery (17:53):

It's changed so much. We really struggled with it, honestly. Everyone else does. Again, I won't claim to have all the magic answers, but you're absolutely right. We went from, "Spend lots of money on traditional advertising to get people to go to a store," to, "Let's do guerrilla marketing to try to get streamers to stream the game for free, and if they won't do that, maybe we'll pay them." But even then, that's tricky, because that doesn't always go the way you expect and on and on and on. If you're making mobile free-to-play games, like we are, it's like, "It's all about paid user acquisition. You have to use Facebook and Twitter and the ironSource and the million other ad networks out there."

David Edery (18:28):

You just literally say, "I'm willing to pay two bucks for a user to download my game and hopefully that's enough to get them in volume," and you can actually make that back, which is easier said than done, obviously. Your traditional press like your GameSpots and Giant Bombs and whatever, all that seems to drive a tiny fraction of what it used to. Your PR activities ... I know whole studios that have said, "Forget it. I'm not bothering with that kind of PR working anymore." Obviously, many still do, but it's interesting to see the extent to which that's shrunk away.

David Edery (19:00):

Then obviously, trying to come up with viral techniques. In Alphabear, at the end of every game, it takes the words you've randomly spelled and it plugs them into Mad Libs. Sometimes those Mad Libs are hilarious and whatever, and then we encourage you to share them. We do that very intentionally, obviously.

Crystin Cox (19:12):

I'm curious, how has that gone for you? How well have the Mad Libs done?

David Edery (19:18):

We have no way to quantify it-

Crystin Cox (19:19):

Interesting.

David Edery (19:19):

... because there's just no way to know that someone installed the game because they saw an image in Instagram or whatever, right?

Crystin Cox (19:28):

Right. You don't have any actual track-back to that.

David Edery (19:33):

Yeah, but our organics were better than people thought they should be at the time we were working on the original Alphabear. I think that's why. It's pretty clear that that was the main driver of it, because there was a time when the first Alphabear launched where Twitter was just all bear pictures all the time, at least for game developers.

Crystin Cox (19:48):

I miss those days.

David Edery (19:51):

It was-

Crystin Cox (19:51):

Twitter was better for a while.

David Edery (19:53):

Yeah. Twitter was a little better back then.

James Gwertzman (19:56):

[inaudible 00:19:56]. There's other externalities that affect [crosstalk 00:19:58].

David Edery (19:59):

Absolutely.

James Gwertzman (20:00):

Purple Planet.

David Edery (20:01):

I guess to abstract just a little bit, because it might be more useful, I feel like there's two major categories of marketing now, at least we keep being told. There's getting streamers and other influencers to pay attention to your game and there's array of techniques. Some of them work well sometimes, some of them almost never work, whatever. Then, like I said, there's just the hardcore, super crunchy paid user acquisition. We have a team of guys who spend all day long doing nothing, but trying to figure out how to game the system and get users for 10 cents less than the other company.

David Edery (20:34):

Both of those methodologies have really taken hold of streamers more in the hardcore AAA space, paid you A more in the free-to-play mobile space, but you're going to see both. Both of those things require pretty different skill sets from what marketing and games required 15 years.

Crystin Cox (20:54):

Have you had experience with streamers in your games?

David Edery (20:58):

Steambirds is the first time that we're making a real effort, and we're just starting now. I don't really have anything intelligent to say about it, other than we're quite anxious going in, because like you asked ... I remember someone sharing these stats recently. I'm going to end up garbling the numbers, because I don't remember exactly what they were, but it used to be that a few years ago, the top a hundred streamers played several hundred games over the course of a year and now they play like 40 or something like that.

David Edery (21:24):

They're doing that, because they've found that if they stream Fortnite, they get more followers, and if they don't stream Fortnite, they lose followers. You know what I mean?

Crystin Cox (21:35):

Yeah.

David Edery (21:35):

They're just reacting to the marketplace. They're doing what we're doing. Streamers are trying to get more customers.

Crystin Cox (21:38):

They have a business to run as well too. I think that's transitioned quite a bit over the last couple of years.

David Edery (21:43):

Yeah. I honestly don't know. It's possible that we'll launch Steambirds, and because it's not the next Fortnite, it's very hard to get streamers to pay attention. So, we have to pay them a fortune, and then whether that's even worth it, who knows? I really don't know. I don't know what's going to happen there. We're going to be doing some experiments. We'll see what occurs. Hopefully, we've done something interesting and unique enough that there'll be some number of at least mid-tier streamers who are like, "This is worth my time."

James Gwertzman (22:08):

Let's change gears for a second. I want to talk a little bit tools for a second. You've been getting more and more savvy about Live Ops, and yet you're still a small company. What would you consider at the heart of your tools chain around Live Ops? If there is such a thing, what's your daily scan? What do you look at every day to make sure things are healthy? What are the key metrics you pay attention to?

David Edery (22:31):

In general, the metrics we're paying attention to are the typical ones that I suspect you'll hear over and over again. We're looking at our retention on multiple different intervals, one day, two days, et cetera. We're looking at how much money the average user is spending per day, how many ads they're watching per day. That's a big one in mobile in particular. We're looking at, if we have events, how they're engaging with those events, like how many people are participating in them, how many people are spending money on them, et cetera.

David Edery (22:58):

We're looking at how many people are sharing things from the Mad Lib, because I don't know how many people react to the share, but I know how many people share it to begin with. Stuff like that. We're looking at all that, and anytime we see unexpected dips, we'll say like, "Okay, why is there an unexpected dip?" We'll try to react to it. The interesting thing is that, even though, to someone who's not done free-to-play yet, that might make me sound tremendously sophisticated, we are novices at this compared to some of the big guys. Some of the big guys are ...

David Edery (23:26):

I remember I was actually talking to a friend of mine just a few hours ago. He was telling you about how they have a whole team that just sits by and is watching these numbers roll by hour-by-hour. Literally they're like, "That's weird. There's been a dip for a few hours. It's time to inject some new content into the game." They're reacting at that level. We're so far from that. We react week to week or month to. You know what I mean? Not hour to hour. It just depends on what you're working on, but those are the stats we pay attention to.

David Edery (23:52):

I have an opinion, which I think most people would agree with at least on some level of, which is that if you focus on retention, the rest will work out eventually. Just make sure people are enjoying your game and sticking with it. If you do a good enough job at that, eventually you'll figure out how to make money, assuming, like I said, that you designed the game correctly from the beginning and you have the right bones and you didn't make Triple Town with no hooks.

James Gwertzman (24:15):

Not that you're still hung up on that.

David Edery (24:17):

Not that I'm still hung up on that. It's true. One of the interesting things about working on Alphabear 2 has been that with every single ... We did everything we could prior to launch 2, to work on retention. We still do things to try to improve retention, but they barely moved the needle. We've already picked all the low hanging fruit, and at best, we improve retention half a percent here or there. You know what I mean? Whereas literally every time we do a major release, our revenue per user goes up a significant amount. Because we did a good job in retention, now it's the "Easy stuff."

David Edery (24:53):

You hear that from people over and over again. Retention is hard, monetization is easy, as long as you've done your job correctly. We just do everything we can to have as higher retention as possible in the early days. We don't always nail it. Alphabear is actually a pretty good example. It turns out that no matter how good a game it is, there's just only so many people who want to play word games. That's just what it is. Word games have a niche audience. We got our day one retention to whatever it is. I think it's like 41% in English speaking countries. It's not going to go any higher than that. That's just clearly ...

David Edery (25:25):

We've made lots of efforts. That's a hard wall, as far as I can tell, but like I said, with modernization, we'll spend not even that much time. Like a week of engineering and art effort and see a really big book every single time. That's what Live Ops does for you. You can do that, because you built the game with a limited amount of effort. You can see those bumps.

Crystin Cox (25:49):

Sorry. That's a good transition. I know that Alphabear 2 is the first time you guys really have done live events, and I'd love for you to talk a little bit about why you were so interested in adding live events to your games.

David Edery (26:03):

For us, to be honest, it was like we didn't even have a choice. We just kept getting told over and over again, "You'll fail if you don't do this." I can see why. When we launched the game, we launched with a more ... It was the most polished mobile game we'd ever launched. We launched it back in September, I think, of last year or something. By far and away, the most polished mobile game possibly at all that we've ever launched. It had all the typical things like an attractive new user purchase and all the stuff that we typically have. You make money lots of ways for people to watch ads in ways that I thought were quite clever.

David Edery (26:43):

Your bear is the avatar, and then there's this Doc Bear with crazy white hair, which could either be referenced to Back to the Future or Rick and Morty, depending on what generation you-

Crystin Cox (26:51):

How old you are.

David Edery (26:51):

Yeah, exactly, what generation you belong to. There's a thing in Rick and Morty, if you've seen it, where they watch interdimensional TV. There's just wacky, wacky stuff. They're clearly ad libbing the whole time they're making these episodes. They're great. We were like, "Let's riff on that, because people appreciate that." We have a interdimensional TV that you and your Doc Bear can watch together. You have Doc Bear in your avatar watching interdimensional TV and the interdimensional TV is basically just ads. It's 30-second video ads, but of course, it makes sense.

David Edery (27:23):

The fact that you're watching interdimensional TV explains why you're seeing all these different things and totally different art styles about totally different game. You know what I mean?

Crystin Cox (27:29):

Yeah.

David Edery (27:30):

Then the real kicker is that at the end of every video ad, it cuts back to them watching TV, and then one of them will make a comment, and then the other one will respond. They're almost always dumping on the ad. They're just making fun of it. We don't know what the ad played. It doesn't matter. We just insult it, and it works out great. We had done all of that, and we figured, "Okay, this is good. We have multiple ways you can watch ads, including one very clever way," which I just described and news or banner, whatever.

David Edery (27:53):

Actually, it turned out revenue was quite poor. We were pretty disappointed by that. Then, like I said, we had the hooks in place. We started adding special events like, "There's an event that happens every month, and if you've collected the right bears, you can play in it and use it to earn this super bacon bear. He looks like a piece of bacon, and he's really powerful. Yay! If you don't earn him this month, don't worry. You can keep working on getting the prerequisites and maybe you earn them next month." You throw that in and bang, huge increase in accrue, and every single time we did that.

David Edery (28:25):

We had a new event, bang, increase in our accrue. It was like, "Gosh, we tried so hard to be clever about how we show you ads and so clever about a bunch of things, and none of it mattered." But doing these little events where you can win a cute bear at the end, it works every time. There you go. That's why we're excited about it.

James Gwertzman (28:42):

Wow. What advice would you give other developers about the importance of live events? Or what will you be doing differently in future games based on that experience? More events?

David Edery (28:54):

Yeah. I think we'll just try to be really thoughtful about giving ourselves enough elbow room with the design of the game. Whether it's, like I said, hundreds of bears you can collect with unique powers or anything else. Like in Steambirds, for example, we have a crafting system. Then obviously, crafting systems have been around in games forever, and they've always been important parts of their progression and so on and so forth.

David Edery (29:19):

In Steambirds, we've been very thoughtful about having lots of different things you can collect for the crafting system, lots of different things you can make with that crafting system, but there's no ceiling on that. We could add hundreds of more recipes to crack in the future, and it won't be that hard to do. Then of course, all the events tend to pay out these crafting inputs as part of their rewards. We've been very careful to make sure that all that's easy to do. I want to add a new crafting recipe, that's a quick thing to ... I just type in a few things in the backend. I want to add a new input, that's a few things in the backend.

David Edery (29:56):

I want to schedule a new event, that's a few things in the backend. None of this should take more than minutes to actually set up and go. In theory, I might want to do something special that takes me a lot more engineering and art, but that's optional. That's not required. That's the lesson we've taken away is it needs to be at that level. It's not quite at that level in Alphabear, but it's close. In Steambirds, is at that level, and hopefully we'll just get better and better.

James Gwertzman (30:20):

You've really optimized the operations tool side to help make it possible to more easily make those changes over time, which makes a lot of sense. Let's talk about data for a second, because I know you and I chatted at lunch, I want to say, more than a year ago. I can't remember what had happened that day, but you were really proud of something you had just figured out from the analytics side. I think you got a really good analyst guy, and we were just talking about the challenges of really building up an analytics system, and the kind of talent it takes to do that well.

James Gwertzman (30:50):

A lot of it had to do with asking our questions as well as ... Not just, "Can I write SQL?" But "Do I actually think of interesting questions to ask when I write that SQL?"

David Edery (30:59):

Analytics is super hard, it turns out. One of the hardest things you'll ever try to do. Everyone underestimates how difficult this is. We have seen over and over and over again that ... By the way, stop me if at any point you want me to talk more about the like, "Stop talking about how hard it is and start talking about what you do well." Sure.

James Gwertzman (31:18):

Do both.

David Edery (31:19):

We can do both, but in general, first of all, it's super easy to have a bug in your data and not realize it. So, you're making decisions based on bad data, and you don't even know. You've mis-instrumented something. You're dropping an event somewhere. You think you've got half the retention you do, or twice the retention you do or whatever. You know what I mean? It is shocking how easy it is to make that mistake. It's not like we're incompetent. We have good QA people. We have good analytics people. These mistakes keep happening all the time.

David Edery (31:47):

We actually had an issue recently, ended up costing us almost two weeks of dev effort, which was a huge bummer, where we were convinced that skilled players of Alphabear were retaining worse than unskilled players, which made no sense. Skilled players should absolutely have better retention. To be clear, I'm talking about the most skilled player. The top 25% were retaining not as well as the next. The only logical explanation for this that we could come up with was the game is too easy for them, and they're getting bored, and they're leaving in the early days, because we had had that.

David Edery (32:20):

We had actually had users tweet at us and leave reviews saying, "This game is too easy." I know for a fact that the game is not too easy. The new user experience is always easy by definition. You don't want to burn people out right away. We said, "Okay, it's gotta be that. Let's do an AB test. We'll make the new game harder for people who we think are skilled, have a whole dynamic scoring system to make sure that they can handle it." All that crap. Then right as we were getting ready to roll that out, our analytics guy goes, "Wait, I think I made a mistake." He looks at it and it turns out, they have the best retention.

David Edery (32:53):

The skilled players have the best retention in the whole group, and we wasted two weeks. That makes us sound like jerks, but we're not. That's just a thing that happens. You know what I mean? He did his best. It just was this thing.

Crystin Cox (33:09):

Especially when you don't have a lot of qualitative feedback coming in, because it is so difficult to get actual, real time reactions from players. You don't have thousands of players telling you all day every day how they feel about it. You just have the numbers.

David Edery (33:26):

Yeah. Here's the thing is that in this case, actually the few people who were talking to us were feeding into the myth. There were review saying the game's too easy. "Okay, cool."

Crystin Cox (33:35):

This is all ... Yeah.

David Edery (33:36):

"There, it's all lining up," and "No, it's not." That's an example. A, QA and analytics data is dramatically harder than you think. Making sure that you're both sending it correctly and then interpreting it correctly. You can make mistakes on both sides of that fence. The second problem is that a lot of the tools out there ... I know PlayFab is probably one of the better ones. A lot of the tools out there just don't give you the flexibility you need to make certain query. If you have a deep question that you're trying to answer, in many systems, you just won't be able to write the query that will let you answer that question.

David Edery (34:11):

We did this years ago, before PlayFab even existed. We created our own analytics backend, which there aren't many indie studios that are capable of doing that. But we were, so we did, to try to solve this problem. We never wanted there be a question that we just literally could not answer because of the technology. That seemed unnecessary, and it turns out that it was. Then I guess the other thing that makes this hard, James, you were alluding to this earlier, is that it's really easy to ask the wrong question and spend weeks chasing that wrong question. Like you'll say, "Hey, or retention is bad. Tell me why that is," to your analytics person.

David Edery (34:54):

That's literally like saying, "Does God exist?" You know what I mean? It's not a good idea to ask that question. The range of answers is enormous. We would do that. We would do that, and then we would get pages and pages and pages of response from our analytics person who's desperately trying to answer the question in a meaningful way. It's just not a fair question to ask, as opposed to like, "Hey, let's look at the funnel." "That's interesting. We're losing a ton of people on Mission 5. Why is that? Can you please tell me why we're losing a ton of people in Mission 5?" That's a question you're likely to get a good answer to. You know what I mean?

David Edery (35:34):

Or, "Hey, not as many people playing in the tournament as I thought. I've noticed these following things. Can you tell me if any of these things seem like they might actually be a factor?" You try to be as precise as you can with your question. It still doesn't guarantee that you're going to get a good answer. We many times don't, and that's actually one of the hard lessons you have to learn about analytics. It's sometimes after spending three weeks studying something, your answer will be a shrug of the shoulders. That's life. You know what I mean? Sometimes it ends up being super valuable.

James Gwertzman (36:05):

Do you have any examples or stories you can share about situations where you actually made a substantial change to the game based on analytics data that was actually not an error, but real data and it improved things?

David Edery (36:18):

Yeah. I think in the original Alphabear ... I don't, unfortunately, remember exactly the steps that we took, because this was long enough ago. Anyway, in the original Alphabear, there weren't enough people spending money on our equivalent of the loot box there, our treasure events. The reviews and the tweets and things were not helping us. There was no feedback coming in from players that really explained it, and so we were at a loss. Daniel, my business partner, our lead designer is excellent. His initial instinct was, "Well, we're just charging too much for it. We have to charge way less."

David Edery (36:57):

There was much a disagreement on the team on that. We didn't want to just arbitrarily cut the price. We went back and forth and back and forth. Then, our analytics person at the time basically said, "Listen, have you considered the fact that maybe people are just too impatient?" He had a couple of different stats to show that, "They're tapping really fast. They just want to get going," kind of thing. "They don't want to play ..." Sorry, I didn't explain something critical. Alphabear did something that's unusual. It made it so that the loop box, even if you're spending money, you had to play a game to get your reward. You couldn't just get your reward.

David Edery (37:32):

He was like, "People are jamming the button. They just want another reward quickly. You're making them play a whole game to get it. Maybe that's the problem." So, we said, "Okay. Well, let's try adding a thing that says, "Get ..." I don't remember what the button said. It was something like, "Get another reward," but basically after you played the game once and got your reward, you could immediately spend money and immediately get another reward without having to play again. Adding that button tripled revenue. I'm not even kidding. It was like 3X jump or something.

David Edery (38:01):

It's one of those things that he said, "They're jamming the button. Maybe that means something," and it did.

Crystin Cox (38:08):

Yeah. That's cool. I think we're just about out of time. We're out of time. Do you have any other pearls of wisdom you want to share with devs trying to learn about Live Ops.

David Edery (38:23):

I think it's really ... This is particularly for people who are new to it. It's really easy to get overwhelmed. I just came out of a lecture in the mobile game summit given by a guy, Kongregate, who's great. His name is Peter. He's a real expert in this space. If you listen to Peter talk about what they do to make their game successful, and you're a small indie who's never done this before, you're like, "Fuck it, I quit." You're just going to wave the white flag, because what they do is so sophisticated and so labor intensive, and they have so much experience. You have to tune that out.

David Edery (38:55):

Sorry. Don't tune out entirely. Learn from it. You're not going to do what they do, and that's okay. Just start by ... Like I said, build a game that has the hooks such that it can scale well, if it is successful or if it even shows the promise of being successful. Get good at that, and then get good at adding content on a weekly basis or whatever. Be good at that. Don't worry about the, "We're watching on the daily basis." You know what I mean? Don't get freaked out by comments like that. When someone talks about how they have a UA team of 10 guys who spend all day long, trying to figure out how to game-UA systems. You're not going to do that, obviously.

David Edery (39:31):

That's okay. Don't let that freak you out. Focus on having halftime UA person who's understanding this stuff. You know what I mean? You will unfortunately need at least that person pretty good if you're in level. You know what I mean? Be willing to start small. That's okay, as long as you're starting. As long as you're not going in completely blind and saying, "I'm just going to release my game and it's going to magically do well," which obviously, unfortunately doesn't work very much anymore.

Crystin Cox (39:56):

I think that's fantastic advice. Thank you very much.

James Gwertzman (40:00):

All right. Thank you.

Crystin Cox (40:06):

Thanks for listening to the Art of Live Ops podcast.

James Gwertzman (40:09):

If you liked what you heard, remember to rate, review and subscribe, so others can find us.

Crystin Cox (40:14):

Visit playfab.com for more information on solutions for all your Live Ops needs.

James Gwertzman (40:19):

Thanks for tuning in.