No Ops?! Your Software Doesn't Exist - A DevOpsDays TLV Sneak Peek PT 2 | Andrew Clay Shafer and John Willis

In this special English edition of Dev Interrupted we take another peek into the upcoming DevOpsDays Tel Aviv conference, we chat with Andrew Clay Shafer and John Willis two event keynote speakers, who help the global DevOps(Days) community and ecosystem that has impacted us all.  They will take us on a journey to the first days of doing DevOps, the tools, personas, and what impacted some of the biggest changes that impacted how we deliver software.  

Transcript

Yishai Beeri: Hey everyone, thanks for joining us. Today is a special English episode inside the Dev Interrupted Hebrew podcast. And I'm very excited to host John Willis and Andrew Clay Shafer. Great having you. 

John Willis: Hola. 

Andrew Clay Shafer: Thanks for having us. Shalom. 

Yishai Beeri: Shalom. So this episode is a kind of a sneak peek into your keynote sessions coming up in DevOps Days Tel Aviv later in October.

And because this is a different format it lets us maybe dive into a little more details and maybe also take some detours from the main subjects that you're going to talk about. But as always in my podcast, I'd like to ask our guests to give a quick intro into like my journey so far, how I got to where I am right now.

You guys obviously have a lot of journey, but give us a summary. Andrew, why don't you go first?

Andrew Clay Shafer: There's so much journey. I'm getting old, but at least for the DevOps relevant things, I started as a software developer, stumbled into a bunch of startups, just trying to make my way, take care of the, be the kids, the rest of that stuff, and then I got enamored with this idea of I was working for a venture funded startup, so I wanted to raise money and be involved in some of this other stuff.

Then I ended up getting really focused on this domain because. I was part of Puppet and working on Puppet since about 2004. And I spent about five years there till 2009 focused on Puppet. And so there's a lot of conversations that I had and work that I did in that phase that kind of became, we'll call them like the primordial conversations that turned into the DevOps conversation.

And then from there I was a VP of engineering for an open stock startup that got sold to EMC. I spent five years at Pivotal doing. Cloud Foundry and Spring and the rest of that kind of stuff. And then I spent three years at Red Hat with John doing the Kubernetes OpenShift and Ansible stuff there. So have a, whatever kind of background perspective there.

And then I would wager. And I don't have good metrics here, but I think John Willis has probably been to more DevOps days than anyone. But I wager that I've probably been on more slides at DevOps days than anyone. So there's my, my tweets are, my, my quotes have probably been on more DevOps days than anyone else.

Yishai Beeri: Careful what you measure. You, if you measure if your metric is number of slides, you're going to get slides.

Andrew Clay Shafer: No, my quotes, my, my face, my quotes, my tweets on people's slides.

Yishai Beeri: Ah, got it. Citations.

John Willis: Citations. Or non citations. People who just stole Andrew's slides.

Andrew Clay Shafer: Or that's another thing people do. They use my words, but they don't give me credit. Yeah.

John Willis: And I, like it's now again, a bunch of stuff. Like I consider myself, Gene Kim calls me the sort of DevOps historian, right? So I just find myself saying. In fact, I did it today. I was listening to a podcast with Sasha and they were talking about the Code of Conduct.

I, so I love Andrew. Andrew's one of my greatest friends. And I, whenever I see somebody talking about something they're not actually giving attribution or citation to Andrew. I like, I have to shout out. So they were talking about code of conference code of conduct. And I'm pretty certain that Andrew, you were the first one to introduce that in the Pittsburgh DevOps days.

You probably don't even remember this. There were no code of conducts until you, you introduced it at DevOps days. So I pointed that out in a comment in Sasha's thing. So

Andrew Clay Shafer: anyway I wrote the code of conduct that most of the DevOps days. That's right. What was on DevOps days org as a Yep.

There, there's all this, I think this is also in interesting, irrelevant to some of the other stuff we're gonna talk about, but Yeah. Yeah. Like we just build on each other's work, right? I didn't make up code of conducts. I just noticed the utility of having that dynamic at an event.

And I was trying to put on a nice event, so there's... A necessity to have a code of conduct. And I, so I made one.

John Willis: This

Yishai Beeri: also goes back to DevOps is all about code, all about processes, machines, but underneath everything is humans, right?

DevOps is not going to be successful without humans and code of conduct is where you're looking at the human behavior,

Andrew Clay Shafer: the protocols it's the protocols.

John Willis: And it was the right protocols to add. So just my background really quickly. I, I've been in this industry for I round down.

I think there's, when I get, when you get to a certain age, you literally, instead of rounding up the amount of years you have experience, you start rounding down. You start forgetting what you did yesterday. You forget, but then also maybe I don't want them to know I have 45 years experience, I only have 40 years experience in this industry.

So I definitely don't want to bore anybody with the beginning of my experience, but, I switched over from sort of mainframe stuff to distributed computing did a bunch of work around that with IBM products, but then realized that there was this thing going on, it was open source.

It was, pre cloud, but open source and Andrew, I created a blog, Andrew reached out to me and we became virtually friendly. I remember as it pertains to DevOps, I think there was a, like a lot of hovering around what this is, what it could be.

And I don't know which came first, but, I listened to Andrew Michael Cote and Israel Gatt on called the Agile executive. I think it was, and Andrew started talking about Agile operations and then I knew Andrew. So I think we might've met already. Before that, but I literally I think I remember driving in a car and going, Oh my God I'm not, I'm an ops person.

I'm not really a dev person, but I've I want to know more about this agile thing that's going on. So I, and then he said you really ought to reach out to this guy named Patrick. And there was a bunch of stuff going on in Europe. And at the time I was at Canonical, I was working for Simon Wardley.

And I called Patrick and I'm like, what is this thing you're doing? And he was DevOps day, right? DevOps days. And he said to me, he said if you get canonical to do a sponsor, or pay your travel to come to this thing in Gantt, we'll we'll go ahead and, give them a sponsorship.

So it was a really awesome thing for him to do. And I just called Simon and said, we. Gotta be at this thing. This is this is the beginning of something. And that's before I even went there. And then I went there and I always say I was the only American, but Andrew corrects me. There was a guy who's not in our community anymore.

So instead of getting yelled at by Andrew I'll say there was only two of us, but the other guy hasn't been around for quite a long time. and I came back and I, I knew Andrew and Damon Edwards. And there was a couple other people, but I was like, Guys, like we got to do something here.

This is just what I saw the, just the enthusiasm, the people. And, if you think about the first DevOps days, like Lindsay Homewood went to Australia and created Crispy basically you created the whole thing in Europe and Stephen Ellis Smith and I already appear a few other created the Europe thing right now.

It's at the same time, like Andrew and Damon, we got to do something here. And we ran the first couple of DevOps days in Silicon Valley. And then, I just. I, I got involved with Chef early on, it was really early in at Chef. I was I got into Cloud and, sold the company to Dell and just me and Andrew have been great partners in collaboration and learning, I've learned so much from Andrew, it's just incredible over the years and the one thing Andrew does, like I know I'm hogging a lot of time, but Andrew is like an incredible, like person to have a dialogue because he makes you work.

For your answers, which is awesome. Like you literally, you can't just say something. Andrew will okay, John and then, or even sometimes I think you game me where you'll take the opposite position, but I haven't quite figured

Andrew Clay Shafer: that out. That, that, that's the that's the result of spending my college years on a debate scholarship that I want to like,

John Willis: But you learn, he learns, I learn, John, I

Andrew Clay Shafer: learned an immense amount from John. And I think one of the things John does a great job of, and he's still doing that now with some of the stuff he's getting excited about is he's like a forward observer. So he's like sensing.

Like I feel like I ended up in a lot of like mud and blood and John is like out in the he's like where all the fairy dust and unicorns are sometimes, and then he gets like, gets me excited about the next thing to to like, so he dragged me into a bunch of stuff about automated governance and then he's getting excited about all the AI stuff.

So there's definitely a symbiotic learning relationship.

John Willis: But in other words, I'm the dork and the squirrels, right? Squirrel. So

Yishai Beeri: you're on the top of the mast on the mast and like looking at what's coming, but you also said you see yourself as a DevOps historian.

John Willis: Yeah I think, cause I, like I was there at the first event and to, and like I said, I can confirm that, that I, I probably have used more of Andrew's.

Images in my slide decks than anybody like if you were to bar chart it out, it'd be like some like really high than the rest at some low level because some of it's like the classic, the thing, it doesn't really drive me nuts, but like people just post and not that I think there's some long tail on how long you should give attributions and, some of my stuff I see now I'm like, all right, it's been 10 years but like the picture, the canonical picture image is the sort of wall of confusion.

The Yep. And, you have the dev and the ops and the way it's the way we were able to so describe this problem of, throwing DevOps, developers thrown it over the wall and, ops catching it. And that was, the sort of crime of DevOps is. There were like three great presentations at the, at Velocity 2009, I believe it was.

There was the famous John Osbar and Greg Hammond, or Paul Hammond and, the flicker which is landmark. And there was Andrews, and there was actually Adam Jacobs, which was pretty interesting. Andrews was not videotaped. And that's where he described agile infrastructure, and that's where he pointed out crashing down that wall.

And that image has been used. That and the, the blind man, blind people and the elephant have probably been the two most used images in all of DevOps presentations. And most of those were, he didn't come up with the blind man and the elephant, but he was the first one to propose it as a sort of, metaphor or something for for DevOps.

So since you brought up velocity, I think velocity Played a special role in the evolution of this conversation for me personally. I'm certainly for a lot of other people. And I believe you as well. But that's that, that John all spa Paul Hammond presentation is really where the word DevOps.

Andrew Clay Shafer: originated from. again, there's no root cause. We don't want the root cause police to come in. But the beginning of this is me watching Paul and, John. And tweeting, live tweeting back when Twitter existed. Remember that, so their topic was about Devon ops, cooperation of flickr.

And I just started tagging everything. I was. Posting as with this tag DevOps from that conversation. And then that's where Patrick was like, we're going to call it DevOps days. And I'd already been working on that program with Patrick and the format for DevOps days with the open space was my, deep down inside.

And I still, to this day, believe this wanted to have an open space conference. Where you could have these discussions. I know it gets hard at the scale. Some of the stuff happens in Tel Aviv, but I always believe that you could get more. And this is one of the special things about velocity for me as well as you could get more from a round table with your peers.

Then you could from having someone on stage broadcasting, because for most topics, there's probably a few exceptions. The audience in total has more expertise than the person on stage for almost every topic. And so getting this kind of dynamic where you can have conversations with your peers is very powerful, but I knew that managers would not send people to conferences.

To talk to each other. So you have to put logos with people that work at fancy places and topics and have presentations to get to that open space that I really wanted to have.

John Willis: Yeah, I know. That's such a critical part of the whole DevOps days. I didn't even know that you were planning that with Patrick before that was I was in parallel trying to figure out how to get Simon Wardley to send me to Belgium, at the same time, talking to Patrick the but that's another sort of, you asked me about the historian I just recently with last year, I corrected some major people who have basically did the first use attribution.

I'm like, none of this really matters, but it matters to me. The first use attribution of DevOps to somebody else. And it was Andrew and I had to, I'll talk about my book in a little bit when you asked me about my presentation, but my book coming out is. I made sure I got that attribution right in my book, that that it was, and I literally didn't take Andrew's word for it, just like Andrew would not take anything my word for it.

He would research it and I went back and I did an extensive search and I'm highly confident that his first use, which is so this

Yishai Beeri: term not just being a brilliant keyword, but the response this got from the community the energy that, created a community To me, at least it says this hit a pain.

This hit that these ideas hit a real pain that many people were struggling with and getting their heads around it around and still are and I'm, I'd to talk about the evolution of that pain and the solutions all. Like the DevOps at large is a umbrella of approaches and terms is approaching.

So Andrew what did you feel was the, we don't need to, we don't need to do it anymore. We have a,

John Willis: I need to get that in before, before Andrew went on his tirade. No, it's all over. Don't worry about it. There's a button. You don't have to worry about what's next. Yeah.

Yishai Beeri: Andrew, why don't you go? So yeah.

What was the, if you like the. The nucleus of that initial pain that got this, like the initial energy to get started.

Andrew Clay Shafer: So there's this funny thing that happened. I think it was the second DevOps days in Silicon Valley. It might've been the third one. It was definitely a LinkedIn, but there was a panel and someone on that panel, and I think this is what John is trolling me about, like we were having this conversation and I was in the audience cause I don't need to talk all the time.

And someone said, Oh, like this is a solved problem. And I like, just basically tore the mic out of

John Willis: the You tackled the guy with the mic, I remember that.

Andrew Clay Shafer: I didn't you cannot say this is a solved problem. There's no aspect of this that's actually solved.

We're doing the best we can to hold this together. So I want to set it up with my own journey a little bit, and then we can come back to the larger vision, right? So I've been working. In these venture funded startups before the puppet story where we were trying to deliver this SAS.

There's this e commerce SAS thing. And I was in this phase where, and it wasn't a great, a huge scale, but we were running something like 30 million in transactions a month through it. And doing whatever some percentage on this e commerce platform thing for our customers. And we were, we had hopes and dreams of being like what basically what Spotify became.

And the night was like a deployment was like traumatic, right? So you go and I would get the biggest rock stars that you could get. So like these 24 ounce cans of caffeine and I would put it on my desk and then I'd have to make a decision because we'd start deploying around 10 PM.

Which is silly now when you think about the global scale and like the way the like, but we were disrupted right with these deployments in that time. So this is like 2005 ish right so it's like you go and you make this decision, are you going to open this can. of caffeine around midnight because you know it's going to be long you're there.

So you put these things on the servers, you got your tail minus F window, and you see the stack traces and you're like, okay, this is going to be a long night, right? So there's this whole personal pain that I experienced and it would be. All these little differences between the development environment on my desktop and this testing environment and the production environment and really a lot of what became that kind of primordial conversation for DevOps for me was trying to apply what I considered agile agile engineering principles of being able to reproduce a really rebuild from source and have continuous integration.

On my infrastructure and using things like puppet to apply those checks against and have that reproducibility and this way to promote through those environments. So that was like my own personal journey. But then in the bigger picture, and this is where I think the conversation of velocity really exploded.

My own perspective on this is these principles Of Agile and like the Agile manifesto and all this stuff, they get spun up around 2001 and the Internet exists at that time, but the majority of software in the beginning of the Agile conversation is shipped on physical media. So you have some software, someone ships it, and then it's not their problem, right?

It's operations is outsourced to whoever has to run that system. And, with the advent of the internet, and the movement to as a service, Then operations becomes a critical part of the value stream, value chain for delivering services. And if your servers aren't running, your service doesn't exist, your software doesn't exist.

And so I think that elevated the operations that used to be about keeping the mail server and like the IT of the business process going to The business is the it now, the business is these servers and these systems. And then without that pressure, that Darwinian pressure of that as a service model, then you wouldn't have that dynamic.

And I think everyone started feeling that pain as they got into more as a service, which is still what I think most of these kinds of like digital transformation investments are about is can we run software as a service? Can we deliver, can we consume software as a service and that all these other things fall out of that pressure to deliver software as a service.

John Willis: And for me, I think, similar sort of vein of, yeah was that, I backed my way into it. So I wasn't an agile, I was operations pure and pure my whole career, right? Like from mainframes to distributed first generation distributed computing. And, one of the things I did before I met Andrew and got into the sort of almost drove off the side of the road when I heard him talk about agile infrastructure is I spent like many years with a product by IBM called Tivoli and one of the big products, there was some configuration management.

And I would call that sort of first generation configuration management, very macro based and all this. And I remember being at OSCON basically in 2007, I believe. And somebody, and I was going there, it's I just started I'm not going to the Tivoli stuff anymore. In fact, I sold a company, so I had non compete to do any services.

So I was going to figure out all the open source variants of what the proprietary IBM software did. And so Nagios was the thing that was like, Oh, monitoring. This is gonna be great. And I was at, I was some guy I met and he's Hey, you want to go to this thing called puppet? And I'm like. What does it do?

And he's like monitoring. I'm like, yeah, let's go. And I'm in the back of the room and like I, this is my sort of imaginative narrative, but there's this pimply faced kid in the front telling the world about how to do a configuration management. And I'm like, yeah, I'm moving to the front of row and I'm going to, I'm going to put this kid in his place, 10 minutes into the presentation.

I realized my life had changed. It was Puppet, it was Lucanese, and and then I spent the next year begging Lucanese for a job, and it's the only time in my life I've actually had to beg for a job. And and along the way, Adam Jacob of Chef was, so in other words, I fell in love with Infrastructure as Code, I'm like, okay, this is the way, this is the gateway drug to everything.

Monitoring is great and, like having open source free versions of versus 10 million versions, like that's a cool narrative, but the, being able to accelerate and Adam Jacob was one of the top power users of Puppet early on, and he had done this Facebook application called I like, and Luke could tell me, you need to talk to this guy.

And so I did an interview with Adam and then I love Luke to death, but I got tired of asking for a job. And so the first time I asked Adam for a job, he's bad word. Yeah. And and and I came in really early. I think it was somewhere between seven and nine as first person in chef and Lily got to take this ride.

And one other thing about that was, I had just left Canonical, but like DevOps was starting to explode. It was starting to happen all over the place and Jesse Robbins, who was the CEO of Chef said, literally gave me a credit card and said, just go DevOps. And that's why, when Andrew says I've probably been the more, still to this day, probably been the more DevOps days than any other person.

A lot of you look

Yishai Beeri: at the. At the, at least early days of Puppet and Chef they this started way before we had infrastructure as a service, and it would, this is about how to ship my codes to my the pizza boxes that I spent two months buying and installing, right?

And then I can move my code there. So change management for code through Puppet and Chef. But the underlying, like spinning up a server, spinning up a new box is still old school. And then there's a shift saying I can actually spin it with an API call and it's up in three seconds.

John Willis: The iLock I like example.

I think Adam was able to, with bare metal, this isn't even cloud, was like, they blew out like 5, 000 servers. In a couple of days using puppet, but the thing that has to be cleared, it never said as much as it should be. This whole genre of infrastructure as code started with Mark Burgess. Again, being a historian, I was about

Andrew Clay Shafer: to bring it.

I was about to

John Willis: bring up. Yeah, he said

Andrew Clay Shafer: it's actually a similar dynamic. It's a similar dynamic to Adam with chef because Luke. Was inspired by CF engine. You would never have puppet without CF

John Willis: engine. That's right. And you would never have chef without puppet. Cause in fact, a little side note I learned chef first.

And when I left chef I was invited, cause then I, once I left chef, then Luke liked me again. He liked me before I went to chef. He did like me when I was a chef and he liked me again when I wasn't. And they let me take some free classes, some puppet classes. And I'm like, Oh, Adam's not the smart guy here.

It was Luke. And then a few years later, I took a CF engine class. I'm like, of course, like all this stuff comes from the physicist turned computer scientist, like mathematical genius Mark Burgess.

Andrew Clay Shafer: So I do think there's some interesting nuances that won't really matter to anyone who doesn't live in.

So there's a couple of things with the abstractions. They're interesting and like difference between way CF engine and puppet and these things frame the world. And then if you actually understand how they work the control loop construct for these things is exactly the same as what with something like Kubernetes.

The difference is that the abstraction in that, that, that generation of tooling. The highest level primitive that you manipulate is the server. So you have this server centric thing that you have a control loop on where when you get to the new things, you have this abstraction that lets you have a declarative service.

And now we're getting into this world where in this multi cluster world, and whatever kind of we'll call them platform, whatever, we'll get into that maybe like now you want to be able to do things across clusters and have like service abstractions, connecting and gluing all these clusters.

So each layer basically has this problem where you have this centric. Primitive that's the highest level primitive. And then you have to glue it together with everything that's above it.

Andrew, you froze for a second. Maybe I haven't, I'm having a bad internet day. I don't know what you lost, but we'll pick it up from here. It was,

John Willis: so you were talking

Yishai Beeri: basically about like nested control loops, right? There's a server layer, then there's the service now. We're looking into abstractions of services across clusters, right?

But it's the same underlying idea.

Andrew Clay Shafer: The central idea is that you have this declarative state, and that you're checking the actual state, the ground truth, against the desired state, and then you have some logic that tries to move the desired from the ground, from the actual state to the desired state.

That's the control loop process, and that's what all of these That's what all

John Willis: these tools do. I feel like an idiot because I should have known that. And I just, as you listened to you say that, I'm I, I've always been able to explain CF engine to, to Puppet to Chef and even to Terraform and Ansible and Terraform, like there's this really beautiful evolution, but it just went over my head that you're right.

It's the same loop. It's the same loop transaction. It's the desired state. So when we talk about platforms. We won't use the word platform next word here in this podcast, but just . I'm not, I'm,

Andrew Clay Shafer: I'm not afraid. I'm

John Willis: not afraid. No, you're not afraid of that debate. Okay. But we'll waste too much

Yishai Beeri: time on it.

We'll get there. We'll get there.

John Willis: No, but I think that's a brilliant observation about platforms being that same evolution of. Of the abstraction and desired state. So that's really the, yeah, that

Yishai Beeri: was really good. And then get ops and similar approaches are basically around how do I mutate the desired state, right?

I already have a control loop to make sure that reality matches the state. Now, how do I change the state? The desired state.

Andrew Clay Shafer: This is the silly thing about these words, right? There was never a time in the puppet or chef community. Where a change to the infrastructure wasn't committing code to get and letting the agents do their magic.

Like that was the workflow, right? So there's not really anything other than what I said before, which is the abstraction got to. Now I wouldn't say it's a total standard, but it's like a de facto standard of kubernetes as a way to do this thing. Another thing that's interesting if you want to get into the weeds on the nuance is This notion of immutability, right?

Because the paradigm for the puppet and chef model Ansible, whatever is basically you have this resource level on these servers and you're going out and you're mutating the state across all those things. Where when you get to the the new kind of container models, you have this self contained deployable artifact, and people will talk about immutable infrastructure, but it's not really immutable, right?

And you still have to tweak this thing for this configuration to connect to this other thing, and there's, it's just a question of what's mutable and on what time scale, and then being able to reason into what promises you can keep about the system from that.

John Willis: And it was never immutable really, right?

Immutable meant from a chef and puppet, a CF engine was a checksum, like literally a checksum of a config file or a checksum of a binary or right. So the mathematicians who I, which I am not, will get all upset and twisted around about immutability and okay, calm down.

But but I get the point that we were using it in a way that allowed us to at least say. Every 30 minutes, I at least want to make an attempt to get it back to the way it should have been. And at least from a chef's perspective, it was all about checking. Primarily, almost every primitive was really a check of some form of a...

Andrew Clay Shafer: That's a control loop, and I think there's a bunch of little mathy things we could get into, but there's idempotence, which is getting things into that state and like a whole discussion about what that means. And then there's this other conversation that evolved somewhere in the middle about immutable infrastructure, which is essentially in like a reductionist version, just being able to have these self contained deployable artifacts.

That you are, that you're deploying like docker images instead of having the file system changes that the. The like puppet chef type stuff would do you just made

Yishai Beeri: the immutable thing a bit larger.

Andrew Clay Shafer: Yeah. You just make what you're mutating

John Willis: larger. Yeah. And feel free to cut us off. But one thing I will say is that was the original paper by Steve Turgotte, which was the difference between convergence.

And what, why is the web failing for the word congruent? And he had written that paper back before containers. I was like, even existing, even containers aren't truly congruent, but like what Andrew's talking about is this idea of moving from when you get into a container world, you literally can move from from convergence to some form of you could call congruent, right?

Anyway, probably we're going way too deep here, but

Andrew Clay Shafer: yeah, so these are deep rabbit

Yishai Beeri: holes. Oh, yeah. And I'll probably look for opportunities to, to spend some more time in these, but, let's take a jump forward. So we talked a little bit about the history and some patterns in early history of DevOps, if you like.

And, naming is hard. And if you nail a name in a way that resonates almost doesn't matter what's in it. I know you both have researched high performing teams, what it means to be a team that actually is Performing well, consistently delivering value at a, what people call high velocity or whatever you want to call it.

But and DevOps definitely has a strong part in what a high performing software or software as a service team means. So can you tell me a little about what you've learned from your research? Like, how do I even look at this problem?

John Willis: Yeah, I want to go first on this one because, we, this is the sort of the 999.

Million dollar question, right? Like it is the, like what we struggle and me, Andrew and Jay blue and Sasha and the old people, like we, we have, we keep going back to this progress thing what have we done? Great. And there's some great things. If we look at the 15 year journey like whether you hate Uber or love Uber, like that doesn't exist without all this sort of stuff that came about, and which was Andrew and Patrick and me, and just lots of people, shoulder of giants. The thing about The thing I've always struggled with, like all the things that we seem to learn better and we have those aha moments and so Andrew will do like incredible presentation and, and I've said some of the presentations Andrew has given in the Davos community are the best presentations ever given, bar none.

And. But then you walk into an environment or a company and you're like, why isn't this working? And I had this sort of, I won't, epiphany is too strong a word, but I did a Japan study trip. And actually Jean had asked me, I'm going to do a plenary talk on it at the Dallas Enterprise Summit.

Cause he loved it so much. But the thing I realized is, when you talk about the miracle in Japan and how we get lean, lean manufacturing, the lean software to possibly agile, but definitely DevOps, like a lot of tenants that came out of Toyota or, like they're traceable, highly traceable.

And I feel like in some ways, I don't want to be defeatist because I think we do a great job and we see, we've got to keep carrying the flag. But like I visited it was a few months ago. We went to like about five or six tier twos Toyota suppliers. And you started seeing just this consistency of community, collaboration, trust and it was just native and, and then on the on the trip, there was some pure sort of lean people that really aren't in IT that doesn't matter no one's better than the other, but they kept asking the question about KPIs.

And they didn't understand the question. They literally, they like, and finally at the end of the week, the translator is threw her hands up because she was tired of trying to figure out how to explain to something that they didn't. We call it kata. You ask a kata, they don't put a word to it.

It's part of their DNA. But here's the thing where I think we run into so much trouble and why we're trying to put a square peg in a round hole all the time. Is we are basically, and I'd like to hear your thoughts on this, Andrew. We are basically Tayloristic

So we, and what we try to do then is you come out of school, high school, you come out of college and you're you're trained a certain way and then you meet these agile and lean and people and DevOps people, and they're like, okay no, you can't do that. You can't do command control.

You can't do that. And we're trying to shave the corners of the square hole. And. And we visited an elementary school over there. In that elementary school, these kids already understood the concept of Kaizen. This was sex of seven year old kids. They served lunch to, they didn't have any lunch people.

They literally rotated lunch. You see these little kids with these lunch people's smocks on and all, and they served to teachers. They didn't have janitors. They literally clean, some of the kids rotate and they're cleaning the bathrooms. So they come right out of elementary school with this binding, in their DNA of community, collaboration, trust, sharing.

And I think, that's part of why we, I think we're always two steps forward, one step back, or, I think, it's not natural for us. To have that DNA that it's just so every supplier after supplier, it was almost like it was staged for us. And I know it wasn't, it was almost like they put on a Kabuki theater for us to Oh, these Americans are going to come to Japan.

Let's just say the same words. But it was. The plant matters over and over it was just so

Yishai Beeri: you're tracing this back to the like cultural very deep cultural differences, right? It feels like I don't see culture in Japan out of that. Yeah, it's different from what we see in the States or in Israel or, and we're trying

John Willis: to, I think we're trying to force fit.

It works and I'll shut up here and say, but I think it works. Andrew has been incredibly successful. I've been successful. There are many people that we can look to change the way people do work because of the banner called DevOps. But when you run into those things where you like Hey, doesn't everybody know that these are the tenants of how you should operate and you walk into a large company and.

And like you spent like months with him and then you come back a year later and nothing's happened. So

Andrew Clay Shafer: what's your take? And so there's this interview with Deming in the 80s where he's talking about how the American car companies have invested all this money trying to automate their factories like the Japanese and his commentary.

And John knows this as well or better than anyone is focused on that. The Americans think they can just copy. But they don't know what to copy and they're only copying the most superficial aspect of it. And I think that's what's happened with a lot of quote unquote DevOps implementations have struggled as people are copying this superficial aspect of this thing.

Where, they read some book, or they saw some blog post, or they saw someone's conference talk. But there's an aspect of a high performing team having an aligned collective focus on an outcome. That a lot of organizations never get and they're actually rewarding aspects of their funding model and their org chart are rewarding that individualistic kind of localized optimum.

So you have all of these KPIs where people are getting literal compensation, monetary bonuses for optimizing things that under analysis of the whole system are actually creating problems for the company. And there's famous quotes going back to Deming, Acoff, Drucker pick one, where if you give them, if you give a manager a target, That they tie to their bonus, they'll destroy the company to get their bonus.

Yeah that

John Willis: I love, it's Toyo Chihono, who is the father of TPS, Toyota Park System. They asked, some of his co workers asked him, Why do you let all these American companies come over? And see how we do what we do. And it's is, feedback or a sort of response to them was they can copy our techniques, but they can't copy our code.

They don't understand our culture. And that's that the sort of companies that we've seen, Etsy is a good example, right? Etsy had been. I don't know that they really are now and there's a whole reason for that, but called private equity, but but the I think, in the early days of Etsy, they were such a poster child for us.

First, they, they put John in and he became part of it. The founder was, the CEO was a developer. There was so many beautiful things about, but it was all about culture there. I got to visit, I sold them chef. And. And I remember going to visit the office and everybody had a stipend to, to buy stuff off Etsy for their desk.

So some people had a, a London telephone booth and like there was just they had things called Meetsy, right? Like in it was just like, I go on and on, but and everybody tried to look at like when people from Etsy posted or created open source projects, they were like, okay, we'll just run that open source project and we'll be Etsy.

And to Andrew's point, you gotta figure out, the guy who was the CEO at one point, Chad Dickinson he started out as a software developer, so now he's in a major commercial retail company it might have been one of the earliest sort of, Real retail companies that compete with everybody else or to a extent retail that actually had a CEO that started out as a sort of agile, lean software development, like that whole track, like Andrew's background as a CEO, right?

That's amazing. I don't know that you just go ahead at Target and Target is a bad example because they did a great job with the DevOps, but like you're in some retail company and okay, we'll just use all the open sources. Etsy has we'll go ahead and hire some of the Etsy people.

We see this all the time at banks. Everybody's trying to hire Capital One people. Like Capital One people, like you'll see it all over the place. If you've worked at Capital One over the last five years, you're gold for any other bank to hire you. But if it's all about culture, if it's The other thing that happens is

people try to hire like big shots from Google and put them in a retail bank. And there's these are tragic stories of like failure. So like

Yishai Beeri: transplant, transplanting a culture through hiring people. That's not

Andrew Clay Shafer: going to work. Yeah. I actually, I started to cringe a bit. When people talk about culture lately and I gave.

Talks about culture and there's a big thread about what I think culture should be about in this talk I gave a long time ago called there is no talent shortage, which is 10 years old this year where culture should be about cultivation but I think the danger of focusing too much on culture is you swing this pendulum so far and you lose the other side of it.

So to me, what you're trying to do. With whatever the buzzword is, right? DevOps SRE platform engineering is build a socio technical system. And the takeaway from socio technical systems theory is that you have to consider these interdependent parts of a single system. Cause I think that there's a danger when people focus on Oh, we're going to fix our culture, especially if they don't fix their incentives and their comp plans and their org charts, the rest of it.

Where, Oh, it's all about culture. It's actually also about tools, right? Like there, you can have all of the collective interest and alignment that you want. But if you're on the battlefield against other people that have the technological advancement, you're going to have a, you're going to have a bad time.

So it's not just about culture. It's about that collective alignment coming together with technical

John Willis: excellence. But I think culture is the can opener though, right? Like I get your point. I think

Andrew Clay Shafer: I'm not disagreeing with that. I'm not

John Willis: disagreeing with that. I think the mistake we made early on is not describing as behavior instead of culture.

But then, but even that wasn't, that was necessary, but not sufficient. Like it was behavior that you change. Culture is sort of part of who you are. But behavior is what you change. But I totally agree with Andrew and you should definitely research and anything him and Jay are publishing about social technical, like it's spot on.

It's it is,

Andrew Clay Shafer: you run into another problem when you start trying to do this work in a place where you, they've told themselves for so long. That our advantages, our people and our culture. And then you show up and you're like, you got to change your culture. They're like,

it's just an immune response, immediate immune response. So

Yishai Beeri: if I'm trying to narrow this down, the question of hyperforming teams. I want to put it like a very concrete, practical angle to it. So I'm, I, I'm a leader of 32. Okay. Or so I thought 17 I'm a leader of of engineering organization or delivery organization.

And I want to know, are we performing well, I want to know, are we performing better than we have been six months ago? Is there a way for me to measure or at least approximate this? Or even, is that even the right question? Because if it's, culture is important, tooling is important. Is there a way for me to know that, yes, I am improving?

This part of my org is doing better than the other

John Willis: part? Yeah, I'll go quickly. I, my, and Andrew might disagree, and I think there's more than one, there's like lots of answers to this, but I think the most effective thing I found in the answer to that question Is a form of qualitative research or work with clients and then be able to find out, don't talk to the C level, don't talk to the directors, talk to people on the keyboard, really go through and Jay Bloom had taught me a lot about how to do this in a very sort of academic way, but using true qualitative analysis and use some methodologies in that and then break that out and show them what they're really doing.

Thank you. And then what I always tell the people that I usually do it for, which is usually a C level person or a director of the C level is don't just do this once. Okay, you think we're okay here or there? Do it again. And then to me, that sort of qualitative blueprint, like at least in a qualitative way shows you, where you've advanced and, and you can be a little more clever and create judo belts or something like that.

But at the very least, I think that sort of qualitative approach to any other sort of quantitative approach gets into, the, all the things that Deming would scream about, like what Andrew was talking about with KPIs and all that. And unfortunately, we're so fixated. Numbers are

Yishai Beeri: so much easier.

So you're advocating for what's, recently getting the buzz word, like developer experience, questionnaires, ask the developers where, like what they're feeling and what they're experiencing and due to everybody.

John Willis: That's the other point, right? It's not just the developers, right? Go and talk to the network people, go talk like in one swath, make sure you get like as many different groups.

Yeah, definitely talk to them. In fact, I found like in some of these meetings, I talked to the developer. I talked to operations and they start telling me how things are crazy. Then I talked to developers. Then I go back to the operations and I'd ask, a simple question like, how come you basically limit the amount of memory a developer wants for, and they'll say like they, they never know what they're doing. I'm like, yeah, but they run an application that brings in about a billion dollars in revenue a year. And I think they know what they're doing. Like, why don't you trust them? Like you like start like seeing these like cross, and the Phoenix project, the Brent like finding the Brent.

Yeah. It's magic and that's Kevin Baird, apparently Brent was a real guy and his name was Brent, but I was, I had a kick out of that, but yeah,

Andrew Clay Shafer: Andrew. So I believe there's active harm that has happened because of people overemphasizing the deployment frequency. That, and I won't name names or, to protect the guilty, but there's organizations right now that are trying to deploy faster and haven't necessarily like the other metrics.

Yeah, we'll get to that later, but it's if we just deploy faster, like it will fix all these things. And I just don't think that's true from my experience and related to this conversation and something you just brought up, you should definitely ask the developers. But if you optimize for the developers, only the developers, then you're going to sub optimize the system for operations, or you have the potential to.

Right now there's all this excitement and focus on, oh like developers, they shouldn't have cognitive load. Where's that cognitive load gonna go? Someone somewhere has to shoulder has to think about this. Yeah someone has to know what's happening, right? So if you optimize everything for developer experience then in many cases, what you've optimized for is technical debt and ever increasing operational burden and ever increasing attack surface.

And I reject the premise that platforms are a product and we can have that conversation because I used to say that actually a lot going back to 2014, I've never talked to anyone about any of this and I use this framing of high performing teams have a platform since 2014. The reason I say platforms are not a product at this point is they have to be a commons.

It's not something you're delivering for the developers. It's something that the developers are an active part of as well. And the most successful and these are Kubernetes and I can like name names and in private Are where you actually get the platform development from the developers themselves, to some degree, it's an open source project.

They can see the code. They can update the code. No one has more context about what the platform should be like. And you run into this other issue, this other dynamic, which I've been part of some of these before, where, we'll build it and they will come. No, you're going to build it and they're going to run because you didn't build it for them.

You build it for you, right? So you got to find that balance point where the collective interest and that's the developer interest. That's the operator interest. That's securities interest. That's compliance interest. All these things are manifest in that platform that keeps promises to all those parties.

That's when you really get the optimal socio technical system is when. Those things are all represented in the platform.

Yishai Beeri: So you're basically saying that there's a new, there's a new kind of wall. That, that old DevOps wall. Now there's a wall where there's a platform team trying to ship a product.

And you're saying, no, this has to be. Emergent from the actual users building their way out of their problems, not using

Andrew Clay Shafer: it in the worst case, in the worst case, what you have right now in some organizations is you have a DevOps team that was funded however many years ago, then later you got an SRE team that was funded and they don't necessarily work together and now you have a platform engineering team that doesn't work together because and this is something that I say frequently in some of my work and conversations is if you show me the org charts and the funding model, then I'll predict everything that's hard for you.

And from the beginning when people started making DevOps teams, I was always cringing because like you, you solved your silo problem by making a new silo. That was not the, not, that was not the answer. So it's about getting that collective interest. It's about taking advantage of expertise in the right way at the right time.

It's not about some magic methodology. And this is the other thing. And I think this goes back to some of the earlier themes is you have all these people talking about high performing teams or DevOps. And there's all these like DevOps consultants and DevOps products and marketing ruins everything running around.

But in many cases, when those consultants show up, and they're telling you how to do their DevOps, none of those people ever worked in a high performing team before. And so so you have this kind of And the metaphor I use is, at least in the United States, you have all these like strip mall, karate studios.

Where you have this master with a black belt and there's all the little kids and they have trophies from some, they did forms in some dance competition or whatever. And those people have never been in a fight. Like that sensei has never been in a fight and their sensei has never been in a fight.

So if your goal is to be in shape and like flexible and maybe get some discipline no, no harm, no foul, go do that. But that is not to be confused with someone who can really bring it when they need to,

John Willis: if there's a fight. And, just to add to that, back to the qualitative versus quantitative approach, right?

Like the reason I love. This sort of the qualitative approach is if you think about the quantitative approach, basically what you're trying to do is say, prove a theory. So the consultants that come in, they're saying, I know DevOps, and they start creating the sort of strict mandates of things you have to do, and then quantitatively measure those things.

And I'm not saying I'm an expert in this, but the thing I love about a qualitative approach, which is very hard for people like me, and probably like Andrew is you have to keep your mouth shut learn how to, like, when somebody brings something up, you want to give them a tutorial oh, you should look at no, button it and just listen.

And so you're working like the adaptive, like instead of, the changing the paradigm of I know what it is and I'm going to prove it quantitatively. It is I don't really know what it is and it's going to tell me what the theory is.

And I think that to Andrew's point, I think a lot of people, whether we call it, quantitative or they come in with this sort of bag, which is, like you spend 10 years learning DevOps and you get pretty good at it but the hard part is just keep your mouth shut, wait.

To give that advice, let this grounded theory come up and see it emerge and then start looking at the patterns that, work well, but like too many consultants to Andrew's point, have never done the scale thing really have never solved a real hard problem at scale and literally just bring the toolbox in.

That is like everybody has a book, they should measure lead time. You should like, Oh my God. If you're not measuring deploys and lead time, like you're doing it wrong. Like, why? Wait a minute. Tubbo pal says just because you can go fast doesn't mean you actually have to go fast.

So

Andrew Clay Shafer: yeah, if you went to a doctor and they always gave every single person that came to them the exact same prescription, you wouldn't think that's a good doctor and I would wager they wouldn't get great results. So there's this context specific. Problem that you need to solve with a contextually specific solution, and most organizations, for a variety of reasons, are not able or willing to go through that authentic journey to get to the other side of it.

Got it.

Yishai Beeri: And what I'm also hearing is that again, this is a lot about humans, right? A lot of the problems, the pains that the qualitative and listening mode is going to surface our human fallacies, right? Broken communication or structure is, affects the way humans interact. So tooling obviously in methods, but a lot about how we solve the human problems.

And that has to be context specific, right? People have history, have loves and hates.

John Willis: You can't get around those. And the qualitative approach, right? And the qualitative approach, right? I know we're going to wind up, and I definitely want to talk about my book before we go. But the the qualitative approach, right?

I can ask the question. So if I've done it not to Pick on any of the sort of DevOps surveys and stuff like that. But I get like a one to five on one question, right? Or one, yeah, one to five, like a score. But but like with the qualitative approach, I get to ask the same question.

And then I get to hear three different answers. And if I'm doing it I get the three different conflicting answers to discuss it with each other. So there's so much more and it costs a lot more. And it takes, where, you would go in and probably do. A DevOps survey and a quantitative approach in less than a week, right?

If you like and say you're gonna do it, right not just send it out to everybody I mean it probably takes in a large organization I know for a fact that like I did one took two months one summer to do a large bank, right? Like most People want, most executives want the easy button, right?

They want the easy answer. So we struggle with the do you really want to take the time? Me and Andrew struggled with this in our last job, right? Do you want to, do you really want to do this? Yeah. Yeah. I want to do it right. It's going to take you this much resource, this much time.

No, I don't want to do

Andrew Clay Shafer: it that way. I think there's a tendency to want to abdicate everything to like some kind of magic process, magic bill. And the evidence I'll provide for that is the existence of safe and the fact that anyone thinks that's a good idea where DevOps is that one little box right down there in the corner that, that will solve your problems.

But that gives you a way to measure the installment of the process instead of this kind of focus on the collective outcome and the actual high performing results. And then you get this feedback loop of what I'll call process wonkery, where everyone's focused on doing the process or are we doing the process?

And if we're doing the process, but then you go into all these places and this has happened over and over. And in many ways was the inspiration for that. There is no talent shortage talk is people will say, and it was agile at the time, then it became DevOps. And then now it's SRE is oh, we're doing agile or we're doing a DevOps.

And it's you're. You're not good at it. You're not good at software. You're not good at any of the operational things. It's but we're agile. It's okay, then if you're done, then. Good luck. You get, and the way that I think about this, and you can apply this to the rest of the buzzwords and the buzzwords scavenger industry is, you get the DevOps, you get the DevOps you deserve.

Yishai Beeri: Yeah, we're starting to run out of time, but John, I would like to hear you you're writing a book, it's like publishing Any minute round. Yeah. And tell me,

John Willis: Next Tuesday, so Tuesday, wow. August 8th. Yeah, it was in fact I'm gonna be at Dev State Chicago and I'm gonna be able to give out some free copies there.

And just good stuff. It's been working on this thing for almost 10 years. It's been a, cash. So we're 10 years. Actually. Give

Yishai Beeri: us a tagline.

John Willis: The tagline is the name of the book, which is Demming DevOps to Deming. It's Deming's Deming's journey to profound knowledge. And the book is about, his life of codifying this idea that ultimately he at 93 years old became his manifesto, which he called the system of profound knowledge.

And and I'll be talking about that in Tel Aviv DevOps, about what that means from operational definitions, and it really ties in really well to like, when we talk about Dora metrics or metrics, What do you really. What is lead time really like I, like I, I got invited to the door.

Nathan Harvey invited me in to do this talk to the Dora sort of group. And I'm like, do you really want me to come? He's absolutely. And I talked about operational definitions, right? And operation stuff is okay. What is lead time? Is it, is it like commit? Is it like, is it to deploy? Is it dark launches?

Is it like, is it, is it first commit? Is it the sort of last pull or merger? And you have an organization that is running like. Like every other group has a different sort of definition and then you're aggregating that all together and saying, we're a high performing organization, stop this nonsense.

But yeah, so it's August 8th and literally over the last two years, man, it's actually three years. It took me about two years to write in a year to create revisions. And I had a great network of people who helped me make it way better than it was if it's first draft. But yeah, August 8th, I'm incredibly excited about it.

It's going to be the Kindle version first. There's a, there's actually still a supply chain issue with printed books. So I'm going to have to wait till to January. They're not doing just in time. No, it's IT revolution and they've really grown up as a more, they're not adjusting time.

They're not doing demand printing. They do real, they've literally turned in from early days for the sort of demand printing

Andrew Clay Shafer: to unit economics also play a part. Yeah,

John Willis: that's right. Yeah. No, they had a bigger scale now. So now they're into the, like how do you coordinate with boards of noble and like just the whole.

And so in that world, it's not a print on demand. So yeah. But that's fine. I just wanted this sort of Kindle version. I just wanted out there, I've got a few people. In fact Andrew gave me, I knew Jim Whitehurst cause I think he was critical to hiring us at Red Hat. But but the like Andrew came, I didn't have this sort of X Red Hat stuff and I pinged him and so I have an incredible.

Backpage quote from him. It's in fact, I sent him an email. I said, Mr. Whitehurst, Jim could you would you mind you don't have to read the whole book. I know you're a busy man, but but it'd be cool if you could say something about Deming and, cause I knew he worked at Delta and like he was aware of Deming and and he wrote back to me, he said, I was going to skim through it, but I started reading it and I couldn't put it down and he said it was a great book.

And then he gave me this incredible back page. Yeah. Quote, which is, I got Jeffrey Leiker, give me back bridge, so I got a pretty cool

Yishai Beeri: Thanks. So this is exciting. This is happening next week. As we

John Willis: were, it's a big deal for me, 10 years, 10 years in the works. And then, hopefully at I might have some early NFD versions to be able to bring with me in Tel Aviv at that point.

Yeah.

Yishai Beeri: And, obviously looking forward to hear both you, John and you, Andrew, here in Tel Aviv and the DevOps days to close and maybe just in a quick sentence, I know this is a big topic AI, LLMs, generative AI. How is that? Even change, like relevant or changing this whole world of delivery.

I can't

John Willis: do it in a minute. Like I can't do it in a minute, but I think it, the answer is it changes everything. It changes everything. And it would take way too long for me to give you the example. I'll tell you one thing I will say. I've coded more, and this is not even the big point, right?

I this is a point I don't care about. I think the part I care about is how organizations are going to fundamentally change when they find out there's ways that they can do, use these technologies. And incredibly, you think about incident management, you think about risk management there are definitely ways to stop hallucinating.

People always complain about hallucinations, they're called vector databases. You can solve that problem. But one sort of just data point for me is I've coded more in the last three months than I've coded in the last 10 years. And that's because of Copilot, like the annoying things that I hated about coding, like figuring out like a fourth level, a JSON object that has a list that I got to pull to and set it into two variables and one instruction, like there's no way, like I, in the past, I'd have to go and go look through four tutorials, find, now I literally give the JSON object to Copilot or JAT GPT.

And say, get me this field, and it gives me the code. It's just, and it's not and I'm not like, I've been, I've coded a lot in my career. I've coded for five years, then I won't code for five years, then I'll code for five years. I started out as a mainframe IBM assembler coder for five years straight.

So I know how to code. But the so I can look at code, and I don't just say, oh, great, thanks, ChedGBT, and just slam it in. I can, I know enough, if you know how to code, If you don't know how to code, this stuff is incredibly dangerous. But if you have, if you've got a memory muscle on coding, then A, you know how to ask questions.

You can look at the output and say, no, that's not what I wanted. And then you know how to test the code, right? Yeah, like we could spend more time on this and tell you, but I think it, the fundamental changes that we're going to see, we're going to see hype based ones, but we're going to see real implementations, which has changed like we've never seen before.

Andrew Clay Shafer: So I think it accelerates, but the fundamental principles don't necessarily. Dynamically change, like what you can generate as an individual. It's a force multiplier for an individual or a team to be able to create things. At the same time, I personally think it will elevate the need for operational excellence because now you're just going to get more and more code that like literally no one understands because it was generated by these LLMs.

So there's going to be there's also the interesting. thread we could pull on around the dynamic of how it impacts things like open source where people might be I think that you're, there's a bunch of other things going on with this, like open source midlife crisis, but it might create a dynamic where people are more reluctant to share certain things because they don't want it to train the AIs.

There's this a whole bunch of like unforeseen things too, but I, like all these things. I feel like you should just embrace the present and embrace the future and not be afraid. I'm not afraid of it. I love having the ability to, it's basically like having an expert that can help you write code, but also can help you write marketing and also can help you write anything else.

I'll give you an

John Willis: example this morning. I put it on LinkedIn. So I'm reviewing Gene's book right now. So Gene calls me on Friday and says, I need you to review my book. I'm like she would have asked me like six months ago. Cause it's so I've been like, cause Gene's done amazing things for my career, like big time.

So I'm like, all right, I'll drop everything. And so I literally, for four days straight, I'm doing a deep analysis on his book. And it's a co authored book Wired for Winning. I think I said, I should get the name right, maybe put it in there. But Wiring the Winning Organization.

But he's with Steam Spirit. Steam Spirit he wrote Higher Velocity Edge. Steam Spirit is just everything he says is just... Fire hydrant knowledge, right? In my opinion. And so there's a lot of spear in this book. And, but the spirit comes from a non IT and has all these, he ran MIT Sloan.

He's you're on the Shingo award, right? And so there's all these references. And so the thing, when you're reading somebody's book, right? One of the problems that you have is okay, as a learner, which Andrew and myself, you like with a lot of us are, right? Like you see something that's really interesting.

And you're like, Man, I'm not going to go read that book. I don't even want to buy it on Kindle right now. I'm trying to read this book and I found myself today. I went ahead and there was a book about the designing, which was like all out of like just industrial design, and a complex systems and industrial design.

And it was a reference to the book. So I asked. Chachi Petit to summarize it and he gave me four beautiful paragraphs and I knew it wasn't Hallucinating because I could read it and tell based on what I read in Gene's book And then I said, you know what and it mentioned the part about complex systems.

So I said in this book can you tell me the relationship? Between the complex systems and the sort of main point, and then it gave me like another 300 words. And at that point it probably started veering off the topic a little bit, but the point was it was absolutely germane to the thing I wanted to understand.

And then the last piece, let's to finish the point, I just said, you know what? I'm thinking maybe I should buy this book, but I should waste my time. And I just, this was more of a giggles, just what the heck I said. As a DevOps author and enthusiast and implementer, would you recommend this book?

And it said, as an ai, I cannot recommend books, but these are the things you would learn in this book. And I was like, yeah, that was a great answer. And I bought the book. So that, to Andrew's point, I think that's where our IQ get just gets raised. The, to the feedback loop, the fact that I could read some book and I don't have to go wait to get it shipped or, go ahead and do a bunch of stupid searches and even in Kindle, it's not

Andrew Clay Shafer: about art.

It's not about artificial intelligence. It's about augmented

John Willis: intelligence. It totally is. Yeah. And so now I use it for like everything. I just, I don't double think anything. I literally just say, let me see what chat GPT says about this. So it's, I dunno, it's, it, like I said

Yishai Beeri: We're in for some exciting times.

Andrew, John, thank you so much. This was amazing. I'm looking forward to meeting you in person in DevOps Days Tel Aviv and hopefully maybe do a. Quick recording on site after your keynotes to do a a local wrap up again, thanks for the time. Really great nuggets in gems here, that I'm sure everyone interested in the DevOps movement in the history and in the future of what's coming, can find here.

Andrew Clay Shafer: Thanks for having us. Thanks for listening.

Yishai Beeri: Thank you and see you

John Willis: soon. What he said. Hey, all all right, guys. Thanks.