The Three Faces of Complexity

As programmers, we want to control complexity and simplify our software, but these things are often hard for us to define. We know them when we see them!

This week, Mike and Erik talk about what it means to say that software is "complex" and along the way they'll offer three definitions of complexity, so the next time you're at work and commenting on pull requests and telling someone that what they've written is too complex, you can cite these random guys on this podcast!

For music this week Mike has been listening to ORQUÍDEAS by Kali Uchis while Erik is listening to SUSS, a work of ambient country music (whatever that means).

Chapters

• 10:00 Human Limits
• 20:00 Fred Brooks' "No Silver Bullet"
• 40:00 Herbert A Simon "The Architecture of Complexity"

Transcript

(upbeat music) - Hello, welcome to Picture Me Coding with Eric Aker and Mike Mall.

Hi, Mike, how are you doing this week? - Doing pretty well, how are you? - I'm doing great this week.

Feel pretty good, although I injured my knees. - It's supposed to rain today, in spite of these things?

I feel great. - Bummer about your knee. - Yeah, I think it's old skateboarding injuries coming back to bite me.

What have you got for me this week?

What have you been listening to? - So there's the album I've had sort of on rotation is actually a new 2024 album, the latest album from Cali Uchis.

It's called Orquideas, which means orchids in Spanish, I believe.

Starts out as a really good pop album and then slowly transforms into a really good reggaeton album.

So it's a pretty fun listen.

Lots of different variations on pop music styles and Colombian music styles.

And there's a track on there with Carol G, which you might like.

So my Spanish is not quite good enough to follow it in most cases, but it's one of those albums that gets better on repeated listening.

So I played it a lot.

My nine year old loves Carol G and my wife is consternation about that because there's a lot of bad words in it. (laughs) But he doesn't understand the bad words.

And so I can't quite tell if this is a horrible parenting move or not. - Yeah, there's a couple of tracks in here, which probably are at least R rated, mostly in Spanish, a little bit of English. - I'm listening to the band Suss.

And they have S-U-S-S and they have an album called Suss.

And it is an ambient country album.

And I don't know how to feel about it.

And I just keep listening to it.

Maybe it's background, but I kind of like it.

I should probably admit that I just finished reading one of the best books I've ever read, Lonesome Dove by Larry McMurtry.

I'd never read it before.

I was shocked how much this book seeped into my blood and my brain.

And it's cast this pall over what I'm reading next.

'Cause it's not Lonesome Dove.

Maybe that's why I got into ambient country music.

Imagine these guys driving the cattle up to Montana from Texas.

Anyway, it's good, I like it.

I don't know. - So what makes ambient country?

I mean, is there like harmonicas or something? - No, there's no words.

It sounds like pedal steel.

I don't think there's any drums.

You know, I couldn't even classify it for you.

It's slow and what's the word I want?

Ponderous, kinda.

Did you listen to it?

Did I send it to you? - I listened to parts of it.

You sent it to me a few weeks ago.

I think at the time I thought it was kind of like, isn't this funny ambient country?

And so I listened to it for a while, but it's actually kind of pleasant.

I don't think I would probably seek out more of it, but it's... - Everyone I sent it to says, "I don't think I like this." (laughing) - I was flipping through things in Bandcamp this last week and they recommended an album and they said, "This is a jambiant album." (laughing) And I was just, I have no idea, but I was just, I had this image of like, extremely mellow, slow fish songs or something.

So I just kind of skipped over it, moved on to the next thing.

But... - When my son was really little, we'd make him peanut butter and jelly sandwiches and he would always tear them apart and like suck out all the jam from the one side.

And so we started calling him the jam pyre.

This week we have a topic that is endlessly fascinating to me.

We wanted to talk about complexity.

Complexity is something that people always reference in our field because it's usually stated to be a bad thing in software.

But sometimes you can't get around it, I guess.

Complexity is a hard topic to get our minds around.

Can you, you wanted to talk about this, can you talk about why you wanted to talk about it?

And also what you don't want to talk about because the word complexity could mean many things. - Yeah, I think that's kind of the initial difficulty with complexity is that even within our field, that means at least a few different things.

Obviously there is the entire field of computational complexity, which most of us engineers have a sort of back of the envelope familiarity with not too deep, enough to know the difference between order and squared and order and log in or something like that. - Yeah, linear, exponential stuff like that, computational complexity. - Right, and there's a field of algorithmic complexity.

Sometimes you'll hear the term Komolgorov complexity, which has to do with what's the shortest program that you can write to generate a particular pattern, program in the sense of like a Turing machine.

And then there's this whole other whole field of complex systems, which has to do with emergent behavior and self-organizing systems and stuff like that.

I think it's not really related to what we're talking about today, but I think it's kind of similar in the sense that it's also kind of embryonic and what Sean Carroll calls pre-paradigmatic.

So they haven't really figured out what it is yet.

So I think this thing that we are talking about today is something that all software engineers are more or less familiar with, but it's hard to define well.

We sort of think of it as being hard to understand or being hard to reason about something.

Is a complex system. - Yeah, simplicity is often a desirable property, people say, they want things to be simple.

They want it to be easy to reason about. - Right, and I also think that's generally sort of defined, maybe tautologically, simplicity is not complexity, but we don't know what complexity is, so it's hard to put our finger on what simplicity is. - Complexity is, that's easy.

That's just something that's not simple, right? - Right. - Why do you want to talk about it?

What appeals to you about the topic? - So I think two things, one is, it goes to a lot of what you run into, especially later in your career with respect to building systems.

Complexity is often an aspect of what you're trying to control with architectural ideas.

So I think there's this sort of dual relationship between complexity and architecture ideas. - I like that, can we focus on that for just a second? - Complexity is a thing you are trying to control.

What do you mean?

Do you have any examples maybe? - As a group of people building a system, you are trying to keep it non-complex, simple, so that you have this sort of vague notion that if something is less complex, but decoupled, so to speak, it's gonna be easier to maintain, it's gonna be easier for other people to build separate parts of it that fit into the whole.

There's kind of an idea that if you don't connect the pieces too strongly, it helps in terms of both the ability of people to understand how things work and also the ability of your pieces to evolve independently. - These sound like desirable properties.

The problem, therefore, is we need to get a group of people to agree on what is complex, and that's what we wanna talk about. - Right, and so I think that's the other side of this, is that I've been doing this for a long time and people have been talking about the complexity of software at least since the late '60s.

There was this famous meeting in '68, I think, about the software crisis and how things were getting out of hand and were too difficult to understand and lots of well-known people chimed in on that. - Yeah, and structured programming right after that, Dykstra, is interested in trying to make things simpler. - Yeah, and there's been some progress there and some lots of ideas that have come out of that, but it still seems to me like a long time later we still struggle with this problem and people are still talking about this being a fairly serious dilemma in the software world.

So I think another reason to talk about it is just because why have we made so little progress on this problem?

Is it just because the systems have gotten so much bigger and these problems grow non-linearly?

Or is it just because nobody quite knows how to proceed from where we're at? - I really like that.

So part of this discussion is we're gonna shake our fists as our ancestors and say, "Why didn't you fix this problem you complained about?

You just dumped it on us."

Like some kind of pollution. - This is our old manuals at Cloud episode. - So to organize a discussion, instead of rambling here, not that we've been doing that so far, I think we had three areas to focus on.

One, we wanted to talk a little bit about human limitations, perception, thinking.

And then we wanted to talk about Fred Brooks.

Fred Brooks is famous, no silver bullet essay.

And finally, there's a book that you handed me called The Sciences of the Artificial, which has a really interesting definition of organized complexity in it.

So three things, human limitations, Fred Brooks, and then this book, The Sciences of the Artificial.

Before we start with those, I wanted to propose that maybe we could try to agree on an example of something that's complex and an example of something that's not complex.

And in preparing examples for this, I found it really hard to sort of definitively say, "Here's a not complex thing."

So for a complex thing, I would give, as an example, and we're talking software examples, the Linux operating system, the kernel probably, right?

Which is eight million lines of code.

Would you agree Linux is complex? - I would agree with that, yes.

I do think this is an interesting example though, because I think almost everybody would say, "Yes, this is a complex piece of software."

But it's also one code base.

This is not really a distributed system.

Good example of how something can be pretty complex without being distributed or without having timing or consensus issues that you come into with distributed systems. - So Linux, an example of something complex, it's self-contained, it's an interesting example.

I like that you said, probably everybody would agree that it's complex.

Okay, so that's an example for us or something that's complex.

Now here's the opposite.

What is a software example of something that is not complex?

Do you have anything that leaps to mind? - It's really hard to find, to think of something that is like a commercial product or a-- - Functionally useful. - Or a well-known open source thing.

I guess there's that, the famous left pad service or something like that. - Left pad as a service. - Not even.

Sorry, go ahead. - I think people write non-complex programs all the time.

It's just that they tend to be things that are one-offs or which are used for very specific purposes and so they don't get distributed far beyond their creator.

My feeling is that probably anything that's become a significant open source project or a significant commercial project is probably meeting some criteria of complexity.

The first thing that leaped to mind, I was seeking examples where people in our field would generally say, "Yes, that's not complex."

It is an example of software, it is not complex.

The first thing that leaped to mind was Fizzbuzz.

Maybe it's a toy example, maybe it's too toyish to even be.

We could sort of run through a Fizzbuzz algorithm here verbally in a minute.

And we'll just sort of assume that people know what that is but maybe it's not a good example.

The second thing I thought of, and I think this is a real subjective, is building an HTTP server using a web framework.

That seems like such a solved problem to me.

But my immediate response was, this is something I've done countless times in my career, is that why it seems not complex?

I'm relying on existing tooling and I'm just patching Lego pieces together, is that why it's not complex? - Yeah, that's a tough one for me.

I mean, I remember the days when a web server was actually a relatively complicated thing because you had to listen to sockets and parse HTTP and so forth.

But yeah, especially now, you can start up in a lot of languages, you can start up a web server with a couple of lines of code. - Yeah, or Hello World type of program, maybe a similar type of example, but not that functionally useful.

That's why I think a web server is a good example.

And I think that gets us to this first part, which is about human limitations.

When I go to work and I listen to people talk about what's hard and what's easy, that doesn't seem like a great indicator to me of what I want to call complex.

I think there's a common phenomenon in any field where we overlook complexity and things that we're familiar with.

And this actually has a name.

The name is the expert blind spot.

So I kind of have a blind spot for stuff that I'm good at.

I don't see what's hard about it or complex about it.

The stuff I already know is easy, the stuff I don't know is hard.

So that subjective, if you hear people just say, that's easy, that's hard, not a great clue to what we would want to agree is complex.

What do you think? - Well, yeah, it's definitely subjective.

I mean, I think my personal touchstone has always been, things are complex when I can no longer hold what they're doing entirely in my brain.

But that obviously is not something that translates well from one human to the next. - But I think that's a good rule of thumb too when talking about human limitation.

It's complex when I can't hold it in my brain.

That recalls to me the famous paper by George Miller.

The everybody probably has heard this idea that you can only hold in working memory so many things in the human brain.

The name of the paper is the magical number seven plus or minus two, some limits on our capacity for processing information.

And I actually looked up a little bit of data about this.

It is one of the most highly cited papers in psychology, came out in 1956.

And it has been interpreted to argue that the number of objects we can hold in short-term memory is seven, give or take two.

And in fact, this became so famous, it came to be known as Miller's law.

So complexity may be a subjective human thing, what we can hold in our brains.

And maybe that is about what we can instantaneously reason about.

And when you talk about the web server example, you said, I remember when that was hard because you had to build a lot of the pieces and now you don't.

And so I could sort of just treat these existing abstractions and glue them together with a few lines of code.

And boom, I have a thing.

And it's because I don't have to hold very much in my head when I'm making that thing, that it doesn't seem complex.

Does that conform with what you were saying there?

I don't have to hold a lot in my head. - Yeah, I think so.

I mean, I remember early in my career telling somebody that I felt like I could write a certain number of lines of code as an individual and still be able to understand what it did.

But once it got over a certain threshold, it was hard for even for code that I had personally written for me to understand how it was all working together sometimes.

There's, I have a little bit of a concern about this idea that to make things more cognitively manageable, we can treat pieces like other external complex pieces like a black box that we don't have to worry about.

I think it's true to a certain extent, obviously, but it also, it seems to, like with the microservices idea that has been popular over the last decade, I, my personal experience has been that, yes, it's easier to work on a individual microservice than it is to work on a full system.

But microservices don't necessarily make it easier to assemble things into a system.

I think the idea that you can sort of make things simple by making external pieces into a black box is not entirely true.

I have a hard time quantifying why that is, but it just doesn't seem to scale that well to me. - Well, you gave the example earlier of distributed systems.

And whenever I think of distributed systems, I think of this Miller's law.

I think what makes implementing something like RAFT so difficult or reading some of these papers by Lamport and Liskoff and others is they get very rapidly, they approach this capacity from my mind to hold information simultaneously.

I need to know what A, B, and C are doing and I need to think about it in time, maybe, and I need to think about how they relate to each other and how there's like this quadratic relationship of A talking to B and C and B talking to A and B and C talking to A and B.

Or sorry, I messed that up.

See, I even got confused there.

The hard part for me is, is there something objective about complexity that exists beyond the limitations of the human mind? - Yeah, I don't have a good answer for this and I keep running through this in my own brain.

I want to answer to the question of whether or not complexity is really a engineering thing or maybe even a mathematical thing or if it's just a thing that has just a thing that's a consequence of human limitations.

A lot of these solutions, quote, unquote, solutions unquote that people have come up with have been more organizational than technical.

So there's methods of trying to make communication more simpler by not having everybody on the team communicate with each other.

Brooks made this point a long time ago that the more people you have, the more difficult communication becomes because you potentially have this n squared connection between people and so people who try to make that simpler and people tried to make the cycles of development shorter so that we don't have to deal with things changing while we're actually writing the code.

But those all seem like organizational things to me. - Organizations are a good thing to bring up because we can say that maybe as a piece of software gets long enough, once it reaches a threshold of length, then it's complex.

There's just no way you can hold the whole thing in your head.

Maybe it's human limitation, that's fine.

But that human limitation has a threshold.

It's gotta have a threshold.

But then we might say, we're gonna take this long piece of software and start organizing it.

And we might have a feeling at that point that once we start organizing it that the complexity is reduced.

But it's the same quantity.

It's the same responsibilities that are invested in that piece of software.

So what does that mean that complexity gets reduced?

Before we answer that though, you brought up Fred Brooks and I had seen references to this paper, No Silver Bullet, Essence and Accident in Software Engineering many times over the years and you invited me to read it.

Tell us how this fits into the conversation of complexity as we're having it here. - So Brooks is probably most famous for the Mythical Man Month.

And in addition to being a sophisticated computer scientist and a software developer in his own right, he had these ideas about group organization and so forth.

And I think he wrote this paper as sort of a response to other ideas that were coming out at the time.

And so that the implication of the title, No Silver Bullet, is that there really is no one thing that we can do as engineers to eliminate complexity.

I think he borrowed these ideas from Aristotle perhaps but I think he talks about what he calls essential and accidental complexity.

And he talks about how we can reduce accidental complexity and people have done so with things like higher level languages.

So with the emergence of higher level languages, people didn't have to write in machine code anymore and that reduced the complexity of building systems and allowed people to build bigger things.

Similarly, he talks about time sharing the ability to for multiple people to use a computer and do things interactively, reduce complexity. - So essential and accidental complexity. - Right, so he talks about you can take steps to reduce the accidental complexity but in his view, you cannot eliminate the essential complexity. - And it behooves us to reduce the accidental, we should be doing that I think is the argument, right? - Yeah, trying to identify ways to do that and building tools to reduce those things is desirable and probably in his view, the only route to reduce complexity.

He says very explicitly though that there are these essential aspects to software development, the design of things and the interrelation of things and communication between people that really cannot be in his view eliminated.

Thus they are essential.

I'm not 100% sure I agree with that idea, but. - Let's take some lines, let's take some quotes for the paper and we can try to unpack them a little bit.

At the very beginning, he says no silver bullet comes from the terror of werewolves because they quote transform unexpectedly into horrors. (laughing) That was funny.

So we're working on software, we get it, we understand, we have a simple feature we need to implement and then suddenly what we've built transforms into horror. - Yeah, it's a good phrase for our daily limits. - And we wish we had silver bullets to fire into our applications.

He says in the opening of this paper, I thought, wow, this is very relevant.

He says, as we look to the horizon of a decade, hence we see no silver bullet.

This is a 1986, by the way.

There is no single development in either technology or in management technique that by itself promises even one order of magnitude improvement in productivity, in reliability, in simplicity.

In this article, I shall try to show why by examining both the nature of the software problem and the properties of the bullets proposed. (laughing) The nature of the software problem and the types of solutions that we try to apply to it.

Here's what's amazing and frustrating about reading this paper, 1986.

This stuff still extremely relevant.

He says the inherent properties of modern software are complexity, conformity, changeability, invisibility.

Those are still the problems we deal with, are they not?

I think so, yes.

Yeah, it is kind of depressing that this paper is approaching 40 years old and most of these problems.

If you wrote this paper today, nobody would probably disagree with the ideas, although you would probably have to change some of the stuff at the end about particular solutions.

Oh, solutions, right.

'Cause he says at the end, we're gonna spoil the ending here.

He says, I'm really looking forward to object-oriented programming as a solution. (laughing) And I was like, well, hold on there, buddy.

Out of the tar pit 20 years later, it's like, we agree with you on the problem, but your solution is completely wrong.

Anyway, here's a section on the distinction that he makes in complexity, which you mentioned, Mike Accidental and Essential.

So he says, following Aristotle, I divide them into essence, the difficulties inherent in the nature of software and accidents, those difficulties that today attend its production but are not inherent.

The essence of a software entity is a construct of interlocking concepts, data sets, relationships among data items, algorithms and invocations of functions.

The essence is abstract in that such a conceptual construct is the same under many different representations.

It is nonetheless highly precise and richly detailed.

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation.

I have to admit, when I read this, it sounds to me like a typical software developer who's saying, actually writing the code is the easiest part.

It's the ideas that are hard. - Yeah, and it's sort of hard to argue with, even now in 2023, what he's saying resonates and it would be hard to say, no, that's clearly wrong.

I think in our time, there's maybe some questions about are these all things that are issues with human beings developing software?

And if you had a magical robot that could evaluate code, would it really care about all of these things that he's saying essential?

I'm not entirely sure. - Maybe just to make it clearer, we could talk about what's accidental.

Do you have in mind any examples of what's accidental?

And to give you a minute to think about it, I'm gonna steal an example from the out of the tar pit paper.

I don't believe we'll have time to talk about out of the tar pit today.

I think we're gonna save that for part two of this discussion.

And out of the tar pit, they give an example, the performance of an algorithm as accidental complexity.

It's a thing that you can accidentally write a program such that it's slower than it needed to be.

And you can write a program which is fast enough and then later on realize, oh, it's a little too slow and you could refactor it and fix that, right?

That's accidental complexity.

Whereas essential complexity is fundamental somehow.

You cannot get around the complexity inherent in maybe representing a particularly complex idea.

What do you think?

Do you have any examples that leap to mind for you, accidental and or essential? - So this goes back a little bit to one of our previous episodes.

And maybe it's just because it's my particular bone to pick, but I think things in our current milieu are things that are accidental and that are not really essential to the functionality we're trying to develop are things like creating Docker files and creating Helm charts and things like that. - Plumbing. - Plumbing.

I think those are things that will inevitably become less of a part of the developer's job as time goes by because they don't have as big a bearing on the essential thing that you're trying to develop.

But part of my fascination here, and this runs as a current through a lot of our discussions whether it's about Open AI with chat GPT or something like that, we as programmers have to take meaning in the world and encode them in software.

And we have to know that we've done that correctly.

We have to take ideas, which are fuzzy, blobby things that exist in brains and write them down in code that performs in a particular way.

And I think that deployment is one of those things.

Deployment seems like plumbing.

It seems like, well, I just wanna get it out in the world, but there are interpretations we make about what it means to get it out in the world where I would put deployment as a kind of essential complexity.

And sure, the means by which you achieve it might be these things like Docker files or accidental complexity, sure, I buy that because somebody may invent some other way for me to write a DSL that magically permutes my project program into a thing that can be sent up to the quote unquote cloud.

So sure, I agree with you that part of that may be accidental, but deploying software, I think is essential complexity. - Yeah, I don't entirely disagree with you.

I think there's certain aspects of what you're encoding and say a Helm chart that have to do with the architecture of your system and that achieve architectural things like efficiency and reliability and availability and so forth, like specifying the number of replicas that you want in a particular environment is probably more essential than accidental.

I just sort of feel like the way that we do those things right now is overly difficult and will probably be simplified as time goes by. - Okay, so accidental complexity, it's a thing that we will be able to remove at some point.

Essential complexity, the implication is it's essential.

It's part of the essence, the being of this program.

You're never gonna get rid of it, right? - That's the idea, yeah, I profess to some skepticism, but keep in mind that Fred Brooks was a, during a word winner and I'm a dude with a podcast, so. (laughs) - Your podcast is like an audio blog, that's how I think of it.

Here's a passage that you drew my attention to, but I want your help interpreting it.

I'm gonna read it, okay? - Okay. - I think it's an attempt from Brooks to describe what he means by essential.

The complexity of software is an essential property, not an accidental one.

Hence, descriptions of a software entity that abstract away its complexity, often abstract away its essence.

This could almost come out of Descartes or something, this statement. (laughs) Should we stop there and try to unpack that?

That's a dense couple sentences. - It is, it reminds me a little bit of the time, five or six years ago, where you had made a deal with yourself that every time you drew an arrow in an architectural diagram, you'd have to do pushups. (laughs) Yeah, too many arrows was my suggestion. - Yeah, the way I interpret this is exactly that way.

So when you're an engineer and you were working on a system, there's this tendency to jump to a whiteboard or maybe some diagramming tool and you start drawing boxes and drawing lines between the boxes and you're trying to explain the structure of the system by this box and arrow type of thing.

And I think that misses, it does describe sort of an aspect of your system, but it misses a lot of things also.

And that's my interpretation of what he's saying here is that if you try to describe something in one particular abstract way, you are inevitably leaving out essential things. - Okay, that's a pretty good description and it helps a lot immediately with the next paragraph, but it also reminds me of this idea.

I came up with a couple of months ago and I pitched to you and you were kind of like, okay, my idea was we can rate the complexity of a service and microservices by talking about how many other services it talks to.

We can kind of give it a score.

And actually I was talking about a fragility score, right?

Every other service it talks to increases its fragility score.

This is similar to if you read the Google SRE book, they talk about your SLA has to be weaker than the SLA of systems you're connected to.

That those, if you wanna actually maintain this SLA, then you have to have a weaker SLA than theirs.

When we start connecting to a lot of other services, it's hard for me not to think about refactoring it.

Is this accidental complexity?

Do I really, really need to talk to these other services?

Because as I do, I'm making a service that's more fragile. - Yeah, and just to be clear, I don't disagree with the concept.

I think that's definitely valid.

I think my concern about what you raised is that there is sort of a trade off of, if you tell people that talking to other services makes their software more fragile, there may also be a tendency for them to try to take everything that service does, including included in their own code. - Their own code. - So duplicating it is not helping.

If you have the same idea in multiple places, the dry principle, maybe dry is an attempt to simplify because you only want one of a complex idea.

If you take that complex idea and you copy it all over the place, you've increased complexity.

So I think you helped me with the next paragraph after Fred Brooks's complexity as a software is an essential property, not an accidental one.

He writes, "The reality of software is not inherently embedded in space.

Hence it has no ready geometric representation in the way that land has maps, silicon chips have diagrams, computers have connectivity schematics.

As soon as we attempt to diagram software structure, we find it to constitute not one, but several general directed graphs superimposed one upon another.

The several graphs may represent the flow of control, the flow of data, patterns of dependency, time sequence, namespace relationships.

These graphs are usually not even planar, much less hierarchical.

Indeed, one of the ways of establishing conceptual control over such structure is to enforce link cutting until one or more of the graphs becomes hierarchical.

There's a lot of ideas in there too, but it sounds exactly like what you're saying.

If you go to the whiteboard, you start drawing out services interacting with other services, you're not fundamentally representing the complexity of software.

You're not representing the essence of it because it's not a spatial relationship.

It's not a time sequence relationship.

It's not necessarily a dependency relationship, and these are all important too. - Yeah, the first sentence of this paragraph is probably one of my favorite sentences in any paper of all time.

I'd probably wear a T-shirt that had this written on it.

The reality of software is not inherently embedded in space.

I just love that sentence.

This is kind of old school stuff.

I don't think many people will remember this anymore, but back in the late '90s, early 2000s, there was a big push for stuff like UML, and in particular, there was this company Rational that was, the people involved with that company were some of the creators of UML, and they created what they called the Rational Unified Process, which was a software building methodology based around UML.

And that's kind of falling out of favor in favor of things that are more agile, but one of the ideas that I really liked about that, and which is still something like kind of clinked to, is there's no single view of a system.

So they advocated the idea that to specify the architecture of a system, you had to have these multiple views.

So you had to have class diagrams, and you had to have-- - Sequence diagrams. - Yeah, sequence diagrams and deployment diagrams, and to really get to all of the aspects of the system you were trying to build, you had to have these multiple architectural views.

And I think that's similar to what he's saying here, and I think it really is true, and it does get to the idea that he's trying to make about certain things being essential. - I didn't know a company came up with UML.

I thought I just came out of the Bible, but it's existed forever. - Yeah, I think it was kind of a bootstrap thing.

So it was a process, it was a standardization process, and then the people who were involved in the standardization started this company.

Also, as relevant to nothing, but because I think it's funny, at the time, Grady Butch was one of the people who was involved with Rational. - Tell us who that is, just for those who don't recall. - Well, he was a pretty well-known software engineer, one of the inventors of UML, among other things.

He worked for Rational, but he didn't want to be the president or whatever.

So at some point, I worked for a company that bought the Rational products, and at one point in time, I came in possession of one of Grady Butch's business cards, which said Grady Butch Rational Fellow, which I still think is the greatest business card of all time. - So you gave me a book to read called The Sciences of the Artificial, and you said, "Don't read the whole thing.

"Read the last two chapters, which are about complexity."

And I did read those, and I found them interesting.

So embedded in that last chunk we read from Fred Books, he talks about how you might have several different graphs to represent your software.

And embedded in there are data flow, dependency patterns, time sequence, namespace relationships.

The graphs are not usually planar, much less hierarchical.

Now, Sciences of the Artificial is written by Herbert A.

Simon, published in 1996, MIT Press.

This is a third edition you gave me.

And Simon's not a computer scientist, as far as I understand.

He's interested in complexity. - Yeah, although he did actually win the Turing Prize, so, but I think it's more for his ideas about complexity and software management and economics than theory or engineering work. - So the last two chapters of this book, The Sciences of the Artificial are, the last one's called The Architecture of Complexity, and the penultimate chapter is called Alternative Views of Complexity.

And in that Alternative Views of Complexity, he introduces Holism.

He says, "Holism is a modern name for a very old idea.

In the words of its author, the South African statesman and philosopher, J.C.

Smuts, quote, "Holism regards natural objects as holes.

It looks upon nature as consisting of discreet, concrete bodies of things which are not entirely resolvable into parts, and which are more than the sum of their parts.

The mechanical putting together of their parts will not produce them or account for their characters and behavior."

So there's a bit of a synergy idea there.

If I try to break a thing into its constituent parts and then reassemble it, it won't actually produce the thing that I experience in the world.

Somehow assembling the parts makes it, there's something bigger out of it, right?

Holism can be given stronger or weaker interpretations, applied to complex systems in general.

It postulates new system properties and relations among subsystems that had no place in the system components.

Hence it calls for emergence.

In a weaker interpretation, emergence simply means that the parts of a complex system have mutual relations that do not exist for the parts in isolation.

So this is sometimes we talk about software having emergent properties.

I think that's what he's talking about. - Yeah, this is a pretty common idea now in complex systems theory of you combine these things together and then surprisingly behaviors emerge that you did not design for.

People are talking now about how this is happening with artificial intelligences.

We didn't really expect some of the things that are happening with LLMs, let's say.

And so people are claiming that these are emergent phenomenon. - He says in a weaker interpretation, which I kind of like, it sounds like something I can wrap my head around, emergence simply means that the parts of a complex system have mutual relations that do not exist for the parts in isolation.

I read that before, but I wanted to re-emphasize it because it's like saying, we're surprised by how, when we put these pieces together, we're surprised how they interact.

I think that's what he's saying. - I think a more basic example is something like an ant colony, people talk about how colonies of ants have this sort of almost intelligent or intentional behavior, but no individual ant in that colony really has any idea what's going on.

They're just performing their job and responding to stimuli, but this sort of pattern of things that are beneficial to the colony as a whole start to emerge from that behavior of the individuals. - Simon gives a lot of examples from biology and he gives examples from physics.

He talks about complexity and he'll say, if you look at the structure of atoms, like we could talk about neutrons and protons and electrons and we have these ideas, but then you can go deeper, right?

And if you talk about the structure of galaxies and stars, it seems simple, but you can obviously go deeper and you can go from the microscopic or the quantum all the way up to this galactic scale.

He brings up this idea of near decomposable systems or near decomposability.

So for him, complexity seems to be involved in things that are nearly decomposable.

I'm not sure why nearly, but of course I thought of decomposition, I thought of Dijkstra, I thought of software.

When we start building bigger systems, we try to make them composable or maybe even maybe using his term decomposable.

He talks about problem solving.

He says, when a mathematician wants to solve a problem, they go through a process of trial and error.

They'll follow a line of thought and they'll try to pursue it and they'll see if it produces the result that they want and then they'll continue on from there, like following different paths in the forest or something.

He says, this is selective trial and error.

Obviously I'm not a mathematician.

If I tried to solve some incredibly difficult math problem, I'm not gonna make much progress.

Why?

He says that the selectivity we use when we solve problems is that there's some sort of feedback of information from my environment.

And he gives that feedback of information.

He says, there's information about quote unquote, stable configurations and information from previous experience.

A previous experience is easy for me to understand because I could say, oh, I can tell that this technique is not gonna work.

I tried it before, right?

Or I could tell that this technique is promising because it shows me something I haven't seen before.

But stable configurations, does that make sense to you?

Stable configurations? - I guess I related to what some people, sometimes called contingency and biological evolution.

So we think of evolution as being this thing where things mutate and then in some cases, those mutations are beneficial.

They help the organism produce more of that organism and so they are, they then lead to evolution.

But people make the point that the mutations are not totally random and the results of them are not totally random.

There's sort of a starting point from which that mutation. - A stable configuration is a starting point. - Yeah, a bird doesn't mutate into a dolphin or something like that.

They're making a mutation that is relatively incremental from the starting point that may or may not be beneficial, but it's not a huge variation from the beginning. - So that relates to this footnote that I found in the book and it relates to this watchmaker story, which I wanted you to tell if you can.

But the footnote was about how someone had performed some analysis where if you look at biological evolution as the random evolving of genes and aspects of organisms, that the time it would take to produce complex organisms is vastly longer than it seems like the time that has taken.

And so instead, if you say, well, it's not that random, it's about the evolution of stable configurations and then the sort of assembly of stable configurations, you can actually vastly reduce the amount of time it would take to then evolve more complex organisms.

I think I did a pretty bad job there, but I think it fits in with his watchmaker.

Can you tell us the watchmaker thing that's in the book?

Do you remember that? - So the basic idea is that he tells this parable essentially of two watchmakers and they are both assembling watches made out of, I wanna say a thousand parts or something like that.

So one watchmaker is trying to assemble all one part or a thousand parts into the watch.

At one time, the second watchmaker takes 100 parts each and assembles them into 10 sub-assemblies and then assembles the watch out of the 10 sub-assemblies.

And so he makes the point that even though it's the same number of parts and you end up with essentially the same functional thing, the second watchmaker is more likely to be able to survive or be able to mitigate problems with the construction because if you make a mistake in one of the sub-assemblies, all you have to do is restart that sub-assembly, whereas the first watchmaker, if they have a problem, they may have to start over that entire process of assembling a thousand parts. - Yeah, it gives as an example too, like interruptions.

If you're gonna interrupt this, and he says ultimately, he calculates that, it would take the watchmaker who doesn't use sub-assemblies or stable configurations.

It would take that watchmaker 4,000 times as long to produce a watch on average as the other one.

I think I got that right. - Yeah, I don't remember the number either, but that's the basic idea.

And you can immediately see how that applies to how we build artificial systems like software. - So we haven't even gotten to the really fascinating part, I think, of Simon's argument.

He talks about these nearly decomposable systems, right?

Stable configurations that get put together.

There's a holism.

Once these stable configurations get put together, there's a weaker emergence, a weaker interpretation of emergence, where you can say, wow, they interact in surprising ways.

There are relations between the components that are maybe unexpected that produce more than the sum of the parts.

And he says that for, he talks about organized and disorganized complexity.

He says, I'm only interested in organized complexity.

So we're kind of, I think, hand wave that one.

His argument is, if you look at organized complexities, they generally form hierarchies.

And for him, hierarchy is not, I've got a boss, which is giving instructions.

It's really like, as we would imagine, like a graph structure, almost, I think.

This is, you can use hierarchies to understand how, what is complex, and really fundamental to complexity for Simons is the concept of the hierarchy.

Does that sound right to you?

Is that how you interpreted what he says? - I think so.

He gives lots of examples from nature and biological systems, matters made of molecules, which are made of atoms, which are made of subatomic particles, and biological organisms are made of a tissue, which is organs and tissues and cells and molecules and so forth.

And so, the implication is that, one is that complex things tend to organize themselves hierarchically.

And also, I believe that it is easier for us as humans to understand the complicated things if they are organized hierarchically. - Yeah, I like what you just said there, because I'm not so interested in solving the question of, is this just a human limitation, or does it exist outside of us?

I don't feel like I'm gonna be qualified to really answer that question.

With Simons, I can say both hierarchies allow me to think about complexities, and they may be a feature of complexities in the world.

He says, "Hierarchies have this property "of near decomposibility.

"Within a hierarchy, the components inside," we can use the word components, "as they relate to each other, "those relationships are generally weaker "than the relationships within a component."

And there are higher frequency dynamics internally for a component, and that's contrasted with lower frequency components.

In physics, he talks about electromagnetism versus gravity.

We think, I think in terms of code bases, I might organize my code in a kind of tree structure, and code that I find easier to reason about doesn't have a lot of linkages all over the place.

It is hierarchical in the Simon sense, and the modules that are located close to each other on the file systems, metaphorically located close to each other, they tend to be more interconnected, and modules that are farther away have fewer linkages.

This is for code that I like, code that I find easy to reason about.

This is like an ideal for me.

When someone submits a pull request which violates this, I think, this is weird.

You've got this thing over here, but mostly these other things over here are using it.

Let's move it.

And so it fits in with Simon's hierarchies and linkages. - Yeah, I mean, I think this is an important idea.

It's come up in a couple of different ways in the software world over the years.

These ideas of coupling and cohesion that were, I think, first articulated in maybe the 70s. - But yeah, I mean, you want systems where you can make a change to one component, and it's not going to significantly affect other components that are distant from it.

But I guess the way I think of it is, it's kind of the way he presents it in the paper.

You know, you can sort of imagine this matrix of all the components in your system, and you would not want all of them to have, all of the matrix elements to have a value indicating that there's a connection between those component.

You want things to be structured in such a way that there are parts of it that you could essentially extract and they don't really affect other components significantly.

And obviously something that we strive for when we're building software systems is to be able to modify things in such a way that they aren't going to break other things. - This Simon's idea of hierarchies really appeals to me because it seems like a description of how I already think about building software.

And when I want to say this is complex, I think it's because it's violating these rules of complexities that are manageable for my brain.

When I say this is too complex, we should fix this, we should simplify it, we should make it so I have to think less about this.

If I wake up early and I'm gruggy and there's an error, I want to be able to identify it pretty quickly.

Can you make it so I have less to think about when I'm looking at this software?

You brought up microservices connecting to other microservices and it fits here, right?

We talk about how it's fragile as you have more inter-component linkages, but we expect there to be intra-component linkages.

Sometimes developers complain that microservices are more complicated than monoliths, especially when you're talking about data being distributed across a whole bunch of microservices.

If I've got a system that has users and they get authenticated and then we need to know what authorizations they have, well those authorizations may relate to dozens of different services.

How do I understand what those things mean?

If I want to make a change, do I have to change dozens of services at the same time?

That sucks.

There are known patterns that we're supposed to avoid such as having a whole bunch of microservices all talk to the same database.

People call this jokingly the distributed monolith.

Maybe the problem with that is it's less nearly decomposable in the Simon sense.

Maybe that's what you were just talking about.

This is a known mistake developers have made since the '60s and '70s. - Yeah, I think that's a good example.

If you have a bunch of microservices, but you also have them talking to the same database, then one, if you make a change to that database, it potentially breaks all of the services that are talking to it.

So you've sort of failed the decomposability test there.

Similarly, you've introduced some complexity because you may not have to worry about things like race conditions and locking records and things like that.

There is a trade-off, I think, though, by making every service have its own database.

You also introduce issues of not being able to enforce integrity constraints and things like that.

So I don't think it's a, I guess, to put it in a Brooksian sense.

I don't think it's a silver bullet. - Right, I guess, as you were speaking, I thought, what's essential here?

What's accidental here? - Yeah. - I think the people who complain about microservices being lame compared to monoliths, are their complaint, their complaint is, you're introducing accidental complexity by breaking these things up into different services. - Yeah, well, I mean, I think there's, we've had discussions about this recently and in the context of our actual work about, you wouldn't want to build three microservices that all talk the same database that's clearly an anti-pattern.

But is there a case to be made that if you have three Microsoft microservices that seem like they should talk the same database?

Should you just make those one service? - Yeah. - It's not clear to me that, necessarily, that breaking things out that way is essential or accidental.

I think this is why suffer is hard and complexity is still hard to reduce.

There's one other thing I thought was pretty interesting in Simon's is architecture of complexity chapter.

He says that hierarchical systems that we can understand are often redundant.

And instead of the word redundant, I would actually say there's patterns.

This is what we often look for as developers.

We read through the code, we look for patterns.

This is why dry comes in sometimes.

Are there patterns in what our software is doing?

Can we reduce redundancy by exploiting those patterns?

If you go back to the magical number seven plus or minus two, the Miller famous essay from 1956 about how much we can hold in our head, there's this concept of chunking.

The brain will sometimes build shorthand for referencing more complex ideas.

And it's a chunk of information.

I can use that chunk instead of trying to hold the bigger thing in my head.

That's a strategy that we naturally do.

And maybe when we talk about reducing complexity, we're looking for not just near decomposibility, but redundancy too.

I thought that was an interesting idea in Simon's. - I'm not quite sure what he means by redundancy there.

I mean, I think there are cases in systems we build now where we do intentionally introduce redundancy because we're trying to achieve some architectural goal.

For instance, we rarely have a service now that doesn't have at least two replicas.

We wanna make sure that if one replica goes down, there's still something there to service requests.

I don't know if he means redundancy in the sense of repeating things that could be combined or if he means redundancy in the sense of having a fail safe or a fallback. - His example was the first, that he had this matrix of letters and it looks kind of like noise at first.

But then if you look at a quadrant of two by two, you can see that that same pattern was repeated all over the matrix.

And so you kind of reduce the whole matrix to just this one pattern and say, take this pattern and print it out 12 times. - Right. - So let's return to Brooks to close this out here.

Brooks introduces this idea of essential and accidental complexity.

Seems like a great distinction for us to think about because I imagine as developers, what we're fighting against, what our war is about is accidental complexity.

Essential is the stuff that we cannot deal with.

And here are the suggestions Fred Brooks has for how to build better software in 1986, how to reduce accidental complexity.

He, aside from things he says he's excited about like language changes and other things in the future, which probably did help.

He says, you should buy versus build.

The most radical possible solution for constructing software he writes is not to construct it at all.

Okay, build versus buy.

He says, you should do more requirements for finement and rapid prototyping.

The hardest single part of building a software system is deciding what to build.

So rapidly prototype, refine your requirements.

That'll reduce accidental complexity.

And finally, this one I thought, probably the most controversial, he says, get great designers.

And he means software designers.

The central question and how to improve the software art centers as it always has on people.

And he's saying, find those people, train those people.

Not those necessarily the same groups in each case.

Train people to be great software designers and find those who seem good at it and make them better.

So build versus buy, requirements for finement, rapid prototyping and great designers.

What do you think?

Are these things going to reduce accidental complexity and allow us finally to write beautiful software that everybody understands?

I would give that a strong maybe. (laughing) I think the build versus buy question was a little bit different at that time.

The question then might have been something like, do we write our own accounting system or do we go out and acquire one?

And in 2003, you go buy an accounting system.

But if you need to verify fraud or something, do you write your own machine learning models or do you rely on a third party service to do it for you?

Something like that, I think is still a non-trivial question to answer.

So I think it's still an important thing to look at as a software organization.

You should always evaluate the possibilities, but I don't think it's as obvious as you would hope it would be.

I wonder if BISIDE2 could be, is there a open source library for that?

Yeah, that's another thing.

I mean, there's the total cost of ownership you have to consider, then there's the how much adaptation do I have to do?

How much learning do I have to do?

Is this the IP that I want to own?

All those questions you kind of have to answer.

The Jeff Atwood thing, is this your core competency for your company?

If so, you should build that.

Yeah.

The issue of some people being good to design and identifying those people and training them, sure, I'm all for that.

I don't want to touch that one. (laughing) That sounds real risky for me to be like, yes, I can identify those people.

I don't like that very much.

As you're aware, I think the robots are going to probably replace us all.

And so my sense is that in 1986, that was a pretty good argument to make, but in 2023, I'm not clear that it is.

I think I personally feel, and I will most likely be wrong, but I personally feel like a lot of that requirements, refinement and specification is going to transfer to the realm of language models and AI in the next, in the coming decades.

Someone is going to have to interpret the human meanings, though, and turn that into something, whether it's an instruction for a large language model, or, and someone's got to deploy that garbage.

Yeah, I think essentially, initially, at least, I think there will probably be tools to allow subject matter experts to use language tools to sort of refine and formalize what their requirements are, sort of like artificial product managers or something like that.

But I really do feel like that is probably more likely to be the route that we see in the future, more so than a specialized software designers, which really has not been that big of a thing in my experience in this industry.

I'm sure in bigger companies, it's more common.

There are people who specialize in architecture and things like that, but I don't think this idea of having specialized designer, software designers has really caught on that much.

So I think we're going to close here.

We did not get to out of the tar pit.

We're going to save that for part two.

Out of the tar pit is a 66-page paper.

I think it deserves a special conversation.

That'll be part two of our conversation.

In the future, when someone comes to me after I've commented on their poll request, can we make this simpler?

Can we plus one this?

Sorry, it's a bad joke somebody said to me.

Can we make this less complex?

I imagine that I'll start talking about hierarchies, abstraction, near decomposability, accidental and essential complexity.

So you know what?

We have satisfactorily defined complexity, or have we? - I'm just going to leave comments in poll requests that say, "Are you sure this is essential?" - People are going to love that. - A lot of people figure out what I meant. - They're going to say, "Yes, the ticket told me to do it."

All right, Mike, this has been a great conversation.

Thank you for the recommendations.

Reading Fred Brooks's No Civil Bullet.

And thank you for passing me the sciences of the artificial to read.

I learned a lot.

I found it stimulating. - You are welcome. - We're going to close there.

This has been Picture Me Coding with Eric Aker and Mike Moll.

My name is Eric Aker.

I will see you next week, Mike. - See you next time. - Bye-bye. - Bye. (upbeat music) [MUSI