Rooftop Ruby Podcast
Two Ruby programmers (Collin Donnell and Joel Drapper) discuss Ruby, web and native software development, technology, and more.
Rooftop Ruby Podcast
17: Technical Debt and Duplication Over Abstraction
Hello, Joel, how's it going?
Joel Drapper:Hey, pretty good. How are you?
Collin:I, I am doing well. I, uh, you can sort of see behind me right now that my office looks like a disaster zone, and that's because I did one of my patented twice a year, maybe quarterly. Take everything off my desk, put it on the floor, take everything out of my closet, put it on the floor, and then I will reorganize it. And then I will repeat that again in three months. So that's, that's, that's the thing I do. That's super cool. I did learn. This is gonna sound really stupid. Um, I did learn that I'd pat my monitor too close to my eyes forever. Uh, cuz I looked up, I'm like, what's the recommended distance to have your monitor? Um, cuz my desk is pretty deep and I thought, I'm not using a lot of space. I don't know why I googled this, but I did. And they said like 20 to 30 inches and I thought, that sounds so far, I wouldn't be able to see anything. And then I moved my monitor 20 inches away. So instead of being like right in front of me, it's at the back of my desk, I'm like, I can still see everything. This is way better. Like when I hear about people using 32 inch monitors, right? I've always thought like, how do they do that? They'd have to be like training their neck because like they couldn't see all of'em. So I feel very silly, but I have so much desk space now. I'm very excited about that. Um, you're going to a conference in like four days.
Joel Drapper:Yes. Um, yeah, I'm going to Brighton, Ruby, which is, uh, I'm, I'm, I've never been before, but I'm really excited.
Collin:Yeah, it's pretty close for you, or are you driving there?
Joel Drapper:yeah, I think, uh, I'm probably gonna go down the day before. Um, Uh, I, I'm guessing that a bunch of people are gonna be arriving the day before and that they'll probably be ISTs hanging out. Um, so hoping to not like, skip that just because I happen to live a few hours away. Um, just
Collin:yeah, that's when the, that's when the party's gonna happen, probably.
Joel Drapper:yeah, and it's, it's like a one day conference, so, um, yeah, I guess. There's, there's only really like the night before and the night after.
Collin:Yeah. Yeah, it seemed like a lot of people were coming in from outside of, uh, of the immediate area to go to it though, which was a little surprising for a one day conference cuz it's pretty far, uh, for a lot of people.
Joel Drapper:I was surprised too. I wonder if it's like, it's a good time of year, I guess, to, uh, to do a holiday in Europe at the same time.
Collin:Yeah.
Joel Drapper:I guess so maybe like a one day conference is almost ideal because it's not gonna take up like too much of your time, but you can go with family and do a holiday or something.
Collin:Yeah, the only part of Europe I've spent time in is Germany,
Joel Drapper:Okay.
Collin:so. I thought about trying to get to this one. It just didn't work out this year, but maybe next year, um, or maybe a different European conference. But yeah, that's how I would do it. I would use it kind of as an excuse to then spend a week doing
Joel Drapper:Yeah, totally.
Collin:Yeah.
Joel Drapper:Um, yeah, it's pretty fun. And you can, you can get to, um, other parts of Europe pretty easily from here as well. I think. Uh, I'm not entirely, I, I've not spent much time at Brighton, but I think it's pretty easy to get up to London and then, then you can just get the train to Paris.
Collin:Yeah, Paris is like two or three hours or whatever on the train from London.
Joel Drapper:think it's like two hours from London
Collin:Yeah, and I, I think, I think London was only like an hour or hour and a half from Brighton. It seemed pretty close.
Joel Drapper:Yeah,
Collin:Um, that's really cool. Uh, it's, it's really cool that there's a train that you can take, uh, and do
Joel Drapper:I know. Isn't it insane?
Collin:it's pretty nuts. Um,
Joel Drapper:Yeah.
Collin:I wish we had that, but like, honestly, like there is a train, um, there. Th there isn't that much here. Like things are pretty spread out. You know what I mean? Like, there isn't another major city like, like there's no two major cities on the west coast of the United States, like Paris and London. That would be like a three or two or three hour train ride from each other. Like, I could get to like Seattle, I guess. Like that's about it. I go from like Portland to Seattle.
Joel Drapper:Yep.
Collin:Um, but, but uh, but anyway. That's very exciting. Hopefully maybe you'll meet some people who listen to the show. That'd be cool.
Joel Drapper:I hope so. Um, if you see me, um, come say hi. Um, yeah, it'd be great to, great to meet people.
Collin:for sure. Um, so I wrote another blog post that you had feelings about.
Joel Drapper:Mm-hmm.
Collin:It was called, you told me that I was really going after like Hacker News bait with this one, and I think you're right. Um, it's not my proudest moment. Um, but it was called your what?
Joel Drapper:the, the context for this is that your previous one blew up on Hacker News
Collin:Yeah. I did not realize that MVC Post we talked about last week. This time it had about 60,000 views, which, you know, for an actual like, you know, big time blogger person, I'm sure that's not a lot. But in Collin world, that's a lot. Uh,
Joel Drapper:You only started this blog, this was your second post.
Collin:so my third post my,
Joel Drapper:third post to this blog.
Collin:post also kind of blew up. It had like 25,000. Um, so, uh, Then, you know, like my next one I'm sure will have like 30 like, or whatever. But, um, it's, it's, it's, uh, it's fun. It's, um, I definitely do get, I definitely do get nervous. Um, all the comments I saw on Hacker News were what I would describe as quite hinged, as opposed to unhinged, um, which I was scared of because I. I, I don't know. It's, it's not, hacker News is so far outside of like, my circle. It's much broader. And so I'm really like, is somebody gonna say something mean about me? Um, but, but they, they generally didn't. And so that was nice. So I, I should probably not be so scared of that happening. And in my mind it's a little bit if somebody was like, Hey, your post is really taking off on four chan, and I'm like, oh God. But that's not fair. Hacker News is apparently full of very nice people. Uh, I've just, it's scare. I, I should not make that comparison. That is not a fair comparison. But it, um, it did scare me a little bit just cause I'm like so many people that I don't know. Um, but they were all very nice. So I'm wrong. Hacker News is full of the nicest people on the internet. Apparently.
Joel Drapper:I mean, I'm not sure if
Collin:not true either. No, I'm not gonna say that. But they weren't mean to me.
Joel Drapper:Um, Yeah, I've read, I've read some pretty unhinged comments on Hacker News.
Collin:Yeah. Yeah, I know.
Joel Drapper:insane.
Collin:yeah, I'm being, I'm being very silly. Um, yeah, so I wrote another blog post, uh, just Cuz I did, it's called You're Never Going to Fix Your Technical Debt. Um, which is kind of a, that is kind of a click bait title, isn't it? Um, I didn't mean it to be, I just sort of was writing something, but kind of the. The gist of it, I would say was this thought that if you write code with the idea of we will come back to this and fix it, um, that usually doesn't happen. That was kind of the impetus, and so my idea was that. You kind of
Joel Drapper:Have to write it right the first time.
Collin:yeah. I don't know how else to say it. Like, my idea was that you can't write any, you, I think what I said was you need to do your best upfront, uh, but that also, you're probably wrong. And so whatever you thought was probably incorrect because you never know less than in the beginning of when you're starting something.
Joel Drapper:right.
Collin:Uh, And so I think that my kinda where I would come to is that the, the best strategy is to delay, delay, delay and to try and make as few decisions as you can and just keep things simple and cohesive so that you are not in one of these situations where you say, try to import this sort of clean architecture, super dry thing. Onto your code base when you didn't know anything. And now that is actually that architecture that was supposed to make this super maintainable is your biggest piece of technical debt that you are never going to get away from now because you based your entire code base on it. Um, that was sort of my thing. What do you think? Can you ever fix code or is impossible?
Joel Drapper:well can, can you ever fix your technical debt? You can definitely fix specific bits of technical debt. For specific amounts of time. Um, I don't know if you can ever completely fix your technical debt. And I think, um, what, so there are a few points you made in this post. One of one of them is basically, uh, I think you're kind of hinting at the fact that you are probably never gonna get time to come back to it. So don't think like, oh, it's okay, we're taking on some technical debt here, but like we can fix that later because you are almost definitely like, Maybe you want to take on that technical debt anyway, but you almost definitely won't be able to just come back and fix it like when you have all the time in the world in the future. That's true,
Collin:Yeah, once the feature works, it seems generally you aren't gonna be given time to go back and redo it for no reason. Um, there are,
Joel Drapper:I think you are, and it, I think it depends on a lot of things about like your work environment and or like even is this a piece of. Like work? Work? Or is it, um, is it like an open source project or something that can, there there's, there are different kind of appetites for fixing technical debt, I think.
Collin:Oh yeah, for sure. I mean, I wasn't thinking of open source projects at all. I'll just like lay that out. That is a totally different world that I do not, like, if you are doing it for. Yourself basically. And to put something into the world, I would think that that completely changes the math of when you're gonna be able to come back. Um, yeah. So I do think that's, yeah, I do think that's a big part of it is that my feeling was you probably aren't going to be given the chance to go back. An exception I would make to this is when. Every so often there is something like, like in iOS world right now with SwiftUI. SwiftUI is giving people a lot of opportunity to rewrite their UI because they want to use the new framework. And so that, that is the chance where I think a lot of people have been able to basically, uh, get rid of a lot of technical debt from an app they've had for several years and fix it. Like Marco's talked about this a lot on ATP. Um, other people have, so like that's an example. Um,
Joel Drapper:How would, how would you describe technical debt to begin with? So I think that's a tricky. Uh, definition to pin down.
Collin:that is a good, that is a good thing. I would say my definition of technical debt. Tell me if you have a different run. Tell me if you have a different one. Is, Something, something in your code that you know is not ideal and the result of it is it slows you, it creates some friction in the future, it's gonna slow you down a little bit. And so kind of like. Kind of like actual debt. If you are spending so much time maintaining your debt to like, you know, just to keep up with the interest payments or whatever, um, that lessens the amount of time you can invest in doing things other than just paying off your debt. Um, and at a certain point you're not even able to make payments on the principle, you know what I mean? You're just mostly maintaining the debt. Um, Does that make sense?
Joel Drapper:Yeah, I think that makes sense. Um, it's like trying to go with the analogy of this is something that gives you a short term boost. Maybe, like, maybe it lets you get that feature done, um, similar to debt, like let's you buy that thing you need, but. You're gonna pay more for it in the long run than if you hadn't done it. Like, if you had saved up instead of like taking debt to buy this thing, then you would've paid less. Whereas if you, if you take on the debt, then, then in the long run, because you're gonna have to fix that and pay that down eventually anyway, then over time you will pay more. It's just that you, you're getting the advantage of you get the thing earlier. Then if you did it the slower way, I guess.
Collin:Yeah, I think that's exactly correct. Like, like if you buy a house, right? And you're paying, uh, interest on like a 30 year loan. By the time you get to the end of that loan, um, you've basically bought another house.
Joel Drapper:right. You've bought like two houses.
Collin:Yeah. And that's how you end up with the math, that if you paid like a hundred dollars, or sorry, a hundred quid more on your mortgage payment every month, um, that you, uh, you know, that you end up paying it off like years or earlier or whatever because of the way interest works. Yeah. So I, I think that's, I think that's a pretty good summation of that.
Joel Drapper:I think with that house analogy though, it's um, it's kind of leads into technical, like talking about good debt and bad debt for when it comes to technical debt because, um, for a lot of people buying a house, Is a way to save money, uh, compared with paying rent. Like if you have an opportunity to buy a house, um, you can actually pay less than like equivalent rent. Or if you already have a house and you're buying other property and you can rent it out to others, then you can actually make money on that house that you are, that, that you have debt for. And the same, I think, can be true of technical debt. Like sometimes it is totally reasonable. Because the feature that you are building is like, so important to your company. Um, it's okay to take on debt. Um, and like it can, it can be, what I'm trying to say is it can be, it can be an asset, if that makes sense. Like technical debt isn't just to buy like liabilities that are fun, but like don't have any value. Usually you're taking on debt for value. Um, and I think that's a, that's a really. Good, uh, aspect to keep in mind is like, what value am I getting from this and is it worth it? Um, and then also like, what is the interest rate? Like, is this a really expensive piece of debt to carry or is it something that like, yeah, it's debt, but like it's actually not gonna cost that much to carry it for like a couple of years. Um,
Collin:Yeah, you really took that analogy and ran with it.
Joel Drapper:I think
Collin:but I, I think it's good
Joel Drapper:but it all, I think it really all applies.
Collin:I I do too. And what that made me think when you were talking about like, what is the interest on this, right? Is if a piece of debt, I think there are different categories and there are different impacts it can have. Right? So in my mind, The worst kind of debt would be like architectural debt. Something where it's like we really went all in on this capital A architecture that we read in a book or on a blog post, and it did not really pan out the way we hoped. And it actually just kind of makes working on this thing suck a little bit and it's going to be really hard to. Changed this because we based our entire thing on it. Um, that would be, in my mind, that's kind of the worst sort of debt. If you did something, like you had one component, like maybe you were making a control or, you know, You didn't quite, uh, you know, structure this one class perfectly. Ideally that, but it, you know, you got the feature out and it's very contained. That's what I mean, like contained debt that is in one thing that doesn't really affect anything else. I would say the interest rate on that is very low, comparatively.
Joel Drapper:Yeah, definitely. I think also going even further with this analogy, we can, we can talk about all code as being, uh, a liability in the sense that it is an asset that you have to maintain and that you could. Like it, it all code can go bad. Like it can, it can be a, a vector for bugs to be introduced. It's something that has to be maintained whenever you make a change. If you've got more code, there's more code you have to update. Um, I, and it's difficult to kind of distinguish between what's debt and what's not debt. Um, I think maybe it's not necessarily always helpful to distinguish that and actually just think about all your code as being. In a sense, like it has a cost. And so it's a case of like trying to be careful about like what costs you are willing to take on, what costs are worth it. Um,
Collin:I mean, I think this is why you'll hear a lot of really good engineers talk about how they're proud that when they look at like their GitHub, uh, you know, contributions or whatever on the whole, they like removed more code than they added to a project
Joel Drapper:Yeah.
Collin:because really, like, yeah, all code is something, you know, have to carry with you and. The best code is less code, right? Or whatever. I didn't mean to, uh, iterate that, but the best code is the code. You don't have to write or whatever. And to me, that's a lot of being able to, uh, like when a framework, its new capabilities that can oftentimes be a good chance to just get rid of code because now it does something that you previously had to have a workaround for or do yourself, you know, like for example, if you had. Uh, your own way to do sort of like what Turbo Dreams does. Maybe, you know, you had something where you were going to be able to update the page in a similar way, and then they come out with a first party thing. You can just get rid of that code and it's very well tested and they've done it for you, and you don't really have to think about this again.
Joel Drapper:right? There's less that you now have to maintain going forward and the. Instead of your custom, like probably not very well tested because it's only your company that's using it. Probably not very well documented, like custom solution you can sometimes fall back to, uh, like just go up a level to the, the framework. Um, a when there's something that's either solved in a framework or in a library that is kind of compatible with your framework.
Collin:Yeah. That sort of touches on one of my, I sort of had some like, okay, I said a bunch of things that aren't super actionable. What should you take away from this at the end of the post? Cuz I, I felt as though I needed to do that, or else, this is kind of a downer. Um, and one of them was basically to me, what I've seen a lot, especially in like iOS and macOS projects, um, and maybe you've seen this in other contexts, is I. I would absolutely avoid heavily abstracting away the API of whatever framework you're writing on top of as much as possible. Because like I, I did a contracting thing like earlier in the year where they had basically, uh, you know, there's whatever kit on iOS, right? And then they had our company kit and it essentially was an abstraction built on top of all these other abstractions and. What that means is one is when you bring somebody new in, they have no idea what any of that is. You need this crazy onboarding process.
Joel Drapper:Yeah.
Collin:Um, but two is that when new things are added to the framework, you don't kind of by default get those and getting too the actual new features in the framework, uh, until, you know, unless you've explicitly updated your abstraction. Is more diff it just makes everything more difficult in the long run to me. And so making big decisions like that, um, are the kind I would really try and avoid if I don't, if I want my code to be, uh, like ma manageable, just like tractable in any way.
Joel Drapper:I completely agree with you. I, I read this advice in a book once, I can't remember what book it was, that you should wrap any API that you interact with in your own wrapper so that you ca, like your application code interacts with that wrapper only. And if that API changes, you have one place to fix it in this like wrapper. Abstraction. And I think it's a terrible, terrible piece of advice you should never do that with. When it comes to like a Ruby API and a Ruby app, you should absolutely not do that. If we're talking about a JSON API on some web service, then of course you need to have some, uh, kind of ruby, um, objects and modules that kind of represent that J S O N API as Ruby. Um, and I think that makes sense, but. But there's, there's no reason to do that for Ruby Library.
Collin:Yeah, I mean, I definitely think there are degrees of that, right? Um, but yeah, like if you were trying to say, I don't know, build your own thing, build your own abstractions on top of ActiveRecord or something, uh, that would be sort of nutty.
Joel Drapper:I'll, I'll give you a, an example from this week. Um, I was working on, uh, changing the way that Sidekiq, uh, stores the parameters for jobs, uh, and looking into using message pack. And in particular, I wanted to use Paquito, which is a gem from Shopify.
Collin:Mm-hmm. And Paquito provides this quite low level API for creating these, um, packer, codex and, um, you know, for the, for the purposes of how we were using it everywhere in the application. We kind of always wanted it to use this specific codex and, um, to have these like special rules around it. So for example, if you have, um, uh, an active record object, then we would want to store just the global ID and then get back to the active record object from like fresh, uh, when, uh, loading it. Um, anyway, so, um, We also, we also, because we're storing this in json, had to like base 64, encode and decode the, the kind of bites that, um, Paquito would output. Um, so for, in that situation, I think it made sense to write a very lightweight wrapper around Paquito. Um, but that's like where I am layering on some functionality that didn't exist before. I think a lot of the time, um, Yeah, I think you're right. It depends. But if a library is being designed to be used by a user directly in your, in your code, um, like use that library. Don't try to like add too much indirection. Yeah, I agree. And I, in general, I feel like the biggest, I think programmers like making systems because it's fun, right? We like coming up with little ways that things communicate, and I think it can be, I think oftentimes people do that before they fully understand like the problem they're solving and if that makes sense. And so you end up with all this indirection, uh, in which maybe had a good goal in mind when you started, but the end result is now you wanted to make two things talk, but there's three things in the middle that you have to go through.
Joel Drapper:Yeah, this is like this. When people take the single responsibility principle too far and you end up with, um, like having to make one, you're trying to make one change and you have to traverse dozens of files to find like three or four lines of code that need to be changed because it's just so spread out. Um, and actually, like, maybe it would've been better to have one thing handle a few related responsibilities.
Collin:Mm-hmm.
Joel Drapper:Um, but in a way that's kind of like cohesive and, um, makes sense for the kinds of changes and maintenance that you're gonna be doing going forwards. Uh, so I think, I think that res that principle can kind of be taken, uh, way too far. I think maybe there, it has a point that, you know, classes shouldn't be, you know, they shouldn't be doing too many different things. Um, but it sure is okay for a class to be, um, Handling a few related, but different, different things, I think.
Collin:Yeah. And I think, so we both read this book, uh, by Dave Copeland, right? You turned me onto it. Uh, called Solid is Not Solid, and I think we're both kind of hinting at, um, that we wanna have him on at some point. But, uh,
Joel Drapper-1:Yeah. It'd be great to get him on
Collin:just speaking of this, he, I, I want to give him credit for this being the point. Cause I don't wanna just say I made it up. Um, but he really goes in on this being about that instead of thinking in terms of. Like following a rule, maybe look at your code and be like, is this cohesive? Like does this, does this like make sense what it's doing and trying? I think any time you are trying to take a rule, like you should always wrap every you use, you should always. Do this or whatever. Right. Um, it's, you need to make yourself dry. You need to do whatever. I think anytime you're trying to apply something as a rule, without any, without context, that's probably not the way to go.
Joel Drapper:Yeah, definitely. Speaking of which, um, I wanna get into the kind of second point in your blog post a bit more. So, um, is it the second point, um, you were saying, you were saying like, you should try to delay as for as as possible. Um, and I think that that is kind of a different way of saying, uh, one of these things that Sandy Metz has been saying, um, which is duplication is better than the wrong abstraction. Um, And I've got some thoughts about that. I think, I think that this is a very well, uh, meaning, uh, phrase. Um, and the idea of like, don't try to over-engineer something until you really understand it, um, makes a lot of sense. And I think her point was like, It's okay to have a bit of duplication, to have a bit of like, maybe not dry code until you figure out like a nice way to do an abstraction. But I've definitely seen this principle be like, way over applied. Um, like to the extent of like engineering managers being like, you're not allowed to write an abstraction until you've shown 10, like, clear examples of duplication.
Collin:10 is a lot
Joel Drapper:Like, I literally was put on a pip for writing an abstraction without 10 examples of duplication. Uh, it's like it's, it's insane how far people can take this stuff. And it's like they, they've read the, the principle, like the heading, but haven't, don't really understand it and don't understand why it's important and why, like, why you would apply it and why you might not apply it. Like, Abstraction is one of those things that is so imp like it's so fundamental to programming. Um, like programming as we know it could not exist without abstraction. And like what I mean by abstraction is taking a concept that is like maybe two or three different things and wrapping them up as like one thing. One abstract thing. Um, so really ex simple example of this. Um, excuse me, sorry. Um, sorry, just had a cough. Um, yeah, so, uh, really simple example of this is like, if you think about like basic logic gates, Um, like we could, we could try to, um, write programs by just doing this kind of basic, rudimentary binary logic. But instead we start by defining this binary logic into logic gates, and then using logic gates to compose other logic gates, right? Like not and becomes nand. Um, And we build up and up on these layers of abstraction until we end up with something like Ruby that lets us write code using like thousands of OB layers of abstraction. Um, in ways that's like just really easy for us to kind of read and understand quite quickly, I think for a lot of people. Um, Anyway, my point is that abstraction is really, really good and really powerful. I think it's one of the most powerful tools that we have. The idea that we can take a complex concept and compress it into a smaller concept. Um, you know, either by, you know, by using one of many techniques, like maybe extracting a class or extracting a private method, um, or like taking a method that did three things. And, um, making those three things like named functions, um, there are all kinds of ways that this can play out. Um, but like it's incredibly important tool and I think, I don't know, I just think that it can be like, people can take this concept, um, or this, this principle and just really overdo it. If, if an abstraction is right for. The situation that you're in right now, and it's not trying to do more than that, then it's okay. Like it's absolutely okay to have that abstraction. Um, but you need to tie this with a, a healthy view of like deleting code, an undoing code that's already there. Like if you wrote an abstraction one way. Then you realize that's not quite right, then you need to be comfortable to go in there and completely rip that out and delete it and do it a different way. Like the sunk cost fallacy kind of comes in here where like you feel like, oh, I spent a lot of time doing this, so therefore, or, or even worse in like a company with lots of developers, someone else spent a lot of time doing this. I don't feel like I can, you know, step on their toes and like change it or whatever. Um, you just can't ever feel like that. Um, It's absolutely fine to say this thing used to work, but we've got this one new requirement, which now means it's just doesn't, it's not, it doesn't make sense and it's fine to change it now.
Collin:Yeah. So that, that's a lot. Um,
Joel Drapper:Sorry,
Collin:no, that's great. I'm just trying to think the idea of being like I. So speaking of technical debt, I would say the idea of you need to demonstrate that we're going to, that manager sounds like an asshole. I'm sorry. The idea that you, uh, you need to have repeated this exact thing 10 times and demonstrate that so that you can then go back and create an abstraction. Like that kind of sounds like what part of my blog post was about,
Joel Drapper:Yeah.
Collin:now you, you did 10 things in parts of your code. And I guess the idea is that you are going to now have a time to go back and unwind all of those, uh, that, that seems optimistic and also bad. Um, I would oftentimes, like I wouldn't put a number to it. Like, like you said, if you got a little bit wrong.
Joel Drapper:to it, like
Collin:yeah.
Joel Drapper:Yeah. The, the, the, the specific example was we were writing, um, like dozens of jobs that would. Basically, basically they were kind of cascaded jobs. One job would call another job, which would call another job. Um, and they would be responsible for, um, basically like cleaning up, deleting, um, private information from database rows, database tables. Um, and so instead of have all of these like disconnected jobs that didn't really follow any kind of a pattern, um, I explored a way to. Define a schema where you could say like, these are my tables. These are the fields that have private information on them. Um, and like, these are the ways that these tables connect. And then a special job would like go through and know how to reverse that schema and figure out how to redact each field. And it would handle things like what if the field, um, You know, was non nullable, right? So it had to be replaced with some unique, or like, it was unique and non nullable, right? So be replaced with a, like a unique replacement value or, um, you know, that that kind of thing would be kind of baked in. And it would, it would also mean that you could never end up with like orphaned data where you could say this table has some data on it that's personally identifiable and belongs to. This, um, you know, this data subject, um, maybe it's a customer or something. Um, if there was no way to like connect back to the customer's table, um, to that table, then it would complain to you because it, it like knows it's able to work through this schema. So, so there, like the idea that I had was like, if we do this thing, we are kind of defining an abstraction. Um, yes, like we could do the duplication until we, until we find these edge cases, but the edge cases are known upfront, like we can solve for them right now. Um, what I want to do is like, capture the essence of the problem that we're solving and try to use that to like, find any, any of these bugs, um, and actually make the whole process much more. Like accurate and safe, um, and also just easier to work with. Um, but, but yeah. The, the, the point I'm trying to make about this is you could know that all upfront just by looking at the requirements. You do not need to actually go and create 20 different jobs and then be like, oh, well these jobs are kind of similar, so maybe we should like, Make a special super jobb class, uh, that they could only inherit from, that kind of removes some of the duplication that, again, wouldn't be the right solution because like it's, it's still not like getting to the root thing, which is I need to capture the essence of the data schema somewhere so that that can be the source of truth. Um, but yeah, so I think, I think that. Abstraction is incredibly important and valuable skill, and you can totally do it upfront. Like if you look at the requirements and you can see that's what we need, then I think it's okay to do it upfront.
Collin:I think. Yeah. I think in the case of we need to do 10 or 20 of this, and then we're going to somehow unwind and back out and then apply this new thing to it. When it was fairly clear what you needed to do after the first, like three times, um, feels, that feels like you're creating more work for yourself. I don't, I don't think you need to do it 10 times. I think I. If it's very apparent to me the first time, then yeah, I'll just do what I'm gonna do, right? Because I can always change it. Um, I'm not gonna make something so crazy complicated. I can't change it. But, you know, if it's not that, then it's usually once I've, I guess what I mean is this, I think the point of the delay and the. You know, duplication versus getting the wrong abstraction is that you want it to duplicate enough until you feel like you have the confidence to like see what the, like the abstraction comes out of that. Like you can see it and it's not there, and now you create it. It's not a rule that you're like, we did it this many times. Like it doesn't, it's not a number thing, you know what I mean?
Joel Drapper:yeah. It's
Collin:you can see it.
Joel Drapper:Do it as many times as it takes until you see the abstraction that you need,
Collin:Yes,
Joel Drapper:Right. And that may be that you can just do that upfront. It may be that. You can't see it upfront, and after a few examples you suddenly realize, well, actually all these jobs are repeating this thing, and maybe I can make that thing common between them. Um, but yeah.
Collin:I think what that advice is trying to. Get you to. Um, and also I, I know like Sandy Metz has like a list of rules or whatever, and I think I even heard her be like, I don't know, like they're rules I guess, but like, don't take them that seriously. You know what I mean?
Joel Drapper:Sure.
Collin:Yeah. they're not laws, um, But I think what that kind of advice would be trying to get you away from is the same thing that I think I was kind of warning about, which is the idea that you come into a new thing that you don't fully understand yet, and then you say, you know, you put your hands together and you, you start rubbing your hands together and you're like, all right, let's make some crazy system now. Right. And I think people would think, could think that that is the same thing as. Creating abstractions, but I think abstractions to me are ideally about like discovering the things that are already there.
Joel Drapper:Mm-hmm.
Collin:And it doesn't, the number isn't what's important. It's about the process of discovery. And you might do that, that might be apparent immediately, or it might be apparent after 10 times, but you can't, there's no rule to assign there. And trying to make that a rule seems extremely silly to me.
Joel Drapper:Yeah, there's, there's, there's also, and I don't really know how to describe this, but there's the kinds of abstractions that you can make by looking at duplication and trying to remove that duplication. They're good, they're good abstractions usually, and you should do them if you see them. But they are never gonna get you all the way, I don't think. Um, and I think that this project I mentioned is a perfect example of that. If we had made like the best abstract job removed, all the, the duplication inside our jobs for doing this, um, like personally identifiable data removal, we would still not have a way of checking that. Um, all of the, all of the private fields are covered because there's no schema that can tell us. Actually, there's a way from this entry point table, the customer's table all the way down through the orders table, through the, uh, line items table, et cetera, all the way down to this table where we know there's p i i That is, so we absolutely know that the job is gonna make it all that way. Be able to delete these relevant fields and successfully delete them in a way that, you know, that uses batches and, um, you know, efficient database queries and all that stuff. Like we could just never have got there. The, the only way to, to like have that assurance is to start with, how do I describe all of the relevant information? We need a schema. Then how do I like use programming, meta programming, whatever it takes to go from that schema into, um, ensuring that whatever jobs we run, like do that thing. In this case, the jobs could be completely automated based on the schema. In another case, maybe you'd have to write the jobs manually, but at least the schema could make sure that you wrote all the right jobs, um, and you didn't miss anything. And I think that, um, is just a, it's a different level of thinking. Um, the idea that you can only get an abstraction from duplication is, I think, just really silly.
Joel Drapper-1:Um,
Joel Drapper:and, and like, it's also, but it's understandable that, that people might read this principle and think, well, okay, so abstraction is bad. I should avoid meta programming, I should avoid doing abstractions too early. And, and you can go too far. I think. Um, you can definitely go too far in the other way. Um, And that's why I like, I like that you, you advise, you know, delay, delay, delay. Like wait until, you know, um, but Yeah.
Collin:Yeah, I, so, yeah, I'm just thinking through all this and you're describing it to me. I actually think that the Sandy Mets advice is pretty good advice, which sounds to me as though or many people have wildly misinterpreted what that's saying. It's not saying you can only create abstractions through duplication that. Is absolutely not like that. That seems so backwards to me. Um, which it feels like that's what your manager was almost saying,
Joel Drapper:yeah. This is basically what they believed.
Collin:like, you can, like new abstractions can only come through duplication. That's not right. What she, I believe is saying is, if you aren't sure yet, and the choice is between duplicate this and then think about it more or do something that like doesn't really feel right, but you feel like you need to, cuz you need to make it super dry or whatever. Uh, don't do that.
Joel Drapper:sure.
Collin:And I think that's pretty good advice that sounds like has room for people to misinterpret it if they don't think very deeply about it.
Joel Drapper:Sure. Yeah. I, I agree. It's, it's good advice,
Collin:Yeah, I would, I think of all of the, I think I, I've become more anti, uh, thought leader I guess in the last little while. Um, I think Sandy Metz is a pretty good one though. Uh, and that I think most of her advice is like pretty practical. But if you try and apply it as like, if you try and apply anything someone else said without, like, as rules, without taking your own like, uh, you know, taste into it is, um, Your own judgment. You have to have good judgment. And I think to me, a big difference between like a more senior programmer and a more junior programmer is a more junior programmer, more mid-level programmer is someone who's interested in following the right rules but doesn't really have a lot of good judgment yet. And as you get more senior, you have a little bit more judgment and you understand what some of these things we're getting at. Would, I would give the Sandy Met stuff, the benefit of that kind of doubt, like the Uncle Bob and the, like, all that stuff. I would actually not, I just think that stuff is crazy. Um, but, uh, that's, that's kinda where I'm at.
Joel Drapper:I think you can read all these programmer books and. Instead of, instead of thinking like, whatever this person says is a rule that I must follow, you can just use it as an example of code that you remember, like, and like, you know, like, here's, um, here's an idea. Here's a problem, here's a way that it can go badly, and here's another way of approaching it that might be better in some situations. Right? That's, that's basically what all of these things are about. And just expose yourself to more of those and develop your own taste, develop your own judgment. Um, and I know that can be difficult at the beginning, but that's, that's ultimately how you're gonna get the most value out of these rules and out of these principles. And these books is to, is to just like, don't take them too seriously, but keep them in mind, I guess. Like you can u yeah, you can use them to build your own palette, build your own, um, opinions. Um, but,
Collin:Yeah. And I.
Joel Drapper:all of them have holes like, and exceptions and, and, and many of them contradict each other.
Collin:Well, it's like, it's like if you read a self-help book, right? Like, uh, that was written by that person for a very broad audience, um, that's not about you specifically. Uh, yeah, I think, I think exposing yourself to a lot is, is good. And I think also being aware that a lot of. These things you hear are coming from a person specific experience at a specific time. Um,
Joel Drapper:Mm-hmm.
Collin:like I would say like when you read the Sandy Met stuff, right? Like she's a small talk programmer and so the ruby she sort of comes up with is very like small talkie ruby. You know what I mean? Which is fine. But it's like that is where she comes from and where some of these other guys might have been writing c plus plus in the nineties, and maybe these principles don't exactly apply in the same way. And maybe trying to turn things into rules instead of just like here is a pallet of things I. A pal, a toolbox that I can sort of think about as I am, as I am doing my typing every day. Um, you know, also, like most of us aren't writing software for the space shuttle. Like you can experiment in a little bit, right? Like it's not gonna explode on the launchpad.
Joel Drapper:so. I think you can experiment a bit and.
Collin:Yeah,
Joel Drapper:Yeah, just like expose yourself to lots of different ideas as well. Like don't just read Sandy Metson object oriented. Also read people on functional and um, you know, like programming with types, right? Ruby has types and the ideas that you get from thinking about types in a type safe language are totally applicable to Ruby, like, And, and, and it helps you understand, I think like ways that you can break those rules as well. Like, like Ruby's very flexible, but knowing when you're doing something that's a bit weird or whatever and, and like knowing that, yeah, this is okay because this is an advantage I get with Ruby, but also I understand what I'm doing. Um,
Collin:Yeah, I mean
Joel Drapper:I.
Collin:it's an, yeah, or it's an like with type stuff, like it's an advantage, it's also a trade off, you know, that Ruby's very loose with types. Um, and that you can take advantage of that. Yeah, I think that makes sense.
Joel Drapper:Yeah, my, my bigger point is like, there are so many different ways of thinking about things, and also none of them apply to your code exactly as it is either. So your best bet is like, expose yourself to loads of different ideas. Um, But yeah. Anyway, your, we've kind of digressed a long way from,
Collin:Yeah, I think that's all great though. I think it's gonna be a really good episode. I'm, uh, We covered a lot. Um, I think it's a good place to wrap it up cuz I, you've got me thinking about other stuff now and I feel it's gonna be a two hour long episode if we, if we keep going. Um, so, uh,
Joel Drapper:I'm looking forward to reading your next one by the way.
Collin:Yeah. You know what, um, do you want me to give you a, i'll, I'll give you a sneak peek. Is that I, so I wrote this MVC post and then I was like, maybe we'll talk about this next week a little bit. Um, we have to talk about it now, but I wrote this MVC post and then I was like, I, okay, I understand the concepts of these different styles of this pattern. How did they actually implement this? Like originally, like in small talk in the eighties, like what did that actually look like so I could understand it more concretely? And so I got back on, uh, what, how I would refer to it as being back on my bullshit of, uh, reading technical documents about small talk from the 1980s. Um, and so I. I may write a post about that this week and, uh, maybe we can talk about that. But in the meantime, uh, if you like the show, please tell your friends, subscribe. If you aren't subscribed, subscribe your friends. Um, leave us a review, I guess. Uh, and we will see you next week.
