Nailing Down the Right Abstractions - Tim Neutkens (Vercel)

How to nail down the right abstractions for optimal DX? Building abstractions is at the core of API design. They become the face of your software, and they dictate a significant part of its developer experience. How to design the right abstractions? What is the difference between an opinionated software and a failing abstraction?

To answer these difficult questions, we're happy to welcome two remarkable guests:

Tim Neutkens is the Lead Engineer of Next.js, one if not the most popular and versatile React framework. He also co-authored MDX, an authoring format that lets you mix Markdown and JSX.

François Chalifour is a software engineer at Algolia. He's been working on Algolia Instantsearch for several years and co-authored our Recommend UI library. He also completely redesigned Algolia's Autocomplete library from scratch after over a year worth of work and research.

• Tim Neutkens: @timneutkens 
• François Chalifour: @francoischlfr
• Sarah Dayan: @frontstuff_io
• Algolia: @algolia / algolia.com
• Vercel: @vercel / vercel.com

Transcript

Tim Neutkens: 0:00

Good DX also has to like weigh against, uh, having good user experience. So a lot of things that we do in next is also like running you for cases where, uh, you're going to affect user experience by including synchronous scripts, or you're going to include an image stack that blocks rendering of the whole page or, uh, that kinda stuff. Like in next 11, we, we have a lot of goal system of plugins that basically warn you like, okay, this is going to be a lot better if you make it, uh, like change it to, to use this other approach. And it goes for like phone preloading all that. And then like, from, from our perspective, we also try to get out of the way very often. So there are some conventions, but like, we're not telling you how to do data fetching, uh, particularly like where you're getting data from, how you're getting the data, because then you, you get the full flexibility of being able to, to build what you want. And that is also led to basically like seeing next sites from like a few pages, personal websites to like massive web applications that, that are served through to millions of people, uh, every day.

Sarah Dayan: 1:15

Hi everyone. And welcome to developer experience a podcast by Algolia. We, we chat with guests who build products for developers about their developer experience strategy, what it means for them, why it's important. And so on, my name is Saha. And on today's episode, we are going to talk about how to nail down the right abstractions. Building abstraction is at the core of API design. They become the face of your software and they dictate a significant part of its developer experience, how to design the right abstractions. What is the difference between opinionated software and a failing abstraction to answer those difficult questions? I have two remarkable guests with me today. Tim Neutkens is the lead engineer of N JS one, if not the most popular and versatile react based framework. He also co-authored MDX an authoring format that will its U mix markdown and JSX team is responsible for many of the awesome patterns we love in next JS and largely contributed to its skyrocketing popularity. Hey Tim.

Tim Neutkens: 2:38

Hey, thanks for having me.

Sarah Dayan: 2:39

Our second guest is my homeboy, François Chalifour. François is a software engineer at Algolia. He's been working on Algolia instant search for several years, and co-author our recommend UI library. He also completely designed Algolia's Autocomplete library from scratch after over a year worth of work and research. Hi,

François Chalifour: 3:04

Hey Sarah.

Sarah Dayan: 3:05

All right. So let's get started. Building abstractions is difficult. I think we can all agree on that premise under abstracting kind of misses the point, but over abstracting can make your abstraction and unusable and force users to work against it. And when I look at next JS, it's really praised for how versatile it is, and it has many examples of really great abstractions. For example, uh, get static props is a really flexible way to pass data to a page, whatever the data, wherever it from. When I look in auto complete, you have things like get sources or the reshape API that open many possibilities for mixing all kinds of suggestions. So my first question, uh, for you both would be how do you approach building abstractions that solve problems the right will way, but also stand a test of time.

Tim Neutkens: 4:05

So in next shift, we have pretty large premise in, in terms of like what the framework itself can do. So basically allows you to build complete websites, such web apps and go from like really small, to really large like applications and say like the, the way that we build things is it's always been around like some of the base principles that we've had since the very beginning, for example, creating a pages directory that's met to route. And then that basically allows you to build on top of this core set of features that we built and enhance stem over time. So an example that you gave with get set props, for example, that didn't exist when we, uh, introduced next. What we did instead was we created a like API called get initial props and we started, uh, using that you basically had initial server request get, get initial props will be called. Uh, then when you, uh, like then the, the content that is generated out of that is sent browser. And then when you start navigating around it calls get initial props on the, in the browser as well. So, uh, in order to reason about that, you basically have to think about, okay, this code is going to run in a, like no GS environment. And then after that, every single page change is going to run in your browser. That actually caused a lot of like confusion and issues. When next year it's got more popular, cuz basically you'd have to, to think about like, okay, it's going to run here. Then it's going to run here. And the environments are different code can break in different, uh, cases. And then like, how do I think about like this rendering then basically this process from like going from getting props to get static props to over like a year, year and a half or so of collecting feedback and making sure that like kneel down that abstraction in terms of like making sure that you could do still do the same things that you wanted to do, uh, in the first place of get initial props, but also give you the slightly more flex as well. Uh, so like what we landed on there is that get static props actually always runs on a notice environment. Um, so, uh, what that means is that if you like do a request to the page, it's going to like all aesthetic props, uh, at least in develop in, in, uh, production build, it's going to be aesthetically generate it. Um, that's like a whole different thing to talk about as well. But, um, basically, uh, then when you start navigating around, it's still going to do a fetch to the server to then get the props back. Uh, and uh, the reason that we did this is that it basically optimizes two things. Like the one is we saw that we we've got initial props, you have to do your own cashing. So that means that if you forget to like add any type of cashing into your get initial props, it would be slower. Um, and then the other is that the code is all running in the same environment. So you only have to think about reason about one specific case. And then, uh, the interesting thing will way of aesthetic props as well is that, uh, we did some more optimizations there that you're not aware of when you're using it, but later on are going to be, uh, massively important. Like when we introduced it, it was like, okay, this is a slight friction in so far that like, for example, you couldn't get access to request response like we did for, uh, get source props and get initial props. But the main reason behind that is that you can then start generating things at build time, for example, uh, or like izing them across multiple workers or making sure that you can run in different environments as well. Cause you don't get like variables that, that are very specific to one environment. Yeah. Like that is where, like that's one example of like how we like change API surface while also adding in more features as well.

Sarah Dayan: 8:08

I'd like to go maybe, uh, maybe upper level when we talk about get static props, because one of the big things for me and like why I think this is really from, you know, a user perspective, why it's a really interesting abstraction is because how simple and powerful it is. And it looks kind of obvious. It's like you get a function that you name a certain way and you export it with a certain name so that the framework recognizes it. And it's just going to pass props to your component, which is your page and the way you fetch that data completely belongs to you. And it's a, it's a different paradigm than, uh, not to say it's, uh, it's less, less good. But like when you look at something like Gadsby, which is a similar product to next JS in terms of what it you can do with it, um, it's different in Gadsby, you would have to, to plug your source to, um, to graph QL. And then you would source from basically a centralized data store, uh, in X JS, you don't have this concept of centralized data store, uh, whatever the page is. It is kind of a contain environment. I think even every page run when you build every page, it runs in a single worker or threat. So you don't have access to everything else, but most of the time it's fine. And when you look at it at the end of the day, you're like, yeah, it's awesome. It works. Uh, of course they built it like that, but what's the thought process, uh, when you building the abstraction, what's the, the thought process of, yeah, this is how people want to fetch data. Some people will have files locally. Some people will want to maybe fetch some something on the GitHub API and cross reference it with some something else. So how do you approach designing such a core part of different mark?

Tim Neutkens: 9:57

Yeah, so like when we started out, uh, building XGS, there was already this constraint of like, we need to fetch data from somewhere else. Like it's not going to be inside of the X app itself per se. And then like the first version of that was basically we need to have some kind of, uh, way to, to colon asynchronous function. And this is like when ay weight was like just a thing, uh, that, that people start using. Like we had to add in like async to generate or transforms that kind of stuff in order to make it work. But basically like eventually next didn't start out as a react framework. Actually it started out as a framework to build, uh, applications using its own template language as well. Uh, and then like later on, uh, before it was open sourced, uh, it was changed to use react. And then like we added some sprinkles on top of react, uh, at that point it's this is when like function components, weren't really a thing. It was like everything was glass components and that actually allowed us to use the static keyword to get like static, get, uh, get into props. Uh, so like on the component itself, you would put your, uh, like data fetching basically. And then what we said is basically like, okay, this is an asynchronous function it's going to block like, like before rendering, we basically, and this function call like being called and you as a user can get data from anywhere. So that could be from the file system or from a, an external API. And then in general, like what we saw in the beginning is that most of the apps that were built with next were built on external APIs. Uh, so this ended up being like, uh, or like cell API, separate repo, lots of microservices were all being called from the front end, basically. Uh, and then you didn't have this like, basically like this abstraction on top of, okay. Like I need to map all the values from the API into like a GraphQL compatible document or anything like that, or even like the API being GraphQL, cuz it was all, uh, microservices didn't have a centralized, um, like API, uh, router basically. And this has all changed over time as well, uh, for sale. But like at the time that like that is where very, it all started. And then like over time we, we got more and more usage and more and more use cases as well of people like either wanting to do data fetching from an external URL. Like that is fairly simple. It's a synchronous function. You just fetch it from somewhere or use the S K that like a CMS provider gives you and then you, you get your data back and then it renders. So like, that was always the thing that we wanted to preserve over time. Like later on we basically like started figuring out like, okay, how can we, like we see so many nexus apps out there, like simpler sites that are doing cuz nexus doing server site rendering really well. So like basically what you've got was every single page that you would request would be server site rendered from the like specific, uh, location that the app was hosted in. And it was like, generally not serve from a CDN, but we realized that a lot of the pages that we saw and people were tweeting and, and all that, uh, or gave feedback or showed us, uh, stuff for showcase these pages didn't even have to get initial props at the time. So, uh, what we realized is like, okay, we can make this optimization, uh, sort of like an abstraction on top of like, like here theistic for how can I make this page fast without actually needing the user to change anything about their app? Like they can just upgrade and it will be faster. So we started out with just static generations, like static generation of pages that did not have get initial props, uh, which we called automatic static optimization. And it basically means that if you have a default next app and you don't use get initial props, it would just be completely static by default. Now this has some, some trade offs as well, like some APIs that had to change a little bit, uh, or like the behavior that they had, uh, cuz of the not having the, the indeed request information. It's like, for example, with RA, like with the rider, that would be an issue. Um, but over time we, we fixed all those as well. It basically means that like when people upgraded to that, that specific version, they then got a, uh, aesthetically rendered page that would previously be source that rendered even though they didn't have to make any changes to their app. Cuz we knew from the constraints that, uh, react gives us that uh, a tree has to render without any asynchronous, uh, data fetching and all that, uh, that we could render it statically based on like if you're not using actually specific APIs and like over time that evolved into get static props, get server site props as well, uh, to, to basically cover a lot more of these cases.

Sarah Dayan: 14:57

When, when I look at, uh, at autocomplete, one of the core parts of autocomplete is the get sources function, which allows you to return the sources for your autocomplete. And I cannot help, but think that there might be either inspiration at, it looks really similar to the concepts of get initial props. Ultimately it's a function that can be a synchronous and that returns data. How did you approach that when you design or to complete V1,

François Chalifour: 15:24

But to complete V1, was that a library that aims at powering such experiences, but the way that we saw such experiences at the time was the, that rich search experience actually combines multiple sources. So we, we didn't want to be opinionated in terms of where you fetch the data, uh, which kind of rings it bits, uh, like Tim was mentioning about next, this kind of concepts are similar, but on pitch is a, is a library developed at Algolia. So of course we, we use mainly the last search engine. Uh, this is where auto computable shines, but Al doesn't provide any features like recent searches or favorite searches or of stuff like that because there is no value in Algolia, uh, actually, um, creating these kind of features. But we as a front end team, uh, realize that actually rich experiences, which search experie is must combine these kind of different sources. So this is where, uh, the get sources, uh, option actually came from. Uh, but then there, there was another, uh, concern that we had, uh, which was how do we actually transform the data that the search engine genes and that the sources, uh, return as the thing with databases and the web in general is show database, uh, is not a strict representation of how your website looks like. And with search engines, we oftentimes reproduce what the data structure is, uh, on the back end on the front end. But this is not usually is a friender way of showing the, the data. And this is where we introduce another API on top of gets sources, which we call the reshape API that actually entered these kind of cases. And the, the very first reason why we needed some kind of APIs like that was to remove, duplicate between sources. So there was one use case that we had, but we didn't really want to introduce an API just for removing duplicates in the search results, because there was quite limited. I like to think of by, in terms of what we call overfiting in statistics and overfiting is basically the, the production of something that correspond too closely or exactly to a particular dataset. And, um, it leads to having basically not matching any future observation that you get, or it's, it's good ifs the data, the data that you're training it with to accurate this. So it's not gonna be extendable in any sense. Uh, so we wanted to avoid this overing problem when designing the API, uh, which is why we, um, actually use some, some kind of, uh, patterns like in version of control and stuff like that. She's also quite similar to, um, get static props and this kind of APIs in. So the goal was not to just allow users to remove duplicates across the associates, but to allow low users to create on functions that place some transformation, for instance, they now can group their sources. They can sort their sources and they can do whatever they want. They can also package the presets themselves. So the way that we design the API is really, we provide you a hook where you can plug your own logic, which we're not gonna create an option for each of the use case that you may have. Uh, this is basically what we thought about it.

Sarah Dayan: 18:33

One thing that I see in common in both ways of, of approaching obstructions is that is the simplicity and the attempt to stay as close as possible to the language or the underlying abstraction get sources is it's a function you can use whatever you want in it. Like you're not forced to use some other abstractions from the library. Uh, you don't have to learn things that the language already does. And I think it's also what contributes to the success of aesthetic props like once. Okay. I know I have to write a function that has that name and that's fairly easy to understand. I need to return it under a key called props, but that's about it. And then the rest is really up to me. So if I already understand the fetch API, if I already understand the file system API or whatever, um, this knowledge is not lost. I, I don't have to just dive into a bunch of documentation to simple things. So like this may be this tendency of trying to micro abstract and only abstract when it's necessary, but stop at the level where the software is going to understand it, but the user can be free to use what they know like X JS uses react and like they wanna use react the way they understand and they'll react. And if you add some something around it or some rapper around it, then maybe it'll not be retract compatible with the, the, the next versions, uh, of react. So it also contributes to the fact that they want to use next. So, so I think this is one of the things that I take away from it is that one way of designing abstractions that will spend a test of time is trying not to over abstract and trying not to make grand abstractions that kind of solve everything and hide everything, but rather find the little places where you need to have the glue and consider them as, as the glue and then recognize the power of other APIs and things that are baked in, in the language or the other abstractions that you use.

François Chalifour: 20:41

Yeah. We usually have a, a three layer process at when we design APIs, which is, uh, okay, can we actually leverage the APIs that we have right now and just write some guidance, some documentation about it. And sometimes this is a good way to actually over time, better understand if the feature is actually required. Do we need to build an abstraction for that? Uh, do users feel some friction when they, when they're actually for the, for the guide? Uh, this is the first step that we try. We try usually to stop there so that we don't have to build new features and new abstractions. Uh, then we try to build features as plugins that could be adds middleware presets and stuff like that, uh, because they are not parts of the API. So they are easier to change and to deprecate usually. And the very last layer is actually creating a car feature, uh, which is sometimes required, but sometimes it's not. And this is where we think that most of the users could benefit from it. And there is value in having, uh, in actually reporting that in the product.

Tim Neutkens: 21:47

Now, one of the, the interesting things with regards to next is that we actually don't have a plugin system per se. Um, so one of the, the que decisions that we made is that we give you this three low level, get static props that basically, uh, like solves the data fetching part and a don't get, um, like you don't need specific plugins per se, as like, okay, I'm going to hook into the whole system and then like have to back, like, make that back compatible as well. Uh, but instead you get this really simple, uh, like asy function contract where I can just import my, uh, like a Golia SDK, for example. And they use that to, to fetch data, um, and use that in my components then immediately, uh, as opposed to like having to install a plugin that then provides you with the SDK maybe in inside of the function or something like that is parameters. And like one of the, the interesting mistakes ish that we made with getting your props as well is that it's a function that just returns props. So that means you return an object and then that object is the, the thing that you can pass to, um, your component. But, uh, as you mentioned, uh, there, there is a, uh, like in tic props get server props. We actually return props an object that has the props key. Uh, this is very, uh, like a conscious decision that we made, which was okay. Uh, if we're going to introduce this new API, we need to make sure that we like can extend it as well, cuz because of Becker's compact. So like we're very, um, like keen making sure that you can just upgrade the latest version of next, uh, most of the time and have many tests to, to back that up. If we were to add a feature to get initial props, to return, for example, not fan page or something like that, we would start breaking, uh, existing apps, cuz those would be returning like not found with some other key that they they're using in their app. Um, so we've got set props, got server server props. You can actually now like you have to return props, uh, as the, the key in object, then next to it, we can actually return other things as well. It's like returning a redirect or returning, uh, not found, uh, that kind of stuff. Uh, and it basically allowed us to like over time evolve the feature set of get props, get server props as well, uh, where previously, like you would have to, for example, return not found as a property and then use our property to render, uh, for a four page in your page itself. Um, so like every page would be completely standalone. Like if you would render it with the props, it would always be the same thing. Um, and like over time we basically edit in these like sort of helper features where, uh, you can just return not found, it's going to do a lot of things automatically for you, uh, which is not possible way of getting props. For example,

Sarah Dayan: 24:52

Frameworks and, and libraries usually strive for flexibility. You know, we want to cover many use cases and we, we, we don't want to overfit as you said earlier, uh, but many successful solutions and definitely thinking of next JS in, uh, when I, when I'm talking about that have opinionated parts. So for example, next JS uses a file system based router. And if you want to programmatically generate pages, you can, but you will kind of have to exit the framework. You will have to, uh, like write your own your own custom server. And it's going to take a few things away from you. For example, you will not be able to natively deploy to Versace. So it's like, there is this opinion about the framework that says, this is how you should do it. We think this is how most people, if not everybody should do it because, uh, like that's that, that's something that, that we've vetted. That's a decision, that's a conscious decision that, that we have taken. Um, and, uh, I can see the kind of, kind of the same thing in auto complete JS. So auto complete JS is the, like the VDO implementation of, of auto complete in auto complete JS. You can completely customize the sources, the terms, but the library fully controls the dam and the state of the surge box. So you can style it the way you want, but the way it's gonna, it's gonna behave like showing the, the loader and, and all of that, all of that is really hidden from you. There is no specific API to access it. So that's an opinion, uh, for, for other complete JS is that, that thing, we think that we are better off if we control it ourselves and you focus on your, uh, on your sources and your items. So my second question would be what is for you, the difference between the wrong abstraction and an opinionated one, when do you consider that it is time to be opinionated and when should you be more flexible?

Tim Neutkens: 26:58

The main thing that we always opted for is that the app that you could build, like, like next, basically like the, the initial commit for next was a readme with the complete API dog for how going to work. Uh, there was no code involved. Like there was no like actual implementation there. Uh, the implementation actually came later, like basically went and wrote down, like all the things we wanted to solve basically. And that turned out to be like the first documentation page, cuz like for the longest time we had to read me being the, the full documentation and over time, like other feature for added and eventually that turned into the website to make sure that you could search it. And uh, and like it basically got way too large for a single page. Um, over like basically when, uh, when we started because of the debt core principle, like basically the initial, um, like with me still works today except for like some, uh, nuances way of styling, for example, like we switched, uh, styling in version two, um, because of the, like the initial version, not being the most ideal way to, to write CSS. Um, but over time, like that app that you would build with like pages, uh, and then like index suggests being mapped to the, the road route, uh, and then even get initial props, which was there from the start, um, that still works today. That was like one of the principles that we had like, okay, we're going to use this and evolve it over time. And then like incrementally improve it based on like feedback from users. Initially it was mostly driven by like internal needs Atell. So like, uh, and I'm talking end of 20 16, 20 17, 20 18, uh, like basically, uh, years ago now. Um, and, uh, that was mostly cuz we were the, the biggest user of next, uh, and then like over time other companies started using it, uh, especially in 2018, like we saw a lot of really large companies start using it with hundreds of developers. And that's when you, you start getting a lot of really detailed feedback on like, okay, like our, uh, our engineers are struggling with, uh, like this particular way of doing things or this API or like I, I keep using get initial props, but, uh, it breaks on the client side, writing is broken, that kind of stuff. And then like we started evolving that API service over time. Like basically like initially we opted for, and especially when, uh, when it started contributing to next, uh, we opted for like adding as many features as possible. Like anything people would ask, we would add it, um, uh, like customizing disc directories, like all kinds of stuff that wasn't super needed for the framework itself to function, uh, but did help some users. Uh, and during it, like that would be like at the time it would be like only a few people like in practice. Uh, and those features are still there. Like we, we maintained be with a lot of these, uh, these initial features that weren't as ideal as it could be. So like over time, like when, like we found that like there isn't better new of doing things or like making things easier or like evolving that we get the old API around, but still, uh, like highly recommend using some other APIs that are like journey easier to understand, easier to use. An example is like the end key in next config, um, that allow to basically create like global environment variables that you can access in clients that code cuz they're in line in the, the browser bundle. We evolve that to like, just have a, like, you can have a environment variable with a prefix called next, under public. And the score with that in front of your environment, variable, it's going to be, uh, inline into the browser bundle automatically. So then you don't have to add like next config items, uh, make sure that like those match up. And then even from the perspective of like you writing the code, it actually is a lot clearer that this is going to be inline into the browser bundle as well. But we still support the NP cuz like people are still using that as well for, for other cases. So like basically like over time, especially for next like things evolve. But the, uh, the nice thing is that they also like, in most cases, these conflict options don't add weight if you don't use them. Like what I mean by that is like, it's not going to increase browser bundle size cuz we actually tree shake away features that you Don use. Um, and stuff like that. So if you don't using international as a internationalized routing, it's not going to be added into the re end. Uh, if you're not using and it's not going to replace like all your environment variables for, for whatever reason. But then like also from the other perspective, we also give you full control over like the web. For example, at the time, for example, create react app, didn't allow you to do that. It allowed people to eject very quickly. People would basically start ejecting their app immediately when they needed to customize anything. We did allow that cuz uh, cuz some people asked for it and over time that that really helped the nexus community grow as well. Like there are people creating, uh, like customizations to next shares that relied on like compilations or like Webpac or like be config as well. And then like over time we started adding a lot of these customizations by default. So like if you want to import images, you can now import images, uh, using next image. Um, if you want to, um, do the same for SVGs that also works like set up record, like record threats or like web records that is, uh, supported by default in web now, uh, all those kinds of things that you would previously have to reach for, uh, custom ation you can now do by default or to conflict flag. That is basically like the, the interesting thing there is that it's actually really beneficial to not have custom configuration your app. Uh, for two reasons like the one is, uh, we know that your app is compatible, uh, cuz we have a massive test suite of integration tests. So we make sure that, uh, those all piles from every pool request, um, and the other is that, um, we can actually start doing optimizations that you would previously not be able to do, um, because you're using custom weapon config. Uh, so an example is if you had on web config, uh, you would be opted out of web five by default in order to, uh, make sure that we didn't break those apps by just like upgrading to the latest access version. Uh, then in major version we change that. So you can opt out of web five if you need to. Um, and then like eventually we can deprecate those options cuz people start naturally moving to the new, the newer thing, but in, in practice that, uh, like if you didn't have custom config, so if you didn't have custom weapon config, we would just opt you into epic five out of nowhere. Like you just upgrade next and you are on the latest, uh, version of WebEx, for example, you're leveraging, uh, discussing optimized, like usage. Like we created a new PLO that is much faster than the default one and added a bunch of other like cashing on top of that. And that would not be possible if you have custom config in all cases. So then we have to be really like careful in preserving, uh, backer's compatibility for, for those use use, uh, that use. And now like even looking to the future, like we're currently working on porting all the next year transforms. So like the things that we do in be a new risk based compiler called S WC and that's going to massively reduce bill times and development integration loop as well. Um, and in doing so, uh, like if you currently have custom be config, like we're going to opt you out and still use be for that. Uh, cuz obviously we don't want to break your existing app, but um, like the, the abstraction that we have of like, okay, we're going to compile it for you and you don't have to add configuration, uh, cuz like by the thought you don't really need it, uh, has really allowed us to evolve and make next, faster, over time. So like if you upgrade, um, like next year, uh, like 10 to 11 and you were using custom and it enables WebEx five is going to be much faster than, uh, than it was before you upgraded basically.

François Chalifour: 35:45

Yeah. It's interesting. And in uh, in the computer, as you mentioned, sir, we were open 80 about some, um, UX and UI perspectives, like, uh, the search we realized, um, along the girls that uh, when users starts, uh, creating their own saltbox it's most of the time, very floated in terms of accessibility and even news age. So our thinking process was really, um, if some people should be good about building search boxes, it should be us really. So we decided, uh, for instance, in doc R B3 and also in auto D one, you kind of shift, um, in terms of we are not gonna just open some kind of dropdown to, to your own search, but we are gonna, uh, create the search for you. Uh, and this allow us to build, uh, way more complex and uh, interactive interactions with the user also for mobile purposes, uh, search experiences and mobile are quite different than such experiences on the web. So really there was a, our thinking also, um, this is one topic that we should actually, uh, be opinion on it about, uh, but for the sources and for the search results, we cannot be too native because this is very, um, custom based on, uh, on the user and the, and the, the use case of the website that they're building really could be eCommerce. It could be measure, it could be any kind of, um, of, uh, of vascular, really

Sarah Dayan: 37:13

One thing you mentioned, uh, team, you, you mentioned the create react app and that it release, parked something for me because, uh, so last last episode we had, uh, we had the, um, Seck from and Aun talking about blending in other people could bases. And Aun mentioned that escape hatches are so important when you're building anything really whatever, and no disrespect to the, a create react app team. Like, uh, I use this for, for a long time, but, but one of the reasons that I stopped doing it and I actually moved on to, to next JS, which I use for most of my projects today is that for me, the line of opinions was maybe a little bit too far. And I enjoy when opinions are not like dictating what I should do without caring about what my specific problem is because at the end of the day, when you're, you're building an auto complete, which is already pretty focused, but when you get into it, you realize like the variety of use cases that you can tackle, uh, with a framework, a JavaScript framework, it it's probably even wider. People are gonna have different problems. And for me, when I was using create app, and I don't know if it's different today, but I wanted to use tailwind. And when you want to use tailwind, you need to have a custom post CSS file that that's it just because you want to use this framework. Uh, and that's not something that you at least back then were able to do with create react app without ejecting. And the problem that I had with this level of opinion is that it felt like this tool that was enabling me was suddenly telling me, okay, we we're letting you go. Uh, because you are going out of the road that he paved for you. And what I enjoy about, uh, a tool is when it tells me, okay, that's not necessarily the best way to do things. So this, you can escape that and it's, it's gonna be different. It's gonna be maybe a bit harder to do, but it's still possible. And that it understands to what level I will need to customize something. Uh, being able to have a, a custom post CSS file is definitely something that many people will want. Uh, maybe having programmatic grounds is not so common. And conversely, when I look at autocomplete, what I find interesting is that I can start with autocomplete JS, uh, and I'm going to be able to build a lot of things. And then let's say, I want to have my custom surge box because this is really something that I need. Like I need to be able to do something extremely custom. I can go to autocomplete core, but what's really interesting. So I, I will have this Ren less, uh, auto complete, uh, like library and then I can do whatever I want, but I will have learned from looking at how autocomplete JS is built so I can pick those principles. So it feels like I'm learning at the same time. Like you're learning the API and it's not like, well now you basically have a, a project that would be what you would've built if you had done it yourself. So I would say that's one of the limits, one of the lines that I draw between the right amount of opinion and too much opinion where it might look like, oh, this is like, this opinion is a substitute for a problem that you really don't want to solve

Tim Neutkens: 40:44

Because like the, the comparison with great react app, like great react up is, is great at what it does, which is get you started using react. And, uh, like you don't have to set up much and, and you can get started, um, in doing so it's super opinionated about, okay, this is how you're going to do the whole compilation part. That is, uh, obviously a trade off cuz like the, the main thing to consider here is that, um, like the example way of tailwind, uh, for example, is that tailwind is built on top of both CSS and post CSS is a tool in itself of, uh, if you want to like do the performance optimizations that we're currently doing, for example, like rewriting parts and rest, all the configuration that we give users is super, uh, like limiting what we can do. Like, uh, an example is if you want to, like, if you would want to move from bevel to rust, uh, like to S SOC inside of create react app, it would be fairly a bit simpler then, like, for example, in next, because of the custom. So because you are allowed to add a bevel or a bevel, ORRC into like an excess app, we have to think about like, okay, how are we going to make this vector compatible? And then for how long are we going to support bevel as the output target likely very long, cuz we're very committed to like Vic. So if you have that config, you, you then have to make, make sure that that stays up to date as well. Right? So like that you can like keep using it over time, even though you're introducing new features. The, the example I gave before, like web four to epic five post CSS, like what if we, uh, like rewrite, uh, like parts of Bo CSS in rest, um, or like replace it completely for like the, the basic case where you don't have custom config, those are all things that we're thinking about and would be interesting. Uh, obviously we're currently very focused on the JavaScript part of things, a trade off either way. Like if you would have a, if we were super strict, we didn't have custom weapon config, we didn't have custom be config. A lot of people, people would not be able to use next today. Cuz like the moment, like you said, the moment you need till wind or the moment you need start components, the moment you need emotion, uh, that kind of stuff, it all requires some custom config like the, for till it's, uh, custom posts, um, for start components it's custom, uh, be config. Like if you don't have that option to use that, it makes it really hard to, to start using these libraries. And it especially becomes apparent when you start doing source at rendering of pre rendering. Cause most of these tools have, uh, custom transforms to make sure that the using be for example, uh, like adding some extra metadata in order to make sure that hydration works in the browser. It's like, that is a, like an interesting case. But then also, uh, by allowing you to have this custom, it allows people to experiment as well. Right. If there wasn't, uh, like if PO CSS didn't exist, uh, till wind would have been like maybe it was something that was built that, um, it would've likely been a lot harder for like Adam to get started, uh, building that. Um, and then obviously like it, it was kind of the reverse in the beginning. So it was like a big season output and now it actually is like a lot smarter and can generate CS for you on demand. Um, but like basically like these tools or these compiler libraries, like have allowed a lot of experimentation, like have allowed, uh, components to be created. That kind of stuff from our perspective, like config is not bad. Like if you have to use config, it's totally fine. But we also have to make trade offs of like, okay, this is what we're going to optimize in next. And like port two to SBC, for example, like custom transformed right now, like in next for example you have five, six custom be transforms. We poured all of those to SBC as well. So like they're all native interest, uh, as well now, which also in practice means that we have to contain two versions of it, which is also from the user perspective. Like this is something that a lot of people don't really see, but like, uh, like maintenance becomes infinitely harder when, uh, you start like basically diverging on like some tools. So like if you start, uh, like, because the config is there, you have to maintain specific conflict option, but then also the version, if it's not enabled, right. Uh, and then like the version and it's not enabled could be a lot faster cuz there is no like artificial overhead that people could introduce into the, the application.

Sarah Dayan: 45:26

Everyone makes mistakes, right? Especially with early versions, like V zeros V one S like when you don't have enough user feedback most of the time, and sometimes you can rectify things, uh, but sometimes you're too far off and you need to start over. How, in your opinion, do you gracefully recover from the wrong abstraction?

Tim Neutkens: 45:50

It becomes super hard to remove something once it's introduced. So what we generally do is that try to spend a lot more time thinking about, okay, if we introduce this, what is it going to solve and how long is it going to make sense to maintain it? And then like for most of the new things that we're making, we basically make sure that it can be maintained for quite a long time. Uh, but then also we already have like different solutions insight. So like an example is get props, get server props could actually go away when, uh, newer versions of react are released direct 18 and then like suspense based solutions actually, and allow us to further evolve, get props, get server props. We've had some of the restrictions thats the day. So like over, over time, the libraries that you use are going to evolve and you sort of have to evolve with them. And like, from our perspective, like we, we're always trying to make sure that like, it's always backwards compatible as well. Like even if we introduce a new feature, we still are going to maintain the older feature. And like in next 11, for example, we remove some features, uh, that had been deprecated since next four or something like that. Uh, so like we keep even keep deprecated features for quite some time, but it really depends on like how much the maintenance burden is as well. So like, like WebEx four being deprecated very soon going to mean that in the next major version, but we've com communicated that all already for quite some time, it's get to be removed in the next major version, uh, so that we can then basically focus on making sure that the, the one K so five is a lot more performant than it is today even.

François Chalifour: 47:30

And in, uh, in instant search libraries, we, we made a mistake at the very beginning, which was to abstract some kind of search concept into fr what we thought as friendlier, uh, APIs, but in the end, these APIs were, were too high level and users could not achieve what they wanted to with them. Uh, so that's, uh, one example of when we had to, uh, basically use incremental recovery, uh, by introducing new APIs or tweaking existing APIs to allow users to basically, uh, send some custom such parameters to do such engine. Uh, so we created two APIs for that. Now we consider that Simpson such is kind of as powerful as the underlying mechanisms, but it took some time to actually recover from that. And also we kind of lost a bit of credibility among the community because of this kind of mistake, which is why I think it really all comes down to communication even with the community. So to recover from a wrong abstraction, usually as long as you actually explain the community, um, why it wasn't made that way. And now they will be able to actually, uh, do it because of a new API and they can actually understand it. Uh, usually you're fine, but we, to me really, um, comes down to communication. Uh, another example where we actually, um, kind of when, when we were an not victim of like a, a wrong abstraction, but an outdated abstraction, uh, was, uh, when we created autocomplete V zero. So the, the previous version of the, the current library, it was started in, um, 2015, it was originally a fork of the type head JS library and the library, uh, autocomplete, the auto library, uh, became like in maintenance mode very shortly after because the code base was too hard to contribute to. And we thought that it was sufficient to build search search experiences at the time with this library, but the search ecosystem grew, uh, and search is now, um, presented in most of the apps. Uh, search is actually the foundation of the experience in most of the apps. Now in capital Google maps, Amazon, YouTube, you couldn't really use these apps, uh, now without search. So we were observing such a gap between a complete V zero and the search experiences that we could design that we decided to start from scratch. So this is a rare case where we actually took the decision to start from scratch, which is usually, um, a risky decision. Uh, so six years later, 2021, we release to complete V and, um, we consider it as a good abstraction nowadays because it allows you to be way more complex such experiences. So, um, I don't think it was necessarily a bad abstraction at first, but it was definitely an outdated

Sarah Dayan: 50:28

Abstraction. Yes. Uh, I, I, I like what you said also on, on communication. I, I think like sometimes, okay, you realize it was the wrong way. We'll learn something and we're gonna move on. We're not just gonna stay stuck and that in that, uh, in that scenario, but what's important is that you communicate also that you own it explaining why you failed, uh, going above and beyond to ease the migration, whether it's with a guide, like a really, really well done guide, or maybe maintaining the old version a little bit more if it's necessary. Um, there are really good examples for examples, uh, on episode, uh, mentioned that whenever they do, they break the API, for example, the schema of the content, they will take the migration on themselves for paying customers, uh, and maybe it's even free customers, but I know at least for, uh, a set of, uh, maybe business or enterprise customers, they will do the migration themselves because they decide it's on us. Like we, we broke it. So if we want people to use the next version, especially for them, because it's a, it's a hosted service, um, we, we have to, to, to do it ourself. And I think it also can be interesting is to, instead of just going like, yeah, this was crap. Let's move, move on. Maybe take a little bit more time to identify the root causes of what led to those like faulty abstractions, because my mistakes usually don't don't happen in a vacuum. Usually there are some, some patterns behind it. We were in a rush, we were too UN experimented. We did not talk to customers. Uh, we forked something because we wanted it to be easy, but because we were in a rush, so trying to reflect back and maybe postmortem or, you know, root cause analysis, I think can be, can be beneficial, even if we usually want to move on and just go to the next thing, because we think we know what to do be, cuz we also tend to repeat mistakes when we, we don't take the time to externalize them. And again, this is one of the, the things that I, I really love about, uh, both next and, and autocomplete is, uh, is the concept of micro abstracting and Moi modularizing, your abstractions, making sure that you don't build this whole monolithic system, that whenever you tweak something, you're gonna pull down the entire, you know, house of cards. Um, that to me is super important. Being able to say, okay, we build that thing, but if we change it to more like we don't have to deprecate the entire thing. Uh, like we don't have to go next JS 20 just because this function no longer works. So trying not to couple all your pieces together so that yeah, you have like this kind of huge drips of glue that make everything not even touchable. So I will, uh, before we wrap up, I will ask you the traditional question of the, of the show. How do you define great DX and what is your personal level of expectations?

Tim Neutkens: 53:40

That's about generally three things. So like one is that you get your update. Like if you make a change, the update is there as soon as it can be. So as, as fast as possible, uh, which is what we've been working towards, uh, very heavily for next, uh, to make sure that like, if it changes really small and it doesn't touch like large parts of the code base, um, or like you're importing a new library, it's going to be there in, uh, like less than a hundred milliseconds. That's the, the goal that we have, uh, it's like, that's the, the first thing, things have to be really fast and that's just like, it goes from booting up the dev server to, uh, opening pages. Uh, like we do all of that on, on the amount for next. Like we don't even start to do work, uh, when it's not needed. Uh, but we do try to do things optimistically. So like when you put up the next server, it's going to optimistically compile some of the files that we know that, that you're going to need either way then is the errors. So I think that is generally overlooked is that if an error happens, it generally is not super helpful. Um, so if you're using a library, it's going to throw an error, it's gonna say like, okay, this value was wrong or something like that. But in general, like what you see very often is that it's not, um, as helpful as it could be. Like, why did it throw that error specifically? Like, okay, I'm passing in a string, it should be an integer or something like that. These are simple cases. The complicated cases are like I'm using, uh, tic props and I'm returning some value that doesn't exist, but what are the other values that, uh, that could be used and, or like you make a typo that kind of stuff, things that we could detect, like, Hey, you're making a typo, you can fix it in this way. Um, so what we do for next is actually like sent, uh, with every error or at least most of them at this point, we sent a, uh, specific link to the documentation with special pages specifically for those errors. So anytime an error is added, what we do is we actually add a documentation page as well in the docs messages, uh, folder. Um, and, uh, that's going to explain why did the R happen? Okay, you put it in the wrong value or like you're using the, like, you're holding it wrong, that kind of stuff. And, uh, then gives you an like a way to investigate it or to fix it. Uh, so it tells you why, uh, and then like, how can I fix it? And then also all the documentation around it. So like useful links that, that you could read through as well. Uh, and it has been really good cuz that also we expanded that onto like also collecting feedback from those pages. So we get a lot of feedback from people that read into errors and then say like, okay, this was not helpful. Or like I fixed it in this way. Please add that to the document cuz then, then other people can benefit from it as well. The really big part of of development is actually like reading errors and figuring out what is going wrong. And then the, the, the third thing is that also like the, the good DX also has to like weigh against, uh, having good user experience. So a lot of things that we do in next is also like running you for cases where, uh, you're going to affect user experience by including synchronous scripts, or you're going to include an image stack that blocks ring of the whole page or, uh, that kind stuff like in next 11, we, we have a lot of goal system of plugins that basically warn you like, okay, this is going to be a lot better if you make it, uh, like change it to, to use this other approach. And it goes for like phone preloading all that. And then like, from from our perspective of, we also try to get out of the way very often. So there are some conventions, but like, we're not telling you how to do data fetching, uh, particularly like where you're getting data from, how you're getting the data, because then you, you get the full flexibility of being able to, to build what you want. And that is also led to basically like seeing, uh, sites from like a few pages, personal websites to like massive web applications that, that are served through to millions of people, uh, every day.

François Chalifour: 57:58

Yeah. Good gig for me is really, um, about focusing on my application features and not necessarily on the platform or the infrastructure of feature. Uh, so as, as you said, Tim, basically getting out of the way of the developer and some examples could be, um, at less UIs, like erratic primitives or stuff like that. If you want to create a context menu by yourself, it takes some time. And I try that and basically you waste so much time and the logic of the component itself and that you could actually invest in your, uh, application. Uh, that's it's a waste of time. So to me, good DX is actually providing me the tools that I, that maybe the platform should have, uh, provided me. Although examples could be next best. I don't need to understand that my route is gonna trigger alarm function somewhere in some server. I don't really need to understand that as a user or as a developer. Uh, so this to me, uh, is what DX is about.

Sarah Dayan: 58:56

I like also thinking about the fact that there are so many projects that we start and we don't finish, uh, because we hit a roadblock or we hit like, oh yeah, that conflict thing, or that, as you mentioned, team errors, I think for so many years, I've been like the, the worst DX, uh, of like the entire software, uh, development field, uh, because you see something and you don't, you have no idea what to do with it. And it's like, it's telling you the consequence, but doesn't tell you anything about the cause. And so, yeah, to me, it's really similar to what you just, uh, that you just described. It's like, give me a head start, but don't pick a path for me. Like there are things I wanna do. Don't do them for me. I want to do it. It's, it's gonna be fun. And, and it's also the value that I add with my program. There are things that I don't wanna do, and there are things that I shouldn't do no matter where I, I want to do them or not. And grad DX really lets me do everything I wanna do relieves me from what I don't want to do and convinces me not to handle what I should not be handling. And because, uh, the library of the framework is gonna do it like a million times better. Uh, I, you know, I don't wanna configure. Webpac like, I know I don't wanna do it. Uh, I don't wanna worry about my Google lighthouse score. And even if I enjoy that, I don't really, I should probably not configure my own server, but I wanna craft the web website of my dreams, like down to the tiniest pixel with the libraries that I love. I like it's not gonna be next year is not gonna be my entire life. I want to use tailwind. I want to use Radix. I want to use whatever auto completes, uh, and same for auto complete. I don't want to read the re specs for combo books. I don't have time for that. Uh, I should probably not try to read, build an auto complete solution, even if I think it would be fun because yeah. When, when you see that it takes, uh, dedicate people who are dedicated to, to the problem a year to come up with like a really robust solution, I will probably not nail it down in a, in an afternoon, but I want to be able to grab suggestions from any, anywhere, any API, any static file and, you know, dynamically control, uh, like the, the, the behavior of all that. And that's what I should be focusing on. Especially knowing that my excitement will fade away after a certain period of time. And if I want to ship, I need to be helped in a way that already ahead, by using the right tool. So where can people go to find you online?

Tim Neutkens: 1:01:32

So, for me, it's just, uh, my full name as my handle, pretty much everywhere. So like can get a Twitter, been tweeting a bit less, uh, the past, uh, few months, mostly because just been really busy with building out next features and performance. Improvement's a lot of good things to come there. If you want to learn more about next, you can go to next org, let's learn, uh, it's a interactive tutorial that, that guides you through basically building a NHA app. Uh, all the, the features that we talked about today is like aesthetic props servers. Our props out sees as, uh, all that. We're also, uh, hosting a conference very soon. So, um, on nexus slash con, you can register for a free ticket there.

François Chalifour: 1:02:17

Cool. And you can find me on GI, uh, with my full name lowercase and on Twitter. Um, my name is actually too long for Twitter. And so it's cons C H L F R.

Sarah Dayan: 1:02:30

All right. And, uh, you can find me on Twitter at front stuff, underscore IO, which is the worst end ever. Uh, and you can also find my work at my full name, SOA with Anh dion.com. If you want to learn about auto complete, you can go on our fantastic short URL, alg.ally/autocomplete. You'll find the documentation and you can learn anything about algolia.com team UA. Thank you so much. Chatting with me today was really fascinating. Conversation went over time, and I wish we could do an extra hour listeners. I hope you had a blast. Thanks a lot for, uh, following developer experience and station. For the next episode. This was developer experience, a podcast brought to you by Golia. You can find this podcast on your favorite podcast platform. We are on Spotify, apple podcast, Google podcast, Pandora, overcast, everywhere. If you want to know more about Algolia check us out on algolia.com and we are at Algolia on Twitter.