How LLMs change my codebase
Fewer dependencies.
I've always been suspicious of pulling in a library to accomplish what a few hundred lines of code could also accomplish. Now, that sentiment is doubled. It's faster to write and maintain this sort of glue code, and in my experience, an LLM has much higher aptitude at scanning a local file than its third-party equivalent—especially if that third-party equivalent has changed its APIs over the years.
'm not saying to get rid of React or stop using major load-bearing dependencies, but rather the things that aren't quite utilities but not much more substantial either. Think: a third-party package that just wraps requests to the Mailchimp API. Not only can an LLM very cheaply and easily build this for me, but it can maintain it, and we can force it to adhere to the same abstractions and DSLs for all of our third-party services.
More testing abstractions.
Writing tests is something many people point to as a great use case for LLMs. This has actually not been my experience. I find LLM-generated tests to be unnecessarily verbose, poorly abstracted, and generally not inspiring confidence in my ability to understand the surface area under test. However, when I give an LLM a very strict and clearly defined harness—a list of strings to be snapshotted, or a list of JSON files to be replayed—things seem to go much better.
This usually isn't a particularly high level of effort or investment, but it's slightly more than I've previously justified, except for the particularly gnarly suites where we have dozens if not hundreds of states to verify.
Land the boring stuff first.
Let's say we're launching a PayPal integration so that paid authors can accept payments via PayPal, not just Stripe. To be clear, we're not doing this. Please do not email me about getting invited to the beta.
Historically, my approach to this kind of project is outside-in: start with a conception of what the user flows should be, build out front-end stubs to play with those flows and make sure they feel correct, then build mocks at the API level to make them feel a little more real, then implement the actual backend structure.
I've found myself gravitating towards the inverted version of that. LLMs are quite good at reading the PayPal API spec and creating all the right Django models for exactly what we need, especially because it's pattern matching onto a number of existing models we have. But they're less good at all of the squishy stuff. In the fairly common case where we have extremely high confidence in the shape of the backend—the data modeling, the glue code, the API specs—but still need to do some work understanding how they cohere into the frontend, I find myself increasingly comfortable shipping the backend work first. It makes it easier to wire everything together and then play around with various front-end configurations. Obviously, this presupposes that the data model won't change based on front-end experimentation. Sometimes that assumption is true and sometimes it's not.
Slurp up all the data.
I wrote about insourcing from an architectural and strategic perspective a couple months ago. One of the things that came out of that essay was the fact that often the most annoying part about insourcing data is the schleppy implementation cost of data sync. Setting up the requisite webhooks and crons so that you have a copy of your Linear database in your production application merely so you can do a couple nice-to-have things—like show in the admin what Linear issues a given newsletter is tagged to—feels like overkill when you weigh the effort against literally anything else you could be doing.
But when you can model that effort out to nearly zero, the value proposition becomes much more attractive. Moreover, the idea of having your own tiny little data warehouse compounds in surprising ways. For instance, we're able to do some pretty nice and clever things combining Stripe and Plain: we can see when someone has written in and is suddenly a churn risk because they paused or set their subscription to lapse. This could easily be done otherwise, but it's hard to justify the upfront effort. Now I find myself defaulting towards just throwing whatever external data is easily available via webhook into our database. The implementation cost rounds down to zero, the upkeep cost rounds down to zero, and there's a lot of asymmetric upside.
Keep content flat.
Lee Robinson beat me to the punch here, so I'll be brief rather than regurgitate his well-reasoned (albeit self-serving) piece. It's true!