It has been a while since I shipped a new thing; it felt good to get back in the saddle, even if the project was so small as to barely warrant the price of the (admittedly fun) domain. I present handbook.directory, a constellation of various once-internal PDFs for company culture. (This was inspired by the brouhaha around Mr. Beast's leaked manual.)

The entire contents is open source, though I'm using Blot to host it.

I was thinking about using this as an opportunity to kick the tires on Astro, but the sheer paucity of content and the fact that I really wanted to get this across the finish line in the course of a one-hour lunch break led me to Blot, which I had never used before but had roughly pigeonholed as "the easiest way to ship very simple static content".

That pigeonholing turned out to be accurate! There was a bit of onboarding jank choosing a theme and getting the DNS records set up (when is there not?) but I was very happy with how Blot performed. Moreover, the thing that really impressed me with Blot was just how fast deploys are. I hit git push blot and by the time I hop over to Chrome and refresh the tab the changes are there.

It is a good reminder of the strides that we've made on the Buttondown front with CI — down roughly from twenty minutes end-to-end to ten — might be impressive in a relative sense but we've got a long way to go in an absolute sense.

There's no real "endgame" with handbook.directory, but it got some solid traction on Twitter and ended up with a dozen GitHub stars and around three thousand pageviews. I'll take it!

Postmark had an eight-hour outage due to an SSL certificate expiry.

Buttondown was not affected; we send a lot of our custom domain traffic through Postmark, but all of it is hitting their API, rather than their SMTP, and their SMTP is what had the SSL issue.

Still, if we had been impacted we would have been fine. A few years ago I accidentally made a great decision to abstract out our backing ESP, which means at any point in time I can run a command like:

Newsletter
  .objects
  .filter(delivery_provider="postmark")
  .update(delivery_provider="sendgrid")

Whenever these things happen, consternation always emerges: some about the specific genre of issue (“an SSL cert? really?”) some about time to resolution, some about comms. ^[1] Sometimes this discourse is productive; often it is not.

In particular, the least valuable genre of discourse is “well, it’s time to move to $COMPETITOR.” I say that this is not valuable because in general, you should operate under the assumption that every vendor you rely on will have an outage at some point in your time as a customer. (And, indeed, the competitor that I saw most often referenced as a replacement for Postmark has had a noteworthily poor amount of uptime over the past two years.) Your job as a technologist is not to find the one magic vendor that never goes down (it doesn’t exist!) but to build resiliency into your system such that the impact of any downtime can be minimized as much as possible.

This is not to say that you shouldn’t re-evaluate your choice in vendor due to outages — I initially moved off of Mailgun for that exact reason! — but that you don’t solve single point of failure issues by replacing one single point with another. ^[2]

Two other notes that this reminded me of:

1. There is no easier way to generate negative karma than to try and use a competitor’s outage to your advantage. (To wit, I will probably never use Render.)
2. I was talking with a founder friend a few months back about existential risks, and we both landed on the same conclusion: the single biggest vendor-related fear we had was Stripe.

---

1. Comms, by the way, is where I extend the least possible grace; I have, as of this writing, not yet received an email from Postmark informing me of the incident. ↩︎

2. This is unrelated, but: this is also why, at a meta-level, running a business where you can essentially be hot-swapped out by changing an endpoint or three is such brutal work. Especially when one of the endpoints that your customer base can switch to is, well, Amazon. ↩︎

The state of play

Much of the discussion about Twitter’s slow death — or, if not death, metamorphosis into a place where I prefer not to spend my time and energy — and its theoretical successors (Bluesky, Threads, et al) quickly turns to the realm of moral imperative, at which point it becomes to me odious and unproductive.

I frankly do not want to replace Twitter, nor do I want it to die. I owe my career and my business to the connections I’ve made there. I am a loyal fan, in much the same way the Joe Philbin era did not cause me to swear off the Dolphins forever (though perhaps it should have.)

For me, my motivation to replace Twitter is twofold and bluntly realpolitik:

1. Many of my friends (digital and otherwise) are no longer on Twitter, and therefore it is a less compelling social network.
2. Twitter no longer generates significant traffic either to my blog or to my business, and therefore it is a less compelling social network.

The past six months or so have been, to a certain extent, a listening tour of all the various heirs apparent to see what struck me as “better”.

• LinkedIn appears to be the network that has most obviously benefited over the past twelve months in terms of volume and activity. The experience of using LinkedIn feels roughly like a 101-level course on “business skills” at a liberal arts school — the algorithmic timeline is polluted with anodyne content (each one of them punctuated with a completely unrelated video to juice consumption metrics), my inbox is uncontrollable and filled with sponsored invitations to online MBA programs, and the amount of legitimate content that I am interested in consuming is probably around 5%. Despite these things, the network part of LinkedIn is great, and I have been able to forge legitimate connections.
• Threads feels very vividly like using a product that is undergoing some sort of internal civil war. There are parts to like and admire: I think it has the best interface of any social networking product I’ve used; I admire Meta’s forays into the developer ecosystem and the fediverse even if they’re probably coming from a cynical place. Unfortunately (and this might be true of Meta products writ large, but I don’t use Facebook or Instagram) Threads is so deeply committed to divorcing you from your follower graph that it’s hard to actually spend more than five minutes in the interface without finding a piece of content so anodyne/stupid that you close the tab, a strategy seemingly encouraged by their refusal to let you default to Following and decision to take a page out of LinkedIn’s book and push recommended content into the notifications tab.
• Mastodon in many ways is the anti-Threads. Using Mastodon is a deeply calm experience; I see exactly the people and content that I have opted in to see, folks are generally very pleasant and polite, and I never feel like I am fighting against ActivityPub or the broader lattice of decentralization. However, the Mastodon experience is (in many cases intentionally!) poor, especially as a business: the inability to search instances for someone mentioning buttondown.com, the lag and jank of notifications, and so on, are not deal-breakers but do leave something to be desired.
• Slack/Discord/WhatsApp are, as many people know, where the true juice is these days. The highest-bandwidth conversations I’m having are in hyper-specific, highly-engaged servers like Email Geeks and Stockholm Syndrome Pythonistas, filled with lots of really great discussions around a single topic. This is (and I say this knowing full well I run the danger of veering towards the realm of moral imperative!) not a social network by any non-trivial standard, because these discussions are not publicly visible and most importantly they are not indexed. This is useful in some respects — privacy augers candor — and sad in others, and I call it out less as a legitimate replacement for The Public Squre and more as a sad acknowledgement that the kinds of terrific, nuanced conversations that we were having in public ten years ago we are now having behind closed doors.
• I am simply too old for TikTok.

Bluesky

This leaves the titular social network of this post, and the one that I find myself enjoying the most: Bluesky.

Setting aside whatever preconceived notions you might have as to Bluesky’s Twitter-inflected past, ATProto’s legitimacy as a protocol, or the word “skeet”, Bluesky checks all of my very specific, curmudgeonly boxes:

1. It has an open and legible API (albeit one that is in flux);
2. My friend and colleagues use Bluesky non-trivially;
3. The interface is not unpleasant to use (which I do not mean in a damning-with-faint-praise way — my bar is simply “I do not want to actively feel bad whilst using the app”, and Bluesky meets that)

I think if you are in any kind of Hacker News-adjacent zeitgeist (which, if you’re reading this, you probably are!) you might want to sign up for an account. It’s a pretty good experience, and feels more than anything else like what being in a nascent-but-vibrant social network felt like a decade ago. It is not (to steal a phrase from Jacob Matson) load-bearing; most people are not yet on there, and whether that is a benefit or a drawback is left up to you to decide.

Bluesky Social PBC

One thing I find myself asking, in a spirit of good faith and charity: what is Bluesky’s endgame?

It is hard to make the finances for a social network really work (pour one out for Cohost). Threads obviates this problem by being part of the Metaverse; Mastodon’s decentralized architecture eliminates their need to really “have” an endgame.

Bluesky raised an $8M series A and appear by all accounts to be managing burn pretty reasonably, but the most optimistic/realistic path I see is a sort of “Twitter on New Game Plus” scenario — the same monetization route, with all of the niceties that they’ve accrued along the way (ATProto, open sourced clients, etc.)

It may be a failure of either imagination or conviction that I can’t see some other path; indeed, Bluesky’s success or failure probably hinges on their ability to find a path forward without having to construct the two-sided-advertising Borg (or to do so at a hitherto-unseen level of amicability).

Regardless of their future, I am quite happy they are giving it a shot, and I’m looking forward to spending more time over there (even if it’s in a decidedly POSSE manner.)

When I was first starting my career at Amazon — even more bright-eyed and rosy-cheeked than I am now — I was thrilled by the concept of an "architecture review", and by extension the concept of a "Principal Engineer" (Amazon's term for a staff-level engineer, someone beyond career level) who was always treated with some level of mystique and reverence.

This person's role, in the abstract, seemed much more exciting and powerful at the time than it does today — to be a sort of technical Commodus, weighing in from the heavens on various proposed schemas and designs, relying on sheer technical metis to make decisions that affect the lives of millions of people. ^[1]

One afternoon, we proposed a cluster of microservices for tracking various metadata across versions of Kindle eBooks to better track duplicates and divergences. After a two-hour presentation, we were thrilled to get only a tiny little bit of actionable feedback: squirreled away in the ERD we presented was some sort of is_active field for each revision of the book, which we would use to filter out revisions of an eBook that weren't circulating on customers' devices. His comment:

Shift is_active to something more generic: a status field, so we can expand it down the line for other parts of the state space.

As luck would have it, this principal engineer was stuck in an elevator with me heading down from Obidos across the street, and he kindly endured my timid battery of questions: what was it like, doing all these reviews? How did he notice the is_active field specifically? What did he think about how we were approaching PubSub? And so on.

He paused for a moment, and then said:

Honestly, 80% of the time in these meetings I just tell people to either use an enum instead of a boolean or to make it more clear which data is events and which is state. I'm always right, it's always useful, and there's never that much fuss about it.

It took me a couple more years to realize the broader implications of what this nice Swedish man was saying, but I find myself returning to the object-level architectural advice more often than not. It's not a free lunch, but prematurely turning state from a binary into an enum is an easy way to save yourself — not quite a migraine headache, but a headache nonetheless — down the line when you inevitably need to expand active | inactive into active | inactive | paused or what have you.

---

1. This is of course a simplification of the role, which speaks in equal parts to my then-naivete and to Amazon's dearth in career ladder education. ↩︎

In Notes on buttondown.com and How Buttondown uses HAProxy, I outlined the slightly kludgy way we serve buttondown.com both as a marketing site (public-facing, Next/Vercel, largely just content pushed by non-developers) and an author-facing app (behind a login, Django/Heroku/Vue) and recommended developers not do that and instead just do the sensible thing of having:

• foo.com, a marketing site
• app.foo.com, an application site

This prompted questions, all of the genre: “why do you need your marketing site to be hosted/built differently from your application site?”

A few reasons:

1. You are, at some point, going to want someone non-technical to be able to contribute to your marketing site. If you host your marketing site from the same framework/repository as your application site, you have suddenly capped your CMS flexibility at “whatever integrates with my stack.”
2. You do not want to couple deployments/SEO of the marketing site with the application site. (Do you want to force CI to run and end-to-end deployments to trigger to fix a typo? Do you want an OOM to take down your blog?)
3. Namespacing is much easier (for things like, say, a whole-ass domain migration) when you don’t have to keep a sitemap that contains both internal and external paths.

There are reasonable counterarguments:

1. Being able to commingle business logic with marketing can lead to powerful programmatic SEO or other clever things.
2. Shunting your marketing site off to a purpose-built CMS like Webflow hamstrings your ability to iterate quickly.

I think the synthesis that we landed on — Next (powered by Keystatic) gives us the best of both worlds. Non-technical writers can publish and edit easily; we can do fancy programmatic things. But none of that obviates what is in my mind the more clear-cut piece of advice:

Even if you’re dead-set on having a single application serve both the marketing and application site, deploy them to separate domains.

Cable Cowboy is a rollicking read that serves better as a primer for a fascinating industry than as a legitimate profile (or hagiography) of the man from whose eyes the history has unfolded. Robichaux lacks the incisive vigor that made Barbarians at the Gate so compelling (and sometimes frustrating) as a character study and the Caro-esque majesty of vision to more carefully connect the dots of the major players and landscapes not just with each other but with the overarching shift in how the world worked.

But to condemn this book for not adequately interrogating power or serving double duty as a treatise on America's technologically-driven shift from statism into modern neoliberal nationhood is unfair, because what it does do is take you through a whirlwind tour of a fascinating set of companies and operators, and their struggle against all takers — the public sector, the financiers, the relentless march of technological progress — to make money.

TCI — and John Malone — are most well-known for their approach to capital, and the book goes over the broad strokes a fair enough amount (albeit not enough to reallly cause any new revelations): a focus on lean capital efficiency, an aggressive love of financial instrumentation, an interest in deals qua deals more than areas of strategic investment. There was nothing noteworthy here.

What I found more fascinating — and what I think gets omitted in the SparkNotes version of TCI's growth and history — is how recurrent every company and player is over a sufficiently long timespan in a commoditized industry. Someone who you are at war with in 1983 is a joint venture partner in 1992; a sales connection in 1972 is a potential acquirer in 1977.

Cable Cowboy makes the industry, for lack of a less cheugy metaphor, look like Settlers of Catan: Malone's gift was neither a ruthless efficiency nor an unparalleled understanding of markets, but an uncanny knack for always finding a deal to be done to eke out a little bit of long-term margin, and never burning any bridges.

Highlights

They killed the pig.

We finished Buttondown’s migration from MDX to Markdoc last week. It went swimmingly, except for one little hitch: our RSS feeds, which sat on top of getServerSideProps and read in the flat .mdoc files, threw 500s in Vercel. (They worked fine locally and in CI, but then those files were purged by Vercel as part of the post-compile deploy.)

I was considering going back to our previous (slightly janky but perfectly reasonable) approach of having a script that generates the RSS files and then just serving them as static asset, but Max Stoiber pointed me in the right direction:

• Create an App Router Route Handler
• Set dynamic="force-static"
• Build the XML in-band.

This means all we had to do was this:

import { createReader } from "@keystatic/core/reader";
import config, { localBaseURL } from "../../../keystatic.config";

const reader = createReader(localBaseURL, config);

export const dynamic = "force-static";

const CHANNEL_METADATA = {
  title: "Buttondown's blog",
  description: "Buttondown's blog — guides, tutorials, and more",
  link: "https://buttondown.com",
};

export async function GET() {
  const slugs = await reader.collections.blog.list();

  const rawPostData = await Promise.all(
    slugs.map(async (slug) => {
      const response = await reader.collections.blog.read(slug, {
        resolveLinkedFiles: true,
      });
      return {
        slug,
        ...response,
      };
    })
  );

  const sortedPostData = rawPostData.sort((a, b) => {
    const coercedADate = new Date(a.date || "");
    const coercedBDate = new Date(b.date || "");
    return coercedBDate.getTime() - coercedADate.getTime();
  });

  const items = sortedPostData.map((post) => ({
    title: post.title,
    description: post.description,
    link: `https://buttondown.com/blog/${post.slug}`,
    pubDate: new Date(post.date || "").toUTCString(),
  }));
  const rssFeed = `<rss version="2.0">
        <channel>
            <title>${CHANNEL_METADATA.title}</title>
            <description>${CHANNEL_METADATA.description}</description>
            <link>${CHANNEL_METADATA.link}</link>
            ${items
              .map(
                (item) => `<item>
                <title>${item.title}</title>
                <description>${item.description}</description>
                <link>${item.link}</link>
                <pubDate>${item.pubDate}</pubDate>
            </item>`
              )
              .join("\n")}
        </channel>
    </rss>`;

  return new Response(rssFeed, {
    headers: {
      "Content-Type": "text/xml",
    },
  });
}

Hopefully, Next will make this all a thing of the past and create a lightweight DSL like they did for sitemaps. In the meantime, though, I hope this helps!

Yesterday, I was trying to set a unique constraint for comments in Buttondown to prevent accidental double-commenting, and I ran into a problem that I hadn't seen before:

index row size 2816 exceeds btree version 4 maximum 2704 for index "emails_comment_email_id_subscriber_id_text_0542cca9_uniq"
DETAIL:  Index row references tuple (165,7) in relation "emails_comment".
HINT:  Values larger than 1/3 of a buffer page cannot be indexed.
Consider a function index of an MD5 hash of the value, or use full text indexing.

Simple enough: indexing a very long string is going to be prohibitively bad. It was immediately clear that the right path forward was to index the MD5 hash of the text rather than the text itself, but the literature on how to do so within the ORM was somewhat lacking:

• A decades-old tracking ticket had nothing useful
• StackOverflow either recommended dropping down to raw SQL or didn't think it was possible for uniqueness constraints

However, the solution is actually quite easy! Since Django 4.0, you can use expression-based uniqueness constraints, and Django even offers a handy MD5 function right out of the box. All I had to do was this:

from django.db.models import UniqueConstraint
from django.db.models.functions import MD5

class Comment(models.Model):
    text = models.TextField()
    email = models.EmailField()

    class Meta:
        constraints = [
            models.UniqueConstraint(MD5("text"), "email", name="unique_text_email_idx")
        ]

And that's it!

In Paul Graham’s latest essay, he writes:

The theme of Brian's talk was that the conventional wisdom about how to run larger companies is mistaken. As Airbnb grew, well-meaning people advised him that he had to run the company in a certain way for it to scale. Their advice could be optimistically summarized as "hire good people and give them room to do their jobs." He followed this advice and the results were disastrous. So he had to figure out a better way on his own, which he did partly by studying how Steve Jobs ran Apple. So far it seems to be working. Airbnb's free cash flow margin is now among the best in Silicon Valley.

Readers are not privy to the exact talk; Graham presents it as a dichotomy between “manager mode” and “founder mode”, where founder mode exists not as a distinct methodology but as a rejection of “manager mode” practices:

Hire good people and give them room to do their jobs. Sounds great when it's described that way, doesn't it? Except in practice, judging from the report of founder after founder, what this often turns out to mean is: hire professional fakers and let them drive the company into the ground.

This dichotomy is reductive, but it hints at two commingled and pernicious issues facing most scaling organizations:

1. The process of interviewing and evaluating managers is incredibly inaccurate.
2. There exists very few organizational incentives or backstops to curtail growth.

Insofaras management is a science, we are still in the “leeches and humours” stage of things — we do not know how to organize knowledge workers and we do not know how to evaluate knowledge work. (To paraphrase Huemer, we are more likely to kill companies through bloodletting than save them through germ theory), and it makes sense that “founder mode” (as defined by a bias towards, more bluntly, dancing with the corporate ethos that brought you) is on net better than the current state of the art.

But “don’t overhire and don’t overstratify your management” is necessary, but not sufficient: if that’s all it took, presumably we’d see a dynamic wherein there were more Valve-shaped companies (small, flat, incredibly prolific.) I think it’s important to cultivate what Sebastian Bensusan calls lieutenancy:

Most people don’t realize it is their job to unblock themselves and that they don’t need permission to do it. You need people who act even when they hit “extraordinary blockers”.

Okay, so how do you cultivate lieutenancy? In three ways, each of which is probably worth writing about more:

1. Prioritize tenure as an organizational health metric.
2. Align compensation with business outcomes.
3. Draw very clear lines of ownership, and very high demands for owners.

I am sure all of my brightless praise of Le Samourai — a film that, in retrospect, I should have watched a decade or so ago — has all been said before, time and time again. The gorgeous, minimalist direction (I need to watch more Melville, clearly, and have added Le Cercle Rouge to my watchlist); the impassive and perfect performance of Alain Delon (and, in obviously a much smaller but equally delightful role, Nathalie Delon); the perfect-no-notes-copied-many-times-but-improved-never ending.

It is rare when a noir succeeds both at the visceral, tangible level and at the spiritual level. And even then: great noirs leave you asking questions about the world: you finish Chinatown or The Parallax View with an understanding of the events you've witnessed but a deep gnawing doubt about the world around you. Le Samourai inverts that: you understand the world, and you understand not just your place and Jef's place in it, but you sit there and wonder: who is this man? What did he want from the world, and what did he know that we don't?

Someone emailed me in response to Two years as an independent technologist, in which I wrote:

I miss of being at a large company, which is dealing with deeply cutting-edge technical problems, but my ability to analyze information, make decisions, and perform at a high-level has grown very quickly.

They followed up:

I had a lot of trepidation around “losing my edge” not working on “hard engineering problems”. It sounds like you had the exact same concerns as well. Reflecting now, do you think you’ve continued to level up or refine your engineering skills?

My response is as follows!

---

Depends on how specifically you want to define “engineering skills.” For instance: everything Buttondown-related is, objectively speaking, pretty trivial in terms of scale. Our largest table is in the order of ~five billion rows, and that’s an outlier (event data!); this is just simply not that much compared to my time at Stripe/Amazon, where a much bigger part of my job was some variation of “figure out how to architect a system that can handle one or two orders of magnitude more volume than conventional wisdom permits.”

So, unless you’re working on a very specific kind of company, I think it’s just not very likely that you’re going to progress in terms of “hard engineering” through independent work compared to spending time at a FAANG where all of the engineering problems are definitionally hard engineering problems.

However! There are two (slightly related) things that make up for this:

1. What you lose in depth, you make up for in breadth. I am exposed to so much more new stuff on a weekly or even daily basis: partially because the ease of adopting and deploying a new piece of technology is much higher (no SecRev, no SOC2, no enterprise sales dance...), partially becauses there is so much ground to be covered, so quickly.
2. You can always outsource your core competencies (don’t want to deal with hardware? use Vercel/Railway! don’t want to deal with hyperscaing a database? use Planetscale! etc.) but you can’t even outsource the decision to outsource those core competencies. There’s no #search team in Slack that you get to pepper with questions about benefits and drawbacks of certain ElasticSearch use case; you learn by doing, and you learn that it’s important to do so really, really quickly.

Incremental games are tricky beasts. I think the best ones are like Farm RPG and Melvor Idle, which share a handful of common traits:

1. An emphasis on pathing and napkin-level theorycrafting, where you feel rewarded for your mastery of simple mechanics by getting from point A to point B 10% faster than you would have otherwise.
2. Calm progression punctuated by bursts of epiphany and dopamine (a rare drop that changes your plans; a new skill or item that dramatically unlocks a genre of work).
3. A overall feeling of a good time, and sufficient levels of facade to distract you from the fact that you're essentially incorporating Progress Quest into your daily routine.

Wizrobe hits on some of these — particularly the second — but not all three simultaneously, and once the excitement of midgame progression is over, it's a bit of a letdown. There are too many systems for you to feel great and in command of the gestalt, and many of the mechanics are simply underwhelming (the adventure/combat system perhaps most notable). It simply does not sell the illusion of agency strongly enough; you feel Skinner's influence a little too strongly, with little outcome to show for it.

There are few technical decisions I regret more with Buttondown than the decision to combine the author-facing app, the subscriber-facing app, and the marketing site all under a single domain. Most technical decisions are reversible with sufficient grit and dedication; this one is not, because it requires customers to change their URLs and domains.

There are a number of reasons why this was a bad decision, and that’s probably worth an essay in its own right, but this is more meant to discuss how we work around the problem.

At a high level, it looks something like this:

All requests run through buttondown-load-balancer, which is a Docker container in Heroku containing HAProxy. I got the bones of this container from a lovely blog post from Plane.

The HAProxy configuration looks something like this:

global
    maxconn 256

defaults
    mode http
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms

frontend http
    # $PORT comes from Heroku, via some sort of dark magic.
    bind "0.0.0.0:$PORT"

    option forwardfor

    redirect scheme https code 301 if ! { hdr(x-forwarded-proto) https }

    # We want to find references to the old TLD and redirect them to the new one.
    # Don't redirect POST requests, because they're used for webhooks.
    acl is_old_tld hdr(host) -i buttondown.com
    http-request redirect prefix https://buttondown.com code 302 if METH_GET is_old_tld

    # Yes, it is unfortunate that this is hardcoded and can't be pulled out
    # from the sitemap of `marketing`. But in practice I do not think it is
    # a big deal.
    # 1. These represent absolute paths without subfolders
    acl is_marketing path -i / /climate /alternatives /pricing /sitemap.xml /sitemap-0.xml /stack /open-source
    # 2. These represent subfolders. (Legal has a trailing slash so usernames can start with legal.)
    acl is_marketing path_beg /features /use-cases /comparisons /legal/ /changelog /rss/changelog.xml /blog /rss/blog.xml /api/og /comparison-guides /stories/ /testimonials /resources

    # Docs technically live on a subdomain, but lots of people try to go to `buttondown.com/docs`.
    acl is_marketing path_beg /docs

    # 3. `_next` is a Next.js-specific thing...
    acl is_marketing path_beg /_next /_vercel

    # 4. But `next-assets` is something I made up just to make it easy to namespace the assets.
    #    (There's a corresponding `/next-assets` folder in `marketing`.)
    acl is_marketing path_beg /next-assets

    # Where the magic happens: route all marketing traffic to the marketing app.
    use_backend buttondown-marketing if is_marketing
    default_backend buttondown-app

backend buttondown-app
    # We need to set `req.hdr(host)` so that `app` can correctly route custom domains.
    http-request set-header X-Forwarded-Host %[req.hdr(host)]
    http-request set-header X-Forwarded-Port %[dst_port]

    reqirep ^Host: Host:\ buttondown.herokuapp.com
    server buttondown-app buttondown.herokuapp.com:443 ssl verify none

backend buttondown-marketing
    http-request set-header X-Forwarded-Host buttondown.com
    http-request set-header X-Forwarded-Port %[dst_port]

    reqirep ^Host: Host:\ marketing.buttondown.com
    server buttondown-marketing marketing.buttondown.com:443 ssl verify none

This allows us to deploy the marketing site and the application site separately and without real worry about polluting the two (and indeed, our marketing site is on Vercel whereas the application site is on Heroku).

The only real source of angst comes from keeping the routing information up to date. As you can see from the above file, we have a thinner list of potential routes on the marketing site so we have to enumerate them, and often we forget to do so and so new pages are “hidden” (ie served by the Django app, which then throws a 404.)

Another challenge was testing this. When I first developed this approach three years ago it was, frankly, pretty reasonable to test in prod — Heroku is very quick at reverting Docker deploys, so I could just push and yoink if necessary. Now, though, a few seconds of downtime corresponds to thousands of page-views lost; we’re using Hurl as a very lovely testing harness, with an approach largely inspired by this blog post from InfoQ.

All in all: this approach is janky, and required a lot of tinkering, but is very stable in its steady state. I cannot emphasize enough that if you are starting an app, you should not even entertain the idea of doing this: namespace your actual application under app. or dashboard. or whatever and call it a day, and your marketing department will thank you. But if you’re already stuck and need a way out: this one works.

(By the way, if someone has a non-insane way to do this, please let me know. This approach has worked well, is stable, and does not feel particularly unperformant, but it feels bad.)

Enough time has passed for me to admit that I thought Room 25 was a poor album. I suspect there's a lot of Trump-era literature that will age poorly in much the same way, which is not to say that it was "too political" but that we found ourselves, briefly, in a time where we were comfortable mistaking incohesion for deconstructionist ambition and poorly-recycled Twitter bits (imagine the mockery if Drake had rapped "I’m struggling to simmer down, maybe I'm an insomni-black") for cleverness. It was a disappointing sophomore release ^[1] from a great artist, and I suspect many of its laudits — not unlike Dedication — came more from critics wanting to honor the artist's previous work.

I say this as a precursor to Sundial addressing almost every flaw of Room 25. Noname's politics are sharper and more honest; she conjures again the intimacy that made Telefone such a treat; the production shifts away from a slightly schizophrenic neo-soul thing back to the (Saba-inflected) light jazz rap. Sundial is (and I don't mean this in a faint-praise way) coherent; it sounds like an obvious evolution of her work and her thesis rather than a rush to get something out the door while everyone's listening.

---

1. Her studio debut, sure, but calling Telefone a mixtape feels like a distinction without a difference. ↩︎

We spent $85,000 for buttondown.com in April; this was the biggest capital expenditure I've ever made, and though it was coming from cash flow generated by Buttondown rather than my own checking account it was by rough estimation the largest non-house purchase I've ever made.

As of August, we're officially migrated over from buttondown.com to buttondown.com. I'm sure I'll do a more corporate blog post on the transition in the future, but for now I want to jot down some process notes:

• The entire process was made much more painful due to Buttondown's architecture, which is a hybrid of Vercel/Next (for the marketing site and docs site) and Django/Heroku (for the core app) managed by a HAProxy load balancer to route requests. We ended up using hurl as a test harness around HAProxy, something we probably should have done three years ago.
• I went in expecting SEO traffic to be hit as Google renegotiates legions of canonical URLs; it hasn't, at least thus far. Instead, everything seems to have just bumped fairly healthily.
• I expected more production issues to come up than actually did. I credit this to a fairly clear scope: the goal was "to migrate all web traffic to .com", which meant that a) we didn't need to re-map any paths and b) we didn't need to worry about mapping SMTP traffic (which still runs through buttondown.com).
• The hardest part of the process was the stuff you can't grep for. URLs on other sites, OAuth redirect URLs, that sort of thing.
• Starting with isolated domains (the documentation site, the demo site) that weren't tied to the aforementioned HAProxy load balancer gave me some good early confidence that the migration would be smooth.

Overall: very happy with how it turned out. I would describe the project roughly as "three months of fretting/planning, one week of grepping, and one week of fallout."

Was it worth it? Yes, I think so. Most theoretical capital expenditures Buttondown can make right now have a non-trivial ongoing cost associated with them (buy another newsletter company or content vertical and now you have to run it on a day-to-day basis; do a big marketing build-out and you have to manage it; etc.) — this was a sharp but fixed cost, and it's something that I knew I wanted to do in the fullness of time. (And, most importantly, people stop referring to Buttondown as "Buttondown Email", a personal pet peeve of mine.)

When it comes to AI tooling, I am equal parts optimist and cynic. I have no moral qualm with using these tools (Supermaven is a pretty heavy part of my day-to-day work), but have found most tools quite bad by the metric of "do they make me more productive on Buttondown's code base?" I think it's important to be able to taste the kool-aid with these kinds of things, and try to carve out an hour every weekend to test something new.

My own personal Turing test as of late has been porting some old Django test cases to pytest. Our codebase is around 75% pytest, and I'd love for that to be 100% but it's not really urgent, but it does have a couple characteristics that make it particularly useful for testing an AI tool:

1. It's immediately obvious whether or not the work was successful (i.e. do the tests execute and pass or not?)
2. It's the kind of work that I really want to be able to delegate to a tool — I can do it myself, but it's monotonous and I don't add much value
3. There's a good amount of prior art on how pytest works, but it's not as common as unittest.
4. pytest fixtures are tricky (they exist in different files, their usage pattern is non-obvious).

Two standalone tools (GitHub's Copilot Workspace, SourceGraph's Cody) have failed this test; Cursor, however, succeeded.

To emphasize, these are not complicated test files. Here's a very basic (real) file that Cursor succeeded at porting from Django's test framework:

from django.test import TestCase

from monetization.events.charge_refunded import handle
from monetization.models import StripeAccount, StripeCharge
from monetization.tests.utils import construct_event


class ChargeRefundedTestCase(TestCase):
    def setUp(self) -> None:
        self.event = construct_event("charge_refunded.json")
        self.account_id = "acct_whomstever"
        self.account = StripeAccount.objects.create(account_id=self.account_id)

    def test_basic(self) -> None:
        charge = StripeCharge.objects.create(
            charge_id="ch_whatever", account=self.account
        )
        handle(self.event.object, self.account_id)
        assert charge.refunds.count() == 1

to pytest!

import pytest

from monetization.events.charge_refunded import handle
from monetization.models import StripeCharge
from monetization.tests.utils import construct_event


@pytest.fixture
def stripe_charge(stripe_account):
    return StripeCharge.objects.create(
        charge_id="ch_whatever", account=stripe_account
    )


def test_basic(stripe_account, stripe_charge):
    event = construct_event("charge_refunded.json")
    handle(event.object, stripe_account.account_id)
    assert stripe_charge.refunds.count() == 1

(Note that it's intentional that the stripe_account fixture is not actually in this file: it's in a global conftest.py that I pointed Cursor to.)

This is basically the most trivial possible port (and, again, Cody + Copilot Workspace both failed). Here's a slightly more complicated one testing out our Exports API:

from unittest import mock
from unittest.mock import MagicMock

from django.test import override_settings

from api.tests.utils import ViewSetTestCase
from emails.models.account.model import Account
from emails.models.export.model import Export
from emails.tests.utils import FakeData


class ExportViewSetTestCase(ViewSetTestCase):
    url = "/v1/exports"

    @override_settings(IS_TEST=False, DEBUG=False)
    def test_list_on_v2(self) -> None:
        account = self.newsletter.owning_account
        account.billing_type = Account.BillingType.V2
        account.save()
        response = self.api_client.get(self.url)
        assert response.status_code == 403, response.content
        assert "upgrade your account" in response.json()["detail"]

    @mock.patch("emails.models.export.actions.s3.put")
    def test_list(self, put_mock: MagicMock) -> None:
        put_mock.return_value = "s3://foo/bar"
        FakeData.export(newsletter=self.newsletter)
        FakeData.export(newsletter=self.newsletter)
        response = self.api_client.get(self.url)
        assert response.status_code == 200, str(response.content)
        assert isinstance(response.json(), dict), response.json()
        assert response.json()["count"] == 2, response.json()

    @mock.patch("emails.models.export.actions.s3.put")
    def test_list_should_not_pollute_across_newsletters(
        self, put_mock: MagicMock
    ) -> None:
        put_mock.return_value = "s3://foo/bar"
        other_newsletter = FakeData.newsletter(
            owning_account=self.newsletter.owning_account
        )
        FakeData.export(newsletter=other_newsletter)
        FakeData.export(newsletter=other_newsletter)
        response = self.api_client.get(self.url)
        assert response.status_code == 200, str(response.content)
        assert isinstance(response.json(), dict), response.json()
        assert response.json()["count"] == 0, response.json()

    @mock.patch("emails.models.export.actions.s3.put")
    def test_export_return_ids(self, put_mock: MagicMock) -> None:
        put_mock.return_value = "s3://foo/bar"
        FakeData.export(newsletter=self.newsletter)
        response = self.api_client.get(self.url)
        assert "id" in response.json()["results"][0], response.content

    @mock.patch("emails.models.export.actions.s3.put")
    def test_POST_request_of_export_api(self, put_mock: MagicMock) -> None:
        put_mock.return_value = "s3://foo/bar"
        self.assertPOSTReturnsStatusCode(
            {
                "collections": ["comments"],
            },
            201,
        )
        assert Export.objects.filter(newsletter=self.newsletter).exists()

    @mock.patch("emails.models.export.actions.s3.put")
    def test_empty_collection_POST_request_of_export_api(
        self, put_mock: MagicMock
    ) -> None:
        put_mock.return_value = "s3://foo/bar"
        self.assertPOSTReturnsStatusCode(
            {
                "collections": [],
            },
            400,
        )

    @mock.patch("emails.models.export.actions.s3.put")
    def test_export_requester_matches_account(self, put_mock: MagicMock) -> None:
        put_mock.return_value = "s3://foo/bar"
        self.user = FakeData.user()
        self.account = Account.objects.get(user_id=self.user.id)
        self.newsletter = FakeData.newsletter(owning_account=self.account)
        self.account.email_address = "[email protected]"
        self.account.save()
        FakeData.export(newsletter=self.newsletter, requester=self.account)
        export = Export.objects.filter(newsletter=self.newsletter).first()
        self.assertEqual(
            export.requester.email_address,
            self.account.email_address,
            "Requester does not match",
        )

Beyond the obvious size of the file, there are a couple other things that made this trickier:

• We're overriding settings in some tests, mocking in others, and inheriting from a custom class.
• We're using FakeData, a sort of factory-pattern generator that would ideally be replaced with pytest fixtures.

Cursor did pretty well, though the final output (seen below) required a couple tweaks on my end:

import json

import pytest
from django.test import override_settings

from emails.models.account.model import Account
from emails.models.export.model import Export
from emails.tests.utils import FakeData


@pytest.fixture
def url():
    return "/v1/exports"


@pytest.fixture
def mocked_s3_put(mocker):
    return mocker.patch(
        "emails.models.export.actions.s3.put", return_value="s3://foo/bar"
    )


@override_settings(IS_TEST=False, DEBUG=False)
def test_list_on_v2(logged_in_client, url, newsletter):
    account = newsletter.owning_account
    account.billing_type = Account.BillingType.V2
    account.save()
    response = logged_in_client.get(url)
    assert response.status_code == 403, response.content
    assert "upgrade your account" in response.json()["detail"]


def test_list(logged_in_client, url, newsletter, mocked_s3_put):
    FakeData.export(newsletter=newsletter)
    FakeData.export(newsletter=newsletter)
    response = logged_in_client.get(url)
    assert response.status_code == 200, str(response.content)
    assert isinstance(response.json(), dict), response.json()
    assert response.json()["count"] == 2, response.json()


def test_list_should_not_pollute_across_newsletters(
    logged_in_client, url, newsletter, mocked_s3_put
):
    other_newsletter = FakeData.newsletter(owning_account=newsletter.owning_account)
    FakeData.export(newsletter=other_newsletter)
    FakeData.export(newsletter=other_newsletter)
    response = logged_in_client.get(url)
    assert response.status_code == 200, str(response.content)
    assert isinstance(response.json(), dict), response.json()
    assert response.json()["count"] == 0, response.json()


def test_export_return_ids(logged_in_client, url, newsletter, mocked_s3_put):
    FakeData.export(newsletter=newsletter)
    response = logged_in_client.get(url)
    assert "id" in response.json()["results"][0], response.content


def test_POST_request_of_export_api(logged_in_client, url, newsletter, mocked_s3_put):
    response = logged_in_client.post(
        url, json.dumps({"collections": ["comments"]}), content_type="application/json"
    )
    assert response.status_code == 201, response.json()
    assert Export.objects.filter(newsletter=newsletter).exists()


def test_empty_collection_POST_request_of_export_api(
    logged_in_client, url, mocked_s3_put
):
    response = logged_in_client.post(
        url, json.dumps({"collections": []}), content_type="application/json"
    )
    assert response.status_code == 400


def test_export_requester_matches_account(db, mocked_s3_put):
    user = FakeData.user()
    account = Account.objects.get(user_id=user.id)
    newsletter = FakeData.newsletter(owning_account=account)
    account.email_address = "[email protected]"
    account.save()
    FakeData.export(newsletter=newsletter, requester=account)
    export = Export.objects.filter(newsletter=newsletter).first()
    assert (
        export.requester.email_address == account.email_address
    ), "Requester does not match"

Some notes on its efforts:

• It got the fixture-mocking API down first try, which is more than I can say for myself (I always end up trying to create a MagicMock or forget to start it or some other such trivial error)
• It originally put those final three tests in a completely empty class for reasons passing understanding; I amended the prompt to say no classes and it fixed it
• The final test (test_export_requester_matches_account) was failing because it did not place the db fixture, which I had to fix manually.

Overall, I was impressed. Not only did Cursor pass the first bar of "actually accomplishing the task", it also passed the second bar of "accomplishing the task faster than it would have taken me."

Not unlike Icarus, I was emboldened by these results. I tried two other genres of task that I suspected Cursor would be able to handle well:

1. Localizing a footer in a handful of different languages. This was, I presume, made easier by the fact that the relevant file already had localization logic.

2. Adding some logic to a Django admin form. Not only did this work, this was the clearest example of Cursor doing something that I didn't know how to do off the top of my head. (It's a trivial change, and I would have been able to figure it out in five minutes of Googling — but ten seconds is better than five minutes.)

Cursor did both! ...and then it choked on some more complicated feature work that spanned multiple files. Which is fine: a tool does not need to be flawless to be useful, and Cursor proved itself useful.

I can say, having finally played enough to burn out / quit, that Farm RPG is pretty much the ideal idle game for me. It is simple, pleasant, non-predatory, and strikes the perfect balance of encouraging habitual play without making you feel bad for not playing it. There's a huge amount of content, the game changes a meaningful and interesting amount over the course of your playthrough, and it rewards creativity & pathing without really requiring you to bust out a spreadsheet.

If there's any flaw in it (besides a meaningful dearth of social aspects, which in my case could almost be argued as a boon), it's that the endgame is toilsome. My last six months of playing Farm RPG were all the same: log in in the morning, spend five minutes doing my daily chores, make a bit of monotone progress on whatever the current milestone was (which was always rote), and then log out. It's hard to hold this against the game specifically because all idle-games seem to struggle with this prolonged endgame — but it's worth calling out nonetheless.

Previous page · Next page