Justin Duke

How I cut my Webpack bundle size in half

In the fall, a young man’s fancy lightly turns to thoughts of front-end performance.

When I initially built out Buttondown, I was focused on two aspects above all else:

  1. It being built quickly.
  2. It working reasonably well.

Notably excluded from that list is performance. Buttondown isn’t a slow app, but it is a heavy one: the bundle size while developing is measured in megabytes, and there’s a non-trivial loading time for first-time users.

Now that the core feature base has stabilized and nothing is particularly in an “on fire” state, I wanted to turn my eye towards maintenance work, and a big piece of that was seeing what I could do to shrink that bundle.

Read more →

There are still dragons everywhere

I’ve been programming for the better part of a decade, and I’ve been programming in Python for the better part of that better part.

As loathe as I am to assume the mantel of “X Engineer”, if I were to describe my career in relation to any technology it would be “Python Engineer”.

It is the language I feel most comfortable with; it is the language I reach to first when starting a new project; it is the language I write daily. It is as frictionless as English.

And yet I still find myself losing hours to things I thought I had mastered.


The other night, I played with the idea of bumping Buttondown to Python 3.6 from 3.5 — I wanted cooler types and faster dictionaries, and it seemed like a painless process, so I ran brew upgrade python and was off to the races.

Except something broke: there was an issue with how I was using grequests that was fine in 3.5 but not 3.6, and I decided to revert.

So I hit up pyenv to try and grab 3.5 (since it was no longer the latest thing installed on my machine) and use that to recreate my virtualenv.

Except now psycopg2 was broken for some reason?

It kept on throwing a bizarre error that had like, three matches on Google total:

ImportError: datetime initialization failed

So, I do what I’ve learned to do: nuke everything and start over.

Except now virtualenv is broken, looking for a global python installation that doesn’t exist: no such file or directory, it snaps back at me.

After a few hours of fruitless Googling and tinkering, I do the thing I’m not supposed to do: I just re-install the old version of Python with brew:

brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/ec545d45d4512ace3570782283df4ecda6bb0044/Formula/python3.rb

And suddenly everything works. I recreate the virtualenv; I reinstall the dependencies; I get back to work.


There’s no big lesson here. I probably should have stuck with 3.6 and figured out the grequests issue; I probably should have been more disciplined with installing 3.6 through pyenv and managing my environments.

But I think it feels good to get lost in a sea of arcana and development hell for an afternoon. I mean, it sucks in the moment — there’s no denying that. But once the moment passes, you’re reminded about how much left there is to learn and to master.

Permalink

App sizes are under control, they just aren't under scrutiny

I wrote at the first of last month about bloat in apps, in response to a thoughtful piece by Ben Sandofsky about how he minimized bloat in Halide.

I had some misgivings with the post and the ideas it espoused — namely, that “bloat is bad” is a reductionist truism — but still admired the approach Sandofsky took and the clear tradeoffs and sacrifices he discussed.

This post by Trevor Elkins making the rounds today has the same message, though it lacks the nuance and pragmatism:

As app developers, we should be more conscious of the space we use. Take some time to remove the cruft that builds up and push back against needless waste.

So really, how does an app that occasionally sends me a connection request and recruiter spam take up 275MB? Maybe I’ll do an analysis at some point, but for now it’s deleted.

Trevor works for Kayak, whose iOS app weighs in at 176 MB.

Perhaps Trevor is actively working on reducing the size of the app bundle, in which case he knows how difficult it is to make decisions like do we spend two weeks reimplementing this framework or just eat the 5MB it adds to our final app size?

Or perhaps he isn’t, but he’s talked with fellow Kayak engineers who have determined that the bloat is worth the cost — maybe they’ve got a lot of high-resolution assets, or have decided that the development velocity a framework-heavy approach enables is worth the drop-off in installs.

Or perhaps he didn’t even know Kayak’s app size is so large in the first place, because it’s a factor that doesn’t impact the majority of users after the initial download. 1

Either way, it’s kinda shitty to insiniuate that an app is bloated due to incompetence; it’s kinda shitty to insiniuate that an app is bloated due to indifference.

Everything is a trade-off. Some companies don’t think it’s worth it to slim down their bundle sizes; some companies do. It’s disingenuous to pretend otherwise.

(For what it’s worth, I think the Kayak iOS app is great.)

Permalink

A nice little list of valid and invalid email addresses

Note This is a growing and evolving list! If you see anything that I'm missing, email me and I'll add it.

I’ve been spending a lot of time working on Buttondown lately and, as you might expect for a newsletter app, lots of this work revolves around dealing with email addresses.

In an effort to save myself and others some time, I’m trying to collate a list of usefully distinct and weird invalid and valid addresses for test harnesses and such. Below is what I have so far: it’s not a complete set, but it’s comprehensive enough to handle most of the weirdnesses out there.

Read more →

Code Reviews

I’ve seen three different iterations of “code review culture”, and all of them have been positive with minor tweaks and changes. What follows is a general list of observations and advice:

“Finding and preventing bugs” is a secondary goal of code reviews, not a primary one.

You shouldn’t be relying on your code reviewers to find bugs; it’s nice to have an extra set of eyes who can point them out, but it’s your job as the implementer and tester to ensure correctness.

Everyone and everything should be subject to code reviews.

An atmosphere where senior engineers don’t have to go through CRs is bad: it implies poor things about your development hierarchy, like the idea that seniority means your code is immune to critique.

An atmosphere where critical bug fixes don’t have to go through CRs is bad: it implies you don’t have a testing/rollback framework set up to avoid the need to push unreviewed code.

Code reviews should not be more than X lines of code.

Anything more than X and diminishing returns set in. What X is varies from language to language and context to context, but 200 is a pretty good number. (If your immediate reaction to this is “but most changes are more than 200 lines of code”, my immediate reaction is: nah.)

Super-basic style comments should not be in code reviews.

Like, they will be — someone (you!) is going to leave out a semicolon or mess up spacing, and that should be addressed in a review comment.

But if that’s the norm and not a occassional slip-up, then that’s a sign that you don’t have a documented code style. Whenever you or someone else comments we prefer null coalescing to ternaries here or whatever, that comment should go into a wiki and a lintfile so such issues can be automatically detected. 1 The goal is to avoid work that can (and should) be automated so reviewers can focus on the important stuff.

(Automation is always preferable to documentation: it’s nice to have a central resource for these things, but it’s much nicer to have that central resource be your IDE or a pre-commit hook.)

Don’t be an asshole.

People have a wide range of how they internalize code reviews, and it is often hard to separate critique of code from critique of the coder. The best way to do this is to approach code reviews not as an adversarial process, but as a collaborative discovery of the best possible implementation.

A great way to do this that some folks gloss over: write positive comments in CRs. Positive reinforcement is just as helpful as constructive criticism: praising someone’s tests or the way they structured some tricky business logic doesn’t just make them feel good, it enforces that behavior for themselves and others.

Be prompt.

The largest personal and institutional grievance folks have with code reviews is that they gum up the works and increase the time it takes to deploy code. Set team-wide practices on healthy expectations for when code should be reviewed and stick to them.

Be rigorous.

The woooooooorst thing to experience as a review-ee:

  1. You post some code.
  2. A reviewer makes comments A, B, and C.
  3. You address those comments and post the new code, asking for a new round of review.
  4. The same reviewer makes comments D, E, and F, all of which could have been made during the original review pass.

The woooooooorst thing to experience as a reviewer:

  1. Someone posts some code.
  2. You make comments A, B, C, and D.
  3. They address only A and D and post the new code, asking for another round of review.

Keep at it!

Code reviews can be long and they can be difficult but they are good for the long term health of your code base, just like building out a test suite or documentation.

Some further reading:

Thanks to Kelly Sutton, Iheanyi Ekechukwu, and Chad Little for reviewing this post. (And thanks to Michelle, Daniel, Andy, and countless others for reviewing my code over the years, and teaching me how to review well. 2)

Permalink

Django vs. Flask

A great, thorough writeup on Django vs. Flask has been floating around the past few days. It’s a great technical breakdown, and I agree with the conclusion:

There’s an informal perception that Batteries included may mean a growing list of ill-maintained API’s that get hooked into every request. In the case of Django, everything works across the board. When an internal Django API changes, Django’s testsuites to break and the appropriate changes are made. So stuff integrates. This is something that’s harder to do when there’s a lot of packages from different authors who have to wait for fixes to be released in Flask’s ecosystem.

However, it also glosses over what is probably the strongest weapon in Django’s arsenal: Django Rest Framework. At this point, I essentially consider DRF a first-party package: it is so completely essential to my Python web development toolbox I can’t imagine working without it.

Having such a powerful, extensible approach to REST (and everything that entails: serialization, permissions, filtering…) is invaluable. Flask is great for toy projects, but for a modern application DRF is unreplaceable, and thus so is Django.

(Another important thing that Django does: it forces a certain level of structure and organization to your application. I used to think this was either unnecessarily or actively harmful: now, having seen my fair share of eldritch Python codebases, I know better.)

Permalink

Agility

Jasdev had a great post on moving too fast:

But, Agile almost entirely misses the other side of the coin: regimented rest. We can’t expect teams to knock out a monotonically increasing amount of story points, sprint after sprint. Sometimes, you have to move slower now to move faster later.

Somewhat adjacent to this is the best advice I’ve ever received as a developer:

Focus on being the highest order derivative you can be.

As an entry-level developer, you’re a simple function: you take time as an input and produce business value as an output. 1

Then as you mature as a developer, your work doesn’t directly produce business value: it produces an increase at the rate at which others can produce business value. Suddenly, you’re a first-order derivative.

And then you go further than that, becoming a second-order derivative: transcending from velocity to acceleration. Your work helps others increase the rate of change.

And so on and so forth.

I think this trajectory of increasing abstraction — stretching out your objectives over a longer and longer timespan — is natural, and good. Don’t worry about how to accomplish this week’s work; worry about how to accomplish this year’s work. Decoupling yourself from short-term inputs, like speed and time, is good: it’s a sign that your concerns are larger than the day-to-day.

Put more succinctly: agility isn’t about distance — it’s about nimbleness.

Or, to quote Murakami in What I Talk About When I Talk About Running:

Sometimes taking time is actually a shortcut.

Permalink

Hugo's internal templates, demystified

In my effort to throw a new coat of paint on this site, I was diving into Hugo’s internal templates to see how to improve their builtin pagination. It was invoked by calling _internal/pagination.html, so I assumed the template was something along the lines of pagination.html.

Turns out all the internal templates are hard-coded in the Hugo library itself, in a file named template_embedded.go. Rather than filing a snarky comment on the wisdom of this approach, I thought it might be helpful to list them out, as it’s definitely annoying to read in the source file.

Read more →

In praise of bloat

Ben Sandofsky, the creator of Halide (and erstwhile lead of Twitter’s iOS/Mac apps) wrote a worthwhile piece about getting his new app’s size to a mere 11MB. 1 There are a bunch of solid, universal pieces of advice (don’t create your own layout engine; compress and catalog assets), some trickier ones (avoid library bloat; eschew analytics), and a questionable kicker:

There really is one weird trick to lose size: focus on your customers.

I am skeptical about this last point in much the same way I am skeptical of folks saying the best way to lose weight is to eat healthy foods: well, sure, but what does that look like in practice?

Obviously focusing on your customers is very good, but I don’t think the truism of customer needs is paramount necessarily translates to slimming down your app is paramount.

This piece reminded me of an experiment Segment did last October, where they bought a random app and experimented with the app size to see how it affected downloads:

The first data point is insignificant (you can’t meaningfully compare download numbers between 2012 and 2016), but the latter three are pretty interesting, and suggest a linear (albeit tenuous) relationship between app size and install rate, to the tune of each megabyte costing the developer 0.45% of conversion rate.

That’s non-trivial! But if you’re an indie developer (or even just a pragmatic one), you have so much on your plate. You’ve got to build out features, you’ve got to iron out bugs, you’ve got to build out a marketing pipeline. All of these things are important, and some of these things might be solveable with a library or framework, which frees up your time to work on (potentially more important) other things.

Let’s take a completely arbitrary and abstract example of what I’m saying.

It’s four weeks before launch and you unexpectedly have a week of slack time. You can do one of these two things with your free time:

  • Removing a 5MB library and replacing it with a home-brew version (bumping up install rate by, according to Segment, ~2.5%)
  • Implementing a feature that bumps up install rate by 5%

The choice should be obvious, right?

Obviously nothing’s this clear cut: you’re never deciding between two things, you’re deciding between twenty things, and the time and value of these tasks are never quite as apparent as you’d like them to be.

But if you’re a small developer, every task you work on is a question of ROI: you only have so many battles you can pick. Sometimes fighting bloat is the right choice (and hey, it lets you write blog posts about how you fought bloat, which always makes for good marketing); sometimes it’s not. 2

I am skeptical that fighting app bloat is always the battle worth picking, but I think we can all agree that it’s at least a battle worth considering.

Permalink

What Vue needs next

Note This is gonna sound like a negative essay. Which it kinda is. But it's very much not a condemnation of Vue!

Vue is great. You should try Vue.

If I had written What Vue Doesn't Need it would be an incredibly long post! This is just things I wish were better about it.

I’ve been using Vue a lot recently on a few test projects, and most notably on Buttondown.

Buttondown’s frontend is 100% Vue: around twenty screens (which are themselves components) and around thirty miscellaneous components, tied together with Vuex and Vue-Router.

Overall, Vue is a really great tool, and the first word I’d use to describe it is pragmatic. Almost everything feels and acts sensible; there are tremendously few times that I am confused or surprised by how it works, which is legitimately novel (and wonderous) after the hours and days and weeks I’ve spent pulling my hair out dealing with random React and Webpack arcana.

Still, there are some bugaboos that I think arise from Vue’s current niche as a “lightweight” approach to creating a functional SPA, in much the same way that Flask carved out a niche as being a lightweight alternative to Django.

Read more →
© 2017 Justin Duke • All rights reserved • I hope you have a nice day.