Archive for the ‘Web’ Category

Fixing memory leaks in web applications

Part of the bargain we struck when we switched from building server-rendered websites to client-rendered SPAs is that we suddenly had to take a lot more care with the resources on the user’s device. Don’t block the UI thread, don’t make the laptop’s fan spin, don’t drain the phone’s battery, etc. We traded better interactivity and “app-like” behavior for a new class of problems that don’t really exist in the server-rendered world.

One of these problems is memory leaks. A poorly-coded SPA can easily eat up megabytes or even gigabytes of memory, continuing to gobble up more and more resources, even as it’s sitting innocently in a background tab. At this point, the page might start to slow to a crawl, or the browser may just terminate the tab and you’ll see Chrome’s familiar “Aw, snap!” page.

Chrome page saying "Aw snap! Something went wrong while displaying this web page."

(Of course, a server-rendered website can also leak memory on the server side. But it’s extremely unlikely to leak memory on the client side, since the browser will clear the memory every time you navigate between pages.)

The subject of memory leaks is not well-covered in the web development literature. And yet, I’m pretty sure that most non-trivial SPAs leak memory, unless the team behind them has a robust infrastructure for catching and fixing memory leaks. It’s just far too easy in JavaScript to accidentally allocate some memory and forget to clean it up.

So why is so little written about memory leaks? My guesses:

  • Lack of complaints: most users are not diligently watching Task Manager while they surf the web. Typically, you won’t hear about it from your users unless the leak is so bad that the tab is crashing or the app is slowing down.
  • Lack of data: the Chrome team doesn’t provide data about how much memory websites are using in the wild. Nor are websites often measuring this themselves.
  • Lack of tooling: it’s still not easy to identify or fix memory leaks with existing tooling.
  • Lack of caring: browsers are pretty good at killing tabs that consume too much memory. Plus people seem to blame the browser rather than the websites.

In this post, I’d like to share some of my experience fixing memory leaks in web applications, and provide some examples of how to effectively track them down.

Anatomy of a memory leak

Modern web app frameworks like React, Vue, and Svelte use a component-based model. Within this model, the most common way to introduce a memory leak is something like this:

window.addEventListener('message', this.onMessage.bind(this));

That’s it. That’s all it takes to introduce a memory leak. If you call addEventListener on some global object (the window, the <body>, etc.) and then forget to clean it up with removeEventListener when the component is unmounted, then you’ve created a memory leak.

Worse, you’ve just leaked your entire component. Because this.onMessage is bound to this, the component has leaked. So have all of its child components. And very likely, so have all the DOM nodes associated with the components. This can get very bad very fast.

Here is the fix:

// Mount phase
this.onMessage = this.onMessage.bind(this);
window.addEventListener('message', this.onMessage);

// Unmount phase
window.removeEventListener('message', this.onMessage);

Note that we saved a reference to the bound onMessage function. You have to pass in exactly the same function to removeEventListener that you passed in to addEventListener, or else it won’t work.

The memory leak landscape

In my experience, the most common sources of memory leaks are APIs like these:

  1. addEventListener. This is the most common one. Call removeEventListener to clean it up.
  2. setTimeout / setInterval. If you create a recurring timer (e.g. to run every 30 seconds), then you need to clean it up with clearTimeout or clearInterval. (setTimeout can leak if it’s used like setInterval – i.e., scheduling a new setTimeout inside of the setTimeout callback.)
  3. IntersectionObserver, ResizeObserver, MutationObserver, etc. These new-ish APIs are very convenient, but they are also likely to leak. If you create one inside of a component, and it’s attached to a globally-available element, then you need to call disconnect() to clean them up. (Note that DOM nodes which are garbage-collected will have their listeners and observers garbage-collected as well. So typically, you only need to worry about global elements, e.g. the <body>, the document, an omnipresent header/footer element, etc.)
  4. Promises, Observables, EventEmitters, etc. Any programming model where you’re setting up a listener can leak memory if you forget to stop listening. (A Promise can leak if it’s never resolved or rejected, in which case any .then() callbacks attached to it will leak.)
  5. Global object stores. With something like Redux the state is global, so if you’re not careful, you can just keep appending memory to it and it will never get cleaned up.
  6. Infinite DOM growth. If you implement an infinite scrolling list without virtualization, then the number of DOM nodes will grow without bound.

Of course, there are plenty of other ways to leak memory, but these are the most common ones I’ve seen.

Identifying memory leaks

This is the hard part. I’ll start off by saying that I just don’t think any of the tooling out there is very good. I’ve tried Firefox’s memory tool, the Edge and IE memory tools, and even Windows Performance Analyzer. The best-in-class is still the Chrome Developer Tools, but it has a lot of rough edges that are worth knowing about.

In the Chrome DevTools, our main tool of choice is going to be the “heap snapshot” tool in the “Memory” tab. There are other memory tools in Chrome, but I don’t find them very helpful for identifying leaks.

Screenshot of the Chrome DevTools Memory tab with the Heap Snapshot tool

The Heap Snapshot tool allows you to take a memory capture of the main thread or web workers or iframes.

When you click the “take snapshot” button, you’ve captured all the live objects in a particular JavaScript VM on that web page. This includes objects referenced by the window, objects referenced by setInterval callbacks, etc. Think of it as a frozen moment in time representing all the memory used by that web page.

The next step is to reproduce some scenario that you think may be leaking – for instance, opening and closing a modal dialog. Once the dialog is closed, you’d expect memory to return back to the previous level. So you take another snapshot, and then diff it with the previous snapshot. This diffing is really the killer feature of the tool.

Diagram showing a first heapsnapshot followed by a leaking scenario followed by a second heap snapshot which should be equal to the first

However, there are a few limitations of the tool that you should be aware of:

  1. Even if you click the little “collect garbage” button, you may need to take a few consecutive snapshots for Chrome to truly clean up the unreferenced memory. In my experience, three should be enough. (Check the total memory size of each snapshot – it should eventually stabilize.)
  2. If you have web workers, service workers, iframes, shared workers, etc., then this memory will not be represented in the heap snapshot, because it lives in another JavaScript VM. You can capture this memory if you want, but just be sure you know which one you’re measuring.
  3. Sometimes the snapshotter will get stuck or crash. In that case, just close the browser tab and start all over again.

At this point, if your app is non-trivial, then you’re probably going to see a lot of leaking objects between the two snapshots. This is where things get tricky, because not all of these are true leaks. Many of these are just normal usage – some object gets de-allocated in favor of another one, something gets cached in a way that will get cleaned up later, etc.

Cutting through the noise

I’ve found that the best way to cut through the noise is to repeat the leaking scenario several times. For instance, instead of just opening and closing a modal dialog once, you might open and close it 7 times. (7 is a nice conspicuous prime number.) Then you can check the heap snapshot diff to see if any objects leaked 7 times. (Or 14 times, or 21 times.)

Screenshot of the Chrome DevTools heap snapshot diff showing six heap snapshot captures with several objects leaking 7 times

A heap snapshot diff. Note that we’re comparing snapshot #6 to snapshot #3, because I take three captures in a row to allow more garbage collection to occur. Also note that several objects are leaking 7 times.

(Another helpful technique is to run through the scenario once before recording the first snapshot. Especially if you are using a lot of code-splitting, then your scenario is likely to have a one-time memory cost of loading the necessary JavaScript modules.)

At this point, you might wonder why we should sort by the number of objects rather than the total memory. Intuitively, we’re trying to reduce the amount of memory leaking, so shouldn’t we focus on the total memory usage? Well, this doesn’t work very well, for an important reason.

When something is leaking, it’s because (to paraphrase Joe Armstrong) you’re holding onto the banana, but you ended up getting the banana, the gorilla holding the banana, and the whole jungle. If you measure based on total bytes, you’re measuring the jungle, not the banana.

Let’s go back to the addEventListener example above. The source of the leak is an event listener, which is referencing a function, which references a component, which probably references a ton of stuff like arrays, strings, and objects.

If you sort the heap snapshot diff by total memory, then it’s going to show you a bunch of arrays, strings, and objects – most of which are probably unrelated to the leak. What you really want to find is the event listener, but this takes up a minuscule amount of memory compared to the stuff it’s referencing. To fix the leak, you want to find the banana, not the jungle.

So if you sort by the number of objects leaked, you’re going to see 7 event listeners. And maybe 7 components, and 14 sub-components, or something like that. That “7” should stand out like a sore thumb, since it’s such an unusual number. And no matter how many times you repeat the scenario, you should see exactly that number of objects leaking. This is how you can quickly find the source of the leak.

Walking the retainer tree

The heap snapshot diff will also show you a “retainer” chain showing which objects are pointing to which other objects and thus keeping the memory alive. This is how you can figure out where the leaking object was allocated.

Screenshot of a retainer chain showing someObject referenced by a closure referenced by an event listener

The retainer chain shows you which object is referencing the leaked object. The way to read it is that each object is referenced by the object below it.

In the above example, there is a variable called someObject which is referenced by a closure (aka “context”), which is referenced by an event listener. If you click the source link, it will take you to the JavaScript declaration, which is fairly straightforward:

class SomeObject () { /* ... */ }

const someObject = new SomeObject();
const onMessage = () => { /* ... */ };
window.addEventListener('message', onMessage);

In the above example, the “context” is the onMessage closure which references the someObject variable. (This is a contrived example; real memory leaks can be much less obvious!)

But the heap snapshotting tool has several limitations:

  1. If you save and re-load the snapshot file, then you will lose all the file references to where the object was allocated. So for instance, you won’t see that the event listener’s closure comes from line 22 of foo.js. Since this is really critical information, it’s almost useless to save and send heap snapshot files.
  2. If there are WeakMaps involved, then Chrome will show you those references even though they don’t really matter – those objects would be de-allocated as soon as the other references are cleaned up. So they are just noise.
  3. Chrome classifies the objects by their prototype. So the more you use actual classes/functions and the less you use anonymous objects, the easier it will be to see what exactly is leaking. As an example, imagine if our leak was somehow due to an object rather than an EventListener. Since object is extremely generic, we’re unlikely to see exactly 7 of them leaking.

This is my basic strategy for identifying memory leaks. I’ve successfully used this technique to find dozens of memory leaks in the past.

This guide is just the start, though – beyond this, you will also have to be handy with setting breakpoints, logging, and testing your fix to see if it resolves the leak. Unfortunately, this is just inherently a time-consuming process.

Automated memory leak analysis

I’ll precede this by saying that I haven’t found a great way to automate the detection of memory leaks. Chrome has a non-standard performance.memory API, but for privacy reasons it doesn’t have a very precise granularity, so you can’t really use it in production to identify leaks. The W3C Web Performance Working Group has discussed memory tooling in the past, but has yet to agree on a new standard to replace this API.

In a lab or synthetic testing environment, you can increase the granularity on this API by using the Chrome flag --enable-precise-memory-info. You can also create heap snapshot files by calling the proprietary Chromedriver command :takeHeapSnapshot. This has the same limitation mentioned above, though – you probably want to take three in a row and discard the first two.

Since event listeners are the most common source of memory leaks, another technique that I’ve used is to monkey-patch the addEventListener and removeEventListener APIs to count the references and ensure they return to zero. Here is an example of how to do that.

In the Chrome DevTools, you can also use the proprietary getEventListeners() API to see the event listeners attached to a particular element. Note that this can only be used in DevTools, though.

Update: Mathias Bynens has informed me of another useful DevTools API: queryObjects(), which can show you all objects created with a particular constructor. Christoph Guttandin also has an interesting blog post about using this API for automated memory leak detection in Puppeteer.

Summary

The state of finding and fixing memory leaks in web apps is still fairly rudimentary. In this blog post, I’ve covered some of the techniques that have worked for me, but admittedly this is still a difficult and time-consuming process.

As with most performance problems, an ounce of prevention can be worth a pound of cure. You might find it worthwhile to put synthetic testing in place rather than trying to debug a memory leak after the fact. Especially if there are several leaks on a page, it will probably turn into an onion-peeling exercise – you fix one leak, then find another, then repeat (while weeping the whole time!). Code review can also help catch common memory leak patterns, if you know what to look for.

JavaScript is a memory-safe language, so it’s somewhat ironic how easy it is to leak memory in a web application. Part of this is just inherent to UI design, though – we need to listen for mouse events, scroll events, keyboard events, etc., and these are all patterns that can easily lead to memory leaks. But by trying to keep our web applications’ memory usage low, we can improve runtime performance, avoid crashes, and be respectful of resource constraints on the user’s device.

Thanks to Jake Archibald and Yang Guo for feedback on a draft of this post. And thanks to Dinko Bajric for inventing the “choose a prime number” technique, which I’ve found so helpful in memory leak analysis.

Browsers, input events, and frame throttling

If there’s one thing I’ve learned about web performance, it’s that you have to approach it with a sense of open-mindedness and humility. Otherwise, prepare to be humbled.

Just as soon as you think you’ve got it all figured out, poof! Browsers change their implementation. Or poof! The spec changes. Or poof! You just flat-out turn out to be wrong. So you have to constantly test and revalidate your assumptions.

In a recent post, I suggested that pointermove events fire more frequently than requestAnimationFrame, and so it’s a good idea to throttle them to rAF. I also rattled off some other events that may fire faster than rAF, such as scroll, wheel, touchmove, and mousemove.

Do these events actually fire faster than rAF, though? It’s an important detail! If browsers already align/throttle these events to rAF, then there’s little point in recreating that same behavior in userland. (Thankfully an extra rAF won’t add an extra frame delay, though, assuming browsers fire the rAF-aligned events right before rAF. Thanks Jake Archibald for this tip!)

TL;DR: it varies across browsers and events. I’d still recommend the rAF-throttling technique described in my previous post.

Step one: check the spec

The first question to ask is: what does the spec say?

After reading the specs for pointermove, mousemove, touchmove, scroll, and wheel, I found that the only mention of animation frame timing was in pointermove and scroll. The spec for pointermove says:

A user agent MUST fire a pointer event named pointermove when a pointer changes button state. […] These events may be coalesced or aligned to animation frame callbacks based on UA decision.

(Emphasis mine.) So browsers are not required to coalesce or align pointermove events to animation frames, but they may do so. (Presumably, this is the point of getCoalescedEvents().)

As for scroll, it’s mentioned in the event loop spec, where it says “for each fully active Document […], run the scroll steps for that Document” as part of the steps before running rAF callbacks. So on the main document at least, scroll is definitely supposed to fire before rAF.

For contrast, here’s touchmove:

A user agent must dispatch this event type to indicate when the user moves a touch point along the touch surface. […] Note that the rate at which the user agent sends touchmove events is implementation-defined, and may depend on hardware capabilities and other implementation details.

(Emphasis mine.) So this time, nothing about animation frames, and also some language about “implementation-defined.” Similarly, here’s mousemove:

The frequency rate of events while the pointing device is moved is implementation-, device-, and platform-specific, but multiple consecutive mousemove events SHOULD be fired for sustained pointer-device movement, rather than a single event for each instance of mouse movement.

(Emphasis mine.) So we’re starting to get a pretty clear picture (or a hazy one, depending on your perspective). It seems that, aside from scroll, the specs don’t have much to say about whether events should be coalesced with rAF or not.

Step two: test it

However, this doesn’t mean browsers don’t do it! After all, it’s clearly in browsers’ interests to coalesce these events to animation frames. Assuming that most web developers do the simplest possible thing and handle the events directly, then any browser that aligns with rAF will avoid some unintentional jank from noisy input events.

Do browsers actually do this, though? Thankfully Jake has written a nice demo which makes it easy to test this. I’ve also extended his demo to test scroll events. And because I apparently have way too much free time on my hands (or I just hate uncertainty when it comes to browser stuff), I went ahead and compiled the data for various browsers and OSes:

pointermove mousemove touchmove wheel scroll
Chrome 76 (Windows 10) Y* Y* N/A Y* Y
Firefox 68 (Windows 10) Y Y N/A N Y
Edge 18 (Windows 10) N N N/A N Y
Chrome 76 (macOS 10.14.6) Y* Y* N/A Y* Y
Firefox 68 (macOS 10.14.6) Y Y N/A N Y
Safari 12.1.2 (macOS 10.14.6) N/A N N/A N N
Safari Technology Preview 13.1 (macOS 10.14.6) N N N/A N N
Chrome 76 (Ubuntu 16.04) Y* Y* N/A Y* Y
Firefox 68 (Ubuntu 16.04) Y Y N/A N Y
GNOME Web 3.28.5 (Ubuntu 16.04) N/A N N/A N N
Chrome 76 (Android 6) Y N/A Y N/A Y
Firefox 68 (Android 6) N/A N/A Y N/A Y
Safari (iOS 12.4) N/A N/A Y N/A N

Abbreviations:

  • Y: Yes, events are coalesced and aligned to rAF.
  • N: No, events fire independently of and faster than rAF.
  • N/A: Event doesn’t apply to this device/browser.
  • *: Except when Dev Tools are opened, apparently.

Conclusion

As you can see from the data, there is a lot of variance in terms of which events and browsers align to rAF. Although for the most part, it seems consistent within browser engines (e.g. GNOME Web is a WebKit-based browser, and it patterns with macOS Safari). Note though that I only tested a regular mouse or trackpad, not exotic input devices such as a Wacom stylus, Surface Pen, etc.

Given this data, I would take the cautious approach and still do the manual rAF-throttling as described in my previous blog post. It has the upside of being guaranteed to work roughly the same across all browsers, at the cost of some extra bookkeeping. [1]

Depending on your supported browser matrix, though, and depending on when you’re reading this (maybe at a point in the future when all browser input events are rAF-aligned!), then you may just handle the input directly and trust the browser to align it to rAF. [2]

Thanks to Ben Kelly and Jake Archibald for feedback on a draft of this blog post. Thanks also to Jake for clueing me in to this rAF-throttling business in the first place.

Footnotes

1. Interestingly, in the case of pointermove at least, the browser behavior can be feature-detected by checking getCoalescedEvents (i.e. Firefox and Chrome have it, Edge and Safari Technology Preview don’t). So you can use PointerEvent.prototype.getCoalescedEvents as a feature check. But there’s little point in feature-detecting, since manual rAF-throttling doesn’t add an extra frame delay in browsers that already rAF-align.

2. Jake also pointed me to an interesting detail: “Although these events are synced to rendering, they’ll flush if another non-synced event happens.” So for instance, keyboard events will interfere with pointermove and cause them to no longer sync to rAF, which you can reproduce in Jake’s demo by typing on the keyboard and moving the mouse at the same time. Another good reason to just rAF-throttle and be sure!

High-performance input handling on the web

Update: In a follow-up post, I explore some of the subtleties across browsers in how they fire input events.

There is a class of UI performance problems that arise from the following situation: An input event is firing faster than the browser can paint frames.

Several events can fit this description:

  • scroll
  • wheel
  • mousemove
  • touchmove
  • pointermove
  • etc.

Intuitively, it makes sense why this would happen. A user can jiggle their mouse and deliver precise x/y updates faster than the browser can paint frames, especially if the UI thread is busy and thus the framerate is being throttled (also known as “jank”).

Screenshot of Chrome Dev Tools showing that a long frame of 546ms can contain as many as four pointermove events

In the above screenshot, pointermove events are firing faster than the framerate can keep up.[1] This can also happen for scroll events, touch events, etc.

Update: In Chrome, pointermove is actually supposed to align/throttle to requestAnimationFrame automatically, but there is a bug where it behaves differently with Dev Tools open.

The performance problem occurs when the developer naïvely chooses to handle the input directly:

element.addEventListener('pointermove', () => {
  doExpensiveOperation()
})

In a previous post, I discussed Lodash’s debounce and throttle functions, which I find very useful for these kinds of situations. Recently however, I found a pattern I like even better, so I want to discuss that here.

Understanding the event loop

Let’s take a step back. What exactly are we trying to achieve here? Well, we want the browser to do only the work necessary to paint the frames that it’s able to paint. For instance, in the case of a pointermove event, we may want to update the x/y coordinates of an element rendered to the DOM.

The problem with Lodash’s throttle()/debounce() is that we would have to choose an arbitrary delay (e.g. 20 milliseconds or 50 milliseconds), which may end up being faster or slower than the browser is actually able to paint, depending on the device and browser. So really, we want to throttle to requestAnimationFrame():

element.addEventListener('pointermove', () => {
  requestAnimationFrame(doExpensiveOperation)
})

With the above code, we are at least aligning our work with the browser’s event loop, i.e. firing right before style and layout are calculated.

However, even this is not really ideal. Imagine that a pointermove event fires three times for every frame. In that case, we will essentially do three times the necessary work on every frame:

Chrome Dev Tools screenshot showing an 82 millisecond frame where there are three pointermove events queued by requestAnimationFrame inside of the frame

This may be harmless if the code is fast enough, or if it’s only writing to the DOM. However, if it’s both writing to and reading from the DOM, then we will end up with the classic layout thrashing scenario,[2] and our rAF-based solution is actually no better than handling the input directly, because we recalculate the style and layout for every pointermove event.

Chrome Dev Tools screenshot of layout thrashing, showing two pointermove events with large Layout blocks and the text "Forced reflow is a likely performance bottleneck"

Note the style and layout recalculations in the purple blocks, which Chrome marks with a red triangle and a warning about “forced reflow.”

Throttling based on framerate

Again, let’s take a step back and figure out what we’re trying to do. If the user is dragging their finger across the screen, and pointermove fires 3 times for every frame, then we actually don’t care about the first and second events. We only care about the third one, because that’s the one we need to paint.

So let’s only run the final callback before each requestAnimationFrame. This pattern will work nicely:

function throttleRAF () {
  let queuedCallback
  return callback => {
    if (!queuedCallback) {
      requestAnimationFrame(() => {
        const cb = queuedCallback
        queuedCallback = null
        cb()
      })
    }
    queuedCallback = callback
  }
}

We could also use cancelAnimationFrame for this, but I prefer the above solution because it’s calling fewer DOM APIs. (It only calls requestAnimationFrame() once per frame.)

This is nice, but at this point we can still optimize it further. Recall that we want to avoid layout thrashing, which means we want to batch all of our reads and writes to avoid unnecessary recalculations.

In “Accurately measuring layout on the web”, I explore some patterns for queuing a timer to fire after style and layout are calculated. Since writing that post, a new web standard called requestPostAnimationFrame has been proposed, and it fits the bill nicely. There is also a good polyfill called afterframe.

To best align our DOM updates with the browser’s event loop, we want to follow these simple rules:

  1. DOM writes go in requestAnimationFrame().
  2. DOM reads go in requestPostAnimationFrame().

The reason this works is because we write to the DOM right before the browser will need to calculate style and layout (in rAF), and then we read from the DOM once the calculations have been made and the DOM is “clean” (in rPAF).

If we do this correctly, then we shouldn’t see any warnings in the Chrome Dev Tools about “forced reflow” (i.e. a forced style/layout outside of the browser’s normal event loop). Instead, all layout calculations should happen during the regular event loop cycle.

Chrome Dev Tools screenshot showing one pointermove per frame and large layout blocks with no "forced reflow" warning

In the Chrome Dev Tools, you can tell the difference between a forced layout (or “reflow”) and a normal one because of the red triangle (and warning) on the purple style/layout blocks. Note that above, there are no warnings.

To accomplish this, let’s make our throttler more generic, and create one that can handle requestPostAnimationFrame as well:

function throttle (timer) {
  let queuedCallback
  return callback => {
    if (!queuedCallback) {
      timer(() => {
        const cb = queuedCallback
        queuedCallback = null
        cb()
      })
    }
    queuedCallback = callback
  }
}

Then we can create multiple throttlers based on whether we’re doing DOM reads or writes:[3]

const throttledWrite = throttle(requestAnimationFrame)
const throttledRead = throttle(requestPostAnimationFrame)

element.addEventListener('pointermove', e => {
  throttledWrite(() => {
    doWrite(e)
  })
  throttledRead(() => {
    doRead(e)
  })
})

Effectively, we have implemented something like fastdom, but using only requestAnimationFrame and requestPostAnimationFrame!

Pointer event pitfalls

The last piece of the puzzle (at least for me, while implementing a UI like this), was to avoid the pointer events polyfill. I found that, even after implementing all the above performance improvements, my UI was still janky in Firefox for Android.

After some digging with WebIDE, I found that Firefox for Android currently does not support Pointer Events, and instead only supports Touch Events. (This is similar to the current version of iOS Safari.) After profiling, I found that the polyfill itself was taking up a lot of my frame budget.

Screenshot of Firefox WebIDE showing a lot of time spent in pointer-events polyfill

So instead, I switched to handling pointer/mouse/touch events myself. Hopefully in the near future this won’t be necessary, and all browsers will support Pointer Events! We’re already close.

Here is the before-and-after of my UI, using Firefox on a Nexus 5:

 

When handling very performance-sensitive scenarios, like a UI that should respond to every pointermove event, it’s important to reduce the amount of work done on each frame. I’m sure that this polyfill is useful in other situations, but in my case, it was just adding too much overhead.

One other optimization I made was to delay updates to the store (which trigger some extra JavaScript computations) until the user’s drag had completed, instead of on every drag event. The end result is that, even on a resource-constrained device like the Nexus 5, the UI can actually keep up with the user’s finger!

Conclusion

I hope this blog post was helpful for anyone handling scroll, touchmove, pointermove, or similar input events. Thinking in terms of how I’d like to align my work with the browser’s event loop (using requestAnimationFrame and requestPostAnimationFrame) was useful for me.

Note that I’m not saying to never use Lodash’s throttle or debounce. I use them all the time! Sometimes it makes sense to just let a timer fire every n milliseconds – e.g. when debouncing window resize events. In other cases, I like using requestIdleCallback – for instance, when updating a non-critical part of the UI based on user input, like a “number of characters remaining” counter when typing into a text box.

In general, though, I hope that once requestPostAnimationFrame makes its way into browsers, web developers will start to think more purposefully about how they do UI updates, leading to fewer instances of layout thrashing. fastdom was written in 2013, and yet its lessons still apply today. Hopefully when rPAF lands, it will be much easier to use this pattern and reduce the impact of layout thrashing on web performance.

Footnotes

1. In the Pointer Events Level 2 spec, it says that pointermove events “may be coalesced or aligned to animation frame callbacks based on UA decision.” So hypothetically, a browser could throttle pointermove to fire only once per rAF (and if you need precise x/y events, e.g. for a drawing app, you can use getCoalescedEvents()). It’s not clear to me, though, that any browser actually does this. Update: see comments below, some browsers do! In any case, throttling the events to rAF in JavaScript accomplishes the same thing, regardless of UA behavior.

2. Technically, the only DOM reads that matter in the case of layout thrashing are DOM APIs that force style/layout, e.g. getBoundingClientRect() and offsetLeft. If you’re just calling getAttribute() or classList.contains(), then you’re not going to trigger style/layout recalculations.

3. Note that if you have different parts of the code that are doing separate reads/writes, then each one will need its own throttler function. Otherwise one throttler could cancel the other one out. This can be a bit tricky to get right, although to be fair the same footgun exists with Lodash’s debounce/throttle.

One year of Pinafore

Screenshot of Pinafore showing a compose input

Pinafore is a standalone web client for Mastodon, which recently hit version 1.9.0. Here are some notable new features:

It’s been about a year since I first launched Pinafore. So I’d like to reflect on where the project came from, and where I hope to take it.

Background

In 2017, I was in a funk. I had stopped contributing to the PouchDB project largely due to burnout, and for various reasons I eventually left my job at Microsoft. In the meantime, I had become enamored of Mastodon and even contributed to it, but I was feeling restless and looking for a new project.

The Mastodon codebase is extremely well-written. I’m convinced that Eugen Rochko is some kind of savant. However, I never took much of a liking to React, and I found it difficult to fix some fundamental problems in the Mastodon UI, such as offline support or the occasionally jerky scrolling. I also really missed the single-column layout of Twitter (I was never a Tweetdeck fan).

So the idea came to me to create my own Mastodon web client. I had been working on web sites for years, but aside from some small prototypes, I had never built an entire web app by myself. This was an opportunity to test some of my ideas about how a web app “should” be, leveraging my experience in web performance and standards. Also, I wanted to teach myself about accessibility, which I had never really studied before.

I knew I wanted to use Svelte, because I agreed with Rich Harris and Tom Dale that JavaScript frameworks should focus less on being runtime APIs and more on being compilers. Incidentally, I was at the same talk by Jed Schmitt that Rich mentions in this video, and it blew my mind as much as it blew his. (The difference between Rich and me is that he actually went off and built a whole framework based on it!)

I started working on Pinafore at the end of December 2017, and released it in April 2018. So after 18 months of development, I’d like to consider where Pinafore has done well and where it can improve.

Success metrics

Pinafore doesn’t have any trackers on it, so I don’t know how many people are using it. Sure, I could use a privacy-respecting tracker like Fathom, but the Mastodon community is pretty allergic to any kind of tracking, so I’ve been hesitant to add it. In any case, I don’t really care, because I would work on Pinafore regardless of how many people are using it.

However, I do get a trickle of questions and bug reports about Pinafore, and the #Pinafore hashtag is pretty active. I’ve also heard from several folks that it’s their preferred Mastodon interface. The reasons they give are usually one of the following:

  • Accessibility: I’ve focused a lot on making Pinafore work well with keyboard navigation and screen readers. (Marco Zehe‘s guidance really helped!)
  • Design: the single-column layout of Pinafore is a key differentiator with the Mastodon frontend (although not for long).
  • Instance-switching: people who juggle multiple accounts on different instances don’t necessarily want one browser tab for each.

My favorite user testimonial, though, is from my wife. She told me, “I like Pinafore because it never loses my place in the timeline.” (Much of my motivation for working on Pinafore can be credited to “wife-driven development” – I like making her happy!)

So this confirms that I’ve achieved at least some of the goals from the Pinafore introductory blog post. Although notably, offline support is rarely mentioned, but I’ll get to that later.

Collaboration

Pinafore has also benefited from a lot of community contributions. I’d like to specifically thank:

And of course everyone else who contributed. Thank you so much!

There are some challenges with building a dev community around Pinafore. The app is implemented using Svelte v2 and Sapper, which unfortunately causes two downsides in terms of onboarding: 1) Svelte isn’t a very well-known framework, and 2) Svelte v2 is incompatible with Svelte v3, and there’s no upgrade path currently.

I’ll have to continue grappling with these challenges, but for now I’m very satisfied with Svelte v2. It’s fast, lightweight, and does everything I need it to. So I’m not in a big hurry to upgrade.

And oh yeah: Svelte really is lightweight. Pinafore only loads 32KB of compressed JavaScript for the landing page, and 137KB for the Home timeline. The total size of all JS assets is under 300KB compressed (<1MB raw). It gets a perfect 100 score from Lighthouse.

Screenshot of Lighthouse showing perfect 100 score in all categories, including Performance, Accessibility, Best Practices, and SEO

If you didn’t think I was going to brag about web perf vanity metrics, then you don’t know me very well.

Future plans

My first goal with Pinafore is completeness. Even though I’ve been working on it for over a year, there are still plenty of missing features compared to the Mastodon frontend. And although the gap has been narrowing, Mastodon itself hasn’t stopped innovating, so there’s always new stuff to add. (Polls! Blurhash! Keybase! Does Eugen ever sleep?)

Beyond that, I’d like to start focusing on features that make Pinafore a more pleasant social media experience. One of the virtues of decentralized social media is that we can experiment with features that give people control over their social media experience, even if it hampers addictiveness or growth. To that end, I’ve added a set of wellness features, inspired by Tristan Harris’s Center for Humane Technology. I’ll probably tweak and expand these features as feedback rolls in.

I’d also like to improve offline support. Even though Pinafore does have an offline mode, and even though it uses a Service Worker to cache static assets, it’s not very offline-first. Instead, it uses offline storage more as a fallback for when the network fails, rather than as the primary source of truth.

Given my background working on offline-first technology and advocating for it, I find this a bit disappointing. But it turns out that it’s really difficult to implement an offline-first social media UI. How do you deal with offline writes? How do you handle the gap between fresh content and stale content within the same timeline? These are not easy questions, and for the most part I’ve punted on them. But Pinafore can do better.

Conclusion

Pinafore is a passion project for me. It gives me something interesting to do on weekends and evenings, and it teaches me a lot about how the web platform works.

I also see Pinafore as an opportunity to provide more options to the Mastodon community, and to prove that you don’t have to treat Eugen as a gatekeeper for every minor UI tweak you’d like to see in Mastodon. Mastodon is decentralized; let’s decentralize the interface!

I have every intention to keep working on Pinafore, and I’m curious to know where you think it should go next.

Get off of Twitter

Twitter logo with a red "no" sign over it

Stop complaining about Twitter on Twitter. Deny them your attention, your time, and your data. Get off of Twitter.

The more time you spend on Twitter, the more money you make for Twitter. Get off of Twitter.

You at-mention @jack and call him out for the harassment and disinformation on his platform. You get a few hundred likes and retweets, each one sending your brain a little boost of serotonin. Twitter learns that you are interested in people who criticize @jack and starts to recommend you their tweets. You end up spending more time on Twitter, and advertisers learn a little bit more about you. You make @jack more money.

Get off of Twitter.

You can’t criticize Twitter on Twitter. It just doesn’t work. The medium is the message.

There’s an old joke where one fish says to the other, “How’s the water today?” And the fish responds, “What’s water?” On Twitter, you might ask, “How’s the outrage today?” (The answer, of course, is “I hate it! I’m so outraged about it!”)

Get off of Twitter.

Write blog posts. Use RSS. Use micro.blog. Use Mastodon. Use Pleroma. Use whatever you want, as long as it isn’t manipulating you with algorithms or selling access to your data to advertisers.

You’re worried about losing your influence. How about using your influence for something good? How about using it to stick it to Twitter, if you really dislike Twitter so much? Maybe if you do it, and your friends do it, then it will cause a sea change. After all, who was ever “influential” by following the crowd?

As Gandhi said (in paraphrase), “Be the change you want to see in the world.” Or as another influencer put it: “I’m starting with the man in the mirror.” Or if you prefer: “Practice what you preach.”

Get off of Twitter.

In defense of the Right Thing

It has come to my attention that many people believe the Wrong Thing. I find this to be an intolerable state of affairs, so this is a blog post defending what is Right.

How do I know there are so many people who believe the Wrong Thing? Well because, like everyone, I use Twitter. And holy moly! My feed is chock-full of Wrong Thinkers.

Sometimes it feels like everyone in the world believes the Wrong Thing, and I’m the last lonely person clinging to what’s Right. There must be a global epidemic of Wrongness. Why else would Twitter fill my feed with so many of these dunces and ninnies and halfwits?

I don’t even follow these people. Why should they be in my timeline, unless the whole world is full of Wrong people?

Every time I see their Wrong tweets, I seethe with rage and eagerly click to read the full thread. I might spend hours this way, thumbing through Wrong tweets. “How can so many people be so Wrong?” I’ll say to myself, shaking my head as I continue to scroll.

Sometimes when I find a really Wrong tweet, I’ll quote it and tweet it out with the perfect devastating repartee. That way, more people who agree with me are exposed to these Wrong views. That’ll teach ’em!

I do have to commend the others who proudly rise in defense of what is Right. On Twitter, I often see them caught in an epic battle with the Wrongers – “34 people are talking about this!” Well, here comes a 35th, joining the fray to fight the good fight.

To be honest, I sometimes get tired of feeling angry all the time. But how can I not be, when the world is full of people who are so very Wrong?

Curiously, the Wrongers seem to come from all sides of any issue, and they are legion. People who use margarine instead of butter, people who peel the banana from the top instead of the bottom, people who crack a boiled egg from the big end rather than the small end. These are exactly the things that drive me nuts, and somehow my feed is full of people who believe the opposite of me on precisely those issues! Sometimes I feel utterly embroiled, helpless, a tiny voice of reason shouting against an angry mob of Wrong Thinkers.

But sadly, this is just how the world is these days. The world is full of people arguing, calling each other out, or watching a fight unfold with the horrified glee of a driver craning their neck to get a good look at a car wreck.

I know this is how the world is, because I see it on Twitter. And Twitter is an utterly unbiased mirror of the world, with no algorithms that subtly push the discussion in one direction or the other, regardless of whether it is good for discourse or compassion or human well-being but only whether it is good for Twitter.

Building a modern carousel with CSS scroll snap, smooth scrolling, and pinch-zoom

Recently I had some fun implementing an image carousel for Pinafore. The requirements were pretty simple: users should be able to swipe horizontally through up to 4 images, and also pinch-zoom to get a closer look.

The finished product looks like this:

 

Often when you’re building something like this, it’s tempting to use an off-the-shelf solution. The problem is that this often adds a large dependency size, or the code is inflexible, or it’s framework-specific (React, Vue, etc.), or it may not be optimized for performance and accessibility.

Come on, it’s 2019. Isn’t there a decent way to build a carousel with native browser APIs?

As it turns out, there is. My carousel implementation uses a few simple building blocks:

  1. CSS scroll snap
  2. scrollTo() with smooth behavior
  3. The <pinch-zoom> custom element

CSS scroll snap

Let’s start off with CSS scroll snap. This is what makes the scrollable element “snap” to a certain position as you scroll it.

The browser support is pretty good. The only trick is that you have to write one implementation for the modern scroll snap API (supported by Chrome and Safari), and another for the older scroll snap points API (supported by Firefox[1]).

You can detect support using @supports (scroll-snap-align: start). As usual for iOS Safari, you’ll also need to add -webkit-overflow-scrolling: touch to make the element scrollable.

But lo and behold, we now have the world’s simplest carousel implementation. It doesn’t even require JavaScript – just HTML and CSS!

Note: for best results, you may want to view the above pen in full mode.

The benefit of having all this “snapping” logic inside of CSS rather than JavaScript is that the browser is doing the heavy lifting. We don’t have to use touchmove listeners or requestAnimationFrame to try to get the pixel-perfect snapping behavior with the right animation curve – the browser handles all of it for us, in native code.

And unlike touchmove, this scroll-snapping works for any method of scrolling – touchpad, touchscreen, scrollbar, you name it.

scrollTo() with smooth scrolling

The next piece of the puzzle is that most carousels have little indicator buttons that let you navigate between the items in the list.

Screenshot of a carousel containing an image of a cat with indicator buttons below showing 1 filled circle and 3 unfilled circles

For this, we will need a little bit of JavaScript. We can use the scrollTo() API with {behavior: 'smooth'}, which tells the browser to smoothly scroll to a given offset:

function scrollToItem(itemPosition, numItems, scroller) {
  scroller.scrollTo({
    scrollLeft: Math.floor(
      scroller.scrollWidth * (itemPosition / numItems)
    ),
    behavior: 'smooth'
  })
}

The only trick here is that Safari doesn’t support smooth scroll behavior and Edge doesn’t support scrollTo() at all. But we can detect support and fall back to a JavaScript implementation, such as this one.

Here is my technique for detecting native smooth scrolling:

function testSupportsSmoothScroll () {
  var supports = false
  try {
    var div = document.createElement('div')
    div.scrollTo({
      top: 0,
      get behavior () {
        supports = true
        return 'smooth'
      }
    })
  } catch (err) {}
  return supports
}

Being careful to set aria-labels and aria-pressed states for the buttons, and adding a debounced scroll listener to update the pressed state as the user scrolls, we end up with something like this:

View in full mode

You can also add generic “go left” and “go right” buttons; the principle is the same.

Hiding the scrollbar (optional)

Now, the next piece of the puzzle is that most carousels don’t have a scrollbar, and depending on the browser and OS, you might not like how the scrollbar appears.

Also, our carousel already includes all the buttons needed to scroll left and right, so it effectively has its own scrollbar. So we can consider removing the native one.

To accomplish this, we can start with overflow-x: auto rather than overflow-x: scroll, which ensures that at least if there’s only one image (and thus no possibility of scrolling left or right), the scrollbar doesn’t show.

Beyond that, we may be tempted to add overflow-x: hidden, but this actually makes the list entirely unscrollable. Bummer.

So we can use a little hack instead. Here is some CSS to remove the scrollbar, which works in Chrome, Edge, Firefox, and Safari:

.scroll {
  scrollbar-width: none;
  -ms-overflow-style: none;
}
.scroll::-webkit-scrollbar {
  display: none;
}

And it works! The scrollbar is gone:

View in full mode

Admittedly, though, this is a bit icky. The only standards-based CSS here is scrollbar-width, which is currently only supported by Firefox. The -webkit-scrollbar hack is for Chrome and Safari, and the -ms-overflow-style hack is for Edge/IE.

So if you don’t like vendor-specific CSS, or if you think scrollbars are better for accessibility, then you can just keep the scrollbar around. Follow your heart!

Pinch-zoom

For pinch-zooming, this is one case where I allowed myself an indulgence: I use the <pinch-zoom> element from Google Chrome Labs.

I like it because it’s extremely small (5.2kB minified) and it uses Pointer Events under the hood, meaning it supports mobile touchscreens, touchpads, touchscreen laptops, and any device that supports pinch-zooming.

However, this element isn’t totally compatible with a scrollable list, because dragging your finger left and right causes the image to move left and right, rather than scroll left and right.

 

I thought this was actually a nice touch, though, since it allows you to choose which part of the image to zoom in on. So I decided to keep it.

To make this work inside a scrollable carousel, though, I decided to add a separate mode for zooming. You have to tap the magnifying glass to enable zooming, at which point dragging your finger moves the image itself rather than the carousel.

Toggling the pinch-zoom mode was as simple as removing or adding the <pinch-zoom> element to toggle it [2]. I also decided to add some explicit “zoom in” and “zoom out” buttons for the benefit of users who don’t have a device that supports pinch-zooming.

 

Of course, I could have implemented this myself using raw Pointer Events, but <pinch-zoom> offers a small footprint, a nice API, and good browser compatibility (e.g. on iOS Safari, where Pointer Events are not supported). So it felt like a worthy addition.

Intrinsic sizing

The last piece of the puzzle (I promise!) is a way to keep the images from doing a re-layout when they load. This can lead to janky-looking reflows, especially on slow connections.

 

Assuming we know the dimensions of the images in advance, we can fix this by using the intrinsicsize attribute. Unfortunately this isn’t supported in any browser yet, but it’s coming soon to Chrome! And it’s way easier than any other (hacky) solution you may think of.

Here it is in Chrome 72 with the “experimental web platform features” flag enabled:

 

Notice that the buttons don’t jump around while the image loads. Much nicer!

Accessibility check

Looking over the WAI Carousel Concepts document, there are a few good things to keep in mind when implementing this carousel:

  1. To make the carousel more keyboard-navigable, you may add keyboard shortcuts, for instance and to navigate left and right. (Note though that a scrollable horizontal list can already be focused and scrolled with the keyboard.)
  2. Use <ul> and <li> elements instead of <div>s, so that a screen reader announces it as a list.
  3. The smooth-scrolling can be distracting or nausea-inducing for some folks, so respect prefers-reduced-motion or provide an option to turn it off.
  4. As mentioned previously, use aria-label and aria-pressed for the indicator buttons.

Compatibility check

But what about IE support? I can hear your boss screaming at you already.

If you’re one of the unfortunate souls who still has to maintain IE11 support, rest assured: a scroll-snap carousel is just a normal horizontal-scrolling list on IE. It doesn’t work exactly the same, but hey, does it need to? IE11 users probably don’t expect the best anymore.

Conclusion

So that’s it! I decided not to publish this as a library, and I’m leaving the pinch-zooming and intrinsic sizing as an exercise for the reader. I think the core building blocks are simple enough that folks really ought to just take the native APIs and run with them.

Any decisions I could bake into a library would only limit the flexibility of the carousel, and leave its users high-and-dry when they need to tweak something, because I’ve taught them how to use my library instead of the native browser API.

At some point, it’s just better to go native.

Footnotes

1. For whatever reason, I couldn’t get the old scroll snap points spec to work in Edge. Sarah Drasner apparently ran into the same issue. On the bright side, though, a horizontally scrolling list without snap points is just a regular horizontally scrolling list!

2. The first version of this blog post recommended using pointer-events: none to toggle the zoom mode on or off. It turns out that this breaks right-clicking to download an image. So it seems better to just remove or add the <pinch-zoom> element to toggle it.