Archive for the ‘Web’ Category

Scrolling the main document is better for performance, accessibility, and usability

When I first wrote Pinafore, I thought pretty deeply about some aspects of the scrolling, but not enough about others.

For instance, I implemented a custom virtual list in Svelte.js, as well as an infinite scroll to add more content as you scroll the timeline. When it came to where to put the scrollable element, though, I didn’t think too hard about it.

Screenshot of Pinafore UI showing a top nav with a scrollable content below

A fixed-position nav plus a scrollable section below. Seems simple, right? I went with what seemed to me like an obvious solution: an absolute-position element below the nav bar, which could be scrolled up and down.

Then Sorin Davidoi opened this issue, pointing out that using the entire document (i.e. the <body>) as the scrolling element would allow mobile browsers to hide the address bar while scrolling down. I wasn’t aware of this, so I went ahead and implemented it.

This indeed allowed the URL bar to gracefully shrink or hide, across a wide range of mobile browsers. Here’s Safari for iOS:

And Chrome for Android:

And Firefox for Android:

As it turned out, though, this fix solved more than just the address bar problem – it also improved the framerate of scrolling in Chrome for Android. This is a longstanding issue in Pinafore that had puzzled me up till now. But with the “document as scroller” change, the framerate is magically improved:

Of course, as the person who wrote one of the more comprehensive analyses of cross-browser scrolling performance, this really shouldn’t have surprised me. My own analysis showed that some browsers (notably Chrome) hadn’t optimized subscrolling to nearly the same degree as main-document scrolling. Somehow, though, I didn’t put two-and-two together and realize that this is why Pinafore’s scrolling was janky in Chrome for Android. (It was fine in Firefox for Android and Safari for iOS, which is also perhaps why I didn’t feel pressed to fix it.)

In retrospect, the Chrome Dev Tools’s “scrolling performance issues” tool should have been enough to tip me off, but I wasn’t sure what to do when it says “repaints on scroll.” Nor did I know that moving the scrolling element to the main document would do the trick. Most of the advice online suggests using will-change: transform, but in this case it didn’t help. (Although in the past, I have found that will-change can improve mobile Chrome’s scrolling in some cases.)

Screenshot of Pinafore with a blue overlay saying "repaints on scroll."

The “repaints on scroll” warning. This is gone now that the scrollable element is the document body.

As if the mobile UI and performance improvements weren’t enough, this change also improved accessibility. When users first open Pinafore, they often want to start scrolling by tapping the “down” or “PageDown” key on the keyboard. However, this doesn’t work if you’re using a subscroller, because unlike the main document, the subscroller isn’t focused by default. So we had to add custom behavior to focus the scrollable element when the page first loads. Once I got rid of the subscroller, though, this code could be removed.

Another nice fix was that it’s no longer necessary to add -webkit-overflow-scrolling: touch so that iOS Safari will use smooth scrolling. The main document already scrolls smoothly on iOS.

This subscroller fix may be obvious to more experienced web devs, but to me it was a bit surprising. From a design standpoint, the two options seemed roughly equivalent, and it didn’t occur to me that one or the other would have such a big impact, especially on mobile browsers. Given the difference in performance, accessibility, and usability though, I’ll definitely think harder in the future about exactly which element I want to be the scrollable one.

Note that what I’m not saying in this blog post is that you should avoid subscrollers at all costs. There are some cases where the design absolutely calls for a subscroller, and the fact that Chrome hasn’t optimized for this scenario (whereas other browsers like Firefox, Edge, and Safari have) is a real bug, and I hope they’ll fix it.

However, if the visual design of the page calls for the entire document to be scrollable, then by all means, make the entire document scrollable! And check out document.scrollingElement for a good cross-browser API for managing the scrollTop and scrollHeight.

Update: Steve Genoud points out that there’s an additional benefit to scrolling the main document on iOS: you can tap the status bar to scroll back up to the top. Another usability win!

Update: Michael Howell notes that this technique can cause problems for fragment navigation, e.g. index.html#fragment, because the fixed nav could cover up the target element. Amusingly, I’ve noticed this problem in WordPress.com (where my blog is hosted) if you navigate to a fragment while logged in. I also ran into this in Pinafore in the case of element.scrollIntoView(), which I worked around by updating the scrollTop to account for the nav height right after calling scrollIntoView(true). Good to be aware of!

Accurately measuring layout on the web

We all want to make faster websites. The question is just what to measure, and how to use that information to determine what’s “slow” and what could be made faster.

The browser rendering pipeline is complicated. For that reason, it’s tricky to measure the performance of a webpage, especially when components are rendered client-side and everything becomes an intricate ballet between JavaScript, the DOM, styling, layout, and rendering. Many folks stick to what they understand, and so they may under-measure or completely mis-measure their website’s frontend performance.

So in this post, I want to demystify some of these concepts, and offer techniques for accurately measuring what’s going on when we render things on the web.

The web rendering pipeline

Let’s say we have a component that is rendered client-side, using JavaScript. To keep things simple, I wrote a demo component in vanilla JS, but everything I’m about to say would also apply to React, Vue, Angular, etc.

When we use the handy Performance profiler in the Chrome Dev Tools, we see something like this:

Screenshot of Chrome Dev Tools showing work on the UI thread divided into JavaScript, then Style, then Layout, then Render

This is a view of the CPU costs of our component, in terms of milliseconds on the UI thread. To break things down, here are the steps required:

  1. Execute JavaScript – executing (but not necessarily compiling) JavaScript, including any state manipulation, “virtual DOM diffing,” and modifying the DOM.
  2. Calculate style – taking a CSS stylesheet and matching its selector rules with elements in the DOM. This is also known as “formatting.”
  3. Calculate layout – taking those CSS styles we calculated in step #2 and figuring out where the boxes should be laid out on the screen. This is also known as “reflow.”
  4. Render – the process of actually putting pixels on the screen. This often involves painting, compositing, GPU acceleration, and a separate rendering thread.

All of these steps invoke CPU costs, and therefore all of them can impact the user experience. If any one of them takes a long time, it can lead to the appearance of a slow-loading component.

The naïve approach

Now, the most common mistake that folks make when trying to measure this process is to skip steps 2, 3, and 4 entirely. In other words, they just measure the time spent executing JavaScript, and completely ignore everything after that.

Screenshot of Chrome Dev Tools, showing an arrow pointing after JavaScript but before Style and Layout with the text 'Most devs stop measuring here'

When I worked as a browser performance engineer, I would often look at a trace of a team’s website and ask them which mark they used to measure “done.” More often than not, it turned out that their mark landed right after JavaScript, but before style and layout, meaning the last bit of CPU work wasn’t being measured.

So how do we measure these costs? For the purposes of this post, let’s focus on how we measure style and layout in particular. As it turns out, the render step is much more complicated to measure, and indeed it’s impossible to measure accurately, because rendering is often a complex interplay between separate threads and the GPU, and therefore isn’t even visible to userland JavaScript running on the main thread.

Style and layout calculations, however, are 100% measurable because they block the main thread. And yes, this is true even with something like Firefox’s Stylo engine – even if multiple threads can be employed to speed up the work, ultimately the main thread has to wait on all the other threads to deliver the final result. This is just the way the web works, as specc’ed.

What to measure

So in practical terms, we want to put a performance mark before our JavaScript starts executing, and another one after all the additional work is done:

Screenshot of Chrome Dev Tools, with arrow pointing before JavaScript execution saying 'Ideal start' and arrow pointing after Render (Paint) saying 'Ideal end'

I’ve written previously about various JavaScript timers on the web. Can any of these help us out?

As it turns out, requestAnimationFrame will be our main tool of choice, but there’s a problem. As Jake Archibald explains in his excellent talk on the event loop, browsers disagree on where to fire this callback:

Screenshot of Chrome Dev Tools showing arrow pointing before style/layout saying "Chrome, FF, Edge >= 18" and arrow pointing after style/layout saying "Safari, IE, Edge < 18"

Now, per the HTML5 event loop spec, requestAnimationFrame is indeed supposed to fire before style and layout are calculated. Edge has already fixed this in v18, and perhaps Safari will fix it in the future as well. But that would still leave us with inconsistent behavior in IE, as well as in older versions of Safari and Edge.

Also, if anything, the spec-compliant behavior actually makes it more difficult to measure style and layout! In an ideal world, the spec would have two timers – one for requestAnimationFrame, and another for requestAnimationFrameAfterStyleAndLayout (or something like that). In fact, there has been some discussion at the WHATWG about adding an API for this, but so far it’s just a gleam in the spec authors’ eyes.

Unfortunately, we live in the real world with real constraints, and we can’t wait for browsers to add this timer. So we’ll just have to figure out how to crack this nut, even with browsers disagreeing on when requestAnimationFrame should fire. Is there any solution that will work cross-browser?

Cross-browser “after frame” callback

There’s no solution that will work perfectly to place a callback right after style and layout, but based on the advice of Todd Reifsteck, I believe this comes closest:

requestAnimationFrame(() => {
  setTimeout(() => {
    performance.mark('end')
  })
})

Let’s break down what this code is doing. In the case of spec-compliant browsers, such as Chrome, it looks like this:

Screenshot of Chrome Dev Tools showing 'Start' before JavaScript execution, requestAnimationFrame before style/layout, and setTimeout falling a bit after Paint/Render

Note that rAF fires before style and layout, but the next setTimeout fires just after those steps (including “paint,” in this case).

And here’s how it works in non-spec-compliant browsers, such as Edge 17:

Screenshot of Edge F12 Tools showing 'Start' before JavaScript execution, and requestAnimationFrame/setTimeout both almost immediately after style/layout

Note that rAF fires after style and layout, and the next setTimeout happens so soon that the Edge F12 Tools actually render the two marks on top of each other.

So essentially, the trick is to queue a setTimeout callback inside of a rAF, which ensures that the second callback happens after style and layout, regardless of whether the browser is spec-compliant or not.

Downsides and alternatives

Now to be fair, there are a lot of problems with this technique:

  1. setTimeout is somewhat unpredictable in that it may be clamped to 4ms (or more in some cases).
  2. If there are any other setTimeout callbacks that have been queued elsewhere in the code, then ours may not be the last one to run.
  3. In the non-spec-compliant browsers, doing the setTimeout is actually a waste, because we already have a perfectly good place to set our mark – right inside the rAF!

However, if you’re looking for a one-size-fits-all solution for all browsers, rAF + setTimeout is about as close as you can get. Let’s consider some alternative approaches and why they wouldn’t work so well:

rAF + microtask

requestAnimationFrame(() => {
  Promise.resolve().then(() => {
    performance.mark('after')
  })
})

This one doesn’t work at all, because microtasks (e.g. Promises) run immediately after JavaScript execution has completed. So it doesn’t wait for style and layout at all:

Screenshot of Chrome Dev Tools showing microtask firing before style/layout

rAF + requestIdleCallback

requestAnimationFrame(() => {
  requestIdleCallback(() => {
    performance.mark('after')
  })
})

Calling requestIdleCallback from inside of a requestAnimationFrame will indeed capture style and layout:

Screenshot of Chrome Dev Tools showing requestIdleCallback firing a bit after render/paint

However, if the microtask version fires too early, I would worry that this one would fire too late. The screenshot above shows it firing fairly quickly, but if the main thread is busy doing other work, rIC could be delayed a long time waiting for the browser to decide that it’s safe to run some “idle” work. This one is far less of a sure bet than setTimeout.

rAF + rAF

requestAnimationFrame(() => {
  requestAnimationFrame(() => {
    performance.mark('after')
  })
})

This one, also called a “double rAF,” is a perfectly fine solution, but compared to the setTimeout version, it probably captures more idle time – roughly 16.7ms on a 60Hz screen, as opposed to the standard 4ms for setTimeout – and is therefore slightly more inaccurate.

Screenshot of Chrome Dev Tools showing a second requestAnimationFrame firing a bit after render/paint

You might wonder about that, given that I’ve already talked about setTimeout(0) not really firing in 0 (or even necessarily 4) milliseconds in a previous blog post. But keep in mind that, even though setTimeout() may be clamped by as much as a second, this only occurs in a background tab. And if we’re running in a background tab, we can’t count on rAF at all, because it may be paused altogether. (How to deal with noisy telemetry from background tabs is an interesting but separate question.)

So rAF+setTimeout, despite its flaws, is probably still better than rAF+rAF.

Not fooling ourselves

In any case, whether we choose rAF+setTimeout or double rAF, we can rest assured that we’re capturing any event-loop-driven style and layout costs. With this measure in place, it’s much less likely that we’ll fool ourselves by only measuring JavaScript and direct DOM API performance.

As an example, let’s consider what would happen if our style and layout costs weren’t just invoked by the event loop – that is, if our component were calling one of the many APIs that force style/layout recalculation, such as getBoundingClientRect(), offsetTop, etc.

If we call getBoundingClientRect() just once, notice that the style and layout calculations shift over into the middle of JavaScript execution:

Screenshot of Chrome Dev Tools showing style/layout costs moved to the left inside of JavaScript execution under getBoundingClientRect with red triangles on each purple rectangle

The important point here is that we’re not doing anything any slower or faster – we’ve merely moved the costs around. If we don’t measure the full costs of style and layout, though, we might deceive ourselves into thinking that calling getBoundingClientRect() is slower than not calling it! In fact, though, it’s just a case of robbing Peter to pay Paul.

It’s worth noting, though, that the Chrome Dev Tools have added little red triangles to our style/layout calculations, with the message “Forced reflow is a likely performance bottleneck.” This can be a bit misleading in this case, because again, the costs are not actually any higher – they’ve just moved to earlier in the trace.

(Now it’s true that, if we call getBoundingClientRect() repeatedly and change the DOM in the process, then we might invoke layout thrashing, in which case the overall costs would indeed be higher. So the Chrome Dev Tools are right to warn folks in that case.)

In any case, my point is that it’s easy to fool yourself if you only measure explicit JavaScript execution, and ignore any event-loop-driven style and layout costs that come afterward. The two costs may be scheduled differently, but they both impact performance.

Conclusion

Accurately measuring layout on the web is hard. There’s no perfect metric to capture style and layout – or indeed, rendering – even though all three can impact the user experience just as much as JavaScript.

However, it’s important to understand how the HTML5 event loop works, and to place performance marks at the appropriate points in the component rendering lifecycle. This can help avoid any mistaken conclusions about what’s “slower” or “faster” based on an incomplete view of the pipeline, and ensure that style and layout costs are accounted for.

I hope this blog post was useful, and that the art of measuring client-side performance is a little less mysterious now. And maybe it’s time to push browser vendors to add requestAnimationFrameAfterStyleAndLayout (we’ll bikeshed on the name though!).

Thanks to Ben Kelly, Todd Reifsteck, and Alex Russell for feedback on a draft of this blog post.

Moving on from Microsoft

When I first joined Microsoft, I wrote an idealistic blog post describing the web as a public good, one that I hoped to serve by working for a browser vendor. I still believe in that vision of the web: it’s the freest platform on earth, it’s not totally dominated by any one entity, and it provides both users and developers with a degree of control that isn’t really available in the walled-garden app models of iOS, Android, etc.

Plaque saying "this is for everyone", dedicated, to Tim Berners-Lee

“This is for everyone”, a tribute to the web and Tim Berners-Lee (source)

After joining Microsoft, I continued to be blown away by the dedication and passion of those working on the Edge browser, many of whom shared my feelings about the open web. There were folks who were ex-Mozilla, ex-Opera, ex-Google, ex-Samsung – basically ex-AnyBrowserVendor. There were also plenty of Microsoft veterans who took their expertise in Windows, the CLR, and other Microsoft technologies and applied it with gusto to the unique challenges of the web.

It was fascinating to see so many people from so many different backgrounds working on something as enormously complex as a browser, and collaborating with like-minded folks at other browser vendors in the open forums of the W3C, TC39, WHATWG, and other standards bodies. Getting a peek behind the curtain at how the web “sausage” is made was a unique experience, and one that I cherish.

I’m also proud of the work I accomplished during my tenure at Microsoft. I wrote a blog post on how browser scrolling works, which not only provided a comprehensive overview for web developers, but I suspect may have even spurred Mozilla to up their scrolling game. (Congrats to Firefox for becoming the first browser to support asynchronous keyboard scrolling!)

I also did some work on the Intersection Observer spec, which I became interested in after discovering some cross-browser incompatibilities prompted by a bug in Mastodon. This was exactly the kind of thing I wanted to do for the web:

  1. Find a browser bug while working on an open-source project.
  2. Rather than just work around the bug, actually fix the browsers!
  3. Discuss the problem with the spec owners at other browser vendors.
  4. Submit the fixes to the spec and the web platform tests.

I didn’t do as much spec work as I would have liked (in particular, as a member of the Web Performance Working Group), but I am happy with the small contributions I managed to make.

While at Microsoft, I was also given the opportunity to speak at several conferences, an experience I found exhilarating if not a bit exhausting. (Eight talks in one year was perhaps too ambitious of me!) Overall, though, being a public speaker was a part of the browser gig that I thoroughly enjoyed, and the friendships I made with other conference attendees will surely linger in my mind long after I’ve forgotten whatever it was I gave a talk about. (Thankfully, though, there are always the videos!)

Photo of me delivering the talk "Solving the web performance crisis", a talk I gave on web performance

“Solving the web performance crisis,” a talk I gave on JavaScript performance (video link)

I also wrote a blog post on input responsiveness, which later inspired a post on JavaScript timers. It’s amazing how much you can learn about how a browser works by (wait for it) working on a browser! During my first year at Microsoft, I found myself steeped in discussions about the HTML5 event loop, Promises, setTimeout, setImmediate, and all the other wonderful ways of scheduling things on the web, because at the time we were knee-deep in rewriting the EdgeHTML event loop to improve performance and reliability.

Some of this work was even paralleled by other browser vendors, as shown in this blog post by Ben Kelly about Firefox 52. I fondly recall Ben and me swapping some benchmarks at a TPAC meeting. (When you get into the nitty-gritty of this stuff, sometimes it feels like other browser vendors are the only ones who really understand what you’re going through!)

I also did some work internal to Microsoft that I believe had a positive impact. In short, I met with lots of web teams and coached them on performance – walking through traces of Edge, IE, and Chrome – and helped them improve performance of their site across all browsers. Most of this coaching involved Windows Performance Analyzer, which is a surprisingly powerful tool despite being somewhat under-documented. (Although this post by my colleague Todd Reifsteck goes a long way toward demystifying some of the trickier aspects.)

I discussed this work a bit in an interview I did for Between the Wires, but most of it is private to the teams I worked with, since performance can be a tricky subject to talk about publicly. In general, neither browser vendors nor website owners want to shout to the heavens about their performance problems, so to avoid embarrassing both parties, most of the work I did in this area will probably never be public.

Presentation slide saying "you've founda  perf issue," either "website looks bad compared to competitors," or "browser looks bad compared to competitors" with hand-darwn "websites" and browser logos

A slide from a talk I gave at the Edge Web Summit (video link, photo source)

Still, this work (we called it “Performance Clubs”) was one of my favorite parts of working at Microsoft. Being a “performance consultant,” analyzing traces, and reasoning about the interplay between browser architecture and website architecture was something I really enjoyed. It was part education (I gave a lot of impromptu speeches in front of whiteboards!) and part detective work (lots of puzzling over traces, muttering to myself “this thread isn’t supposed to do that!”). But as someone who is fond of both people and technology, I think I was well-suited for the task.

After Microsoft, I’ll continue doing this same sort of work, but in a new context. I’ll be joining Salesforce as a developer working on the Lightning Platform. I’m looking forward to the challenges of building an easy-to-use web framework that doesn’t sacrifice on performance – anticipating the needs of developers as well as the inherent limitations of CPUs, GPUs, memory, storage, and networks.

It will also be fun to apply my knowledge of cross-browser differences to the enterprise space, where developers often don’t have the luxury of saying “just use another browser.” It’s an unforgiving environment to develop in, but those are exactly the kinds of challenges I relish about the web!

For those who follow me mostly for my open-source work, nothing will change with my transition to Salesforce. I intend to continue working on Mastodon- and JavaScript-related projects in my spare time, including Pinafore. In fact, my experience with Pinafore and SvelteJS may pay dividends at my new gig; one of my new coworkers even mentioned SvelteJS as their north star for building a great JavaScript framework. (Seems I may have found my tribe!) Much of my Salesforce work will also be open-source, so I’m looking forward to spending more time back on GitHub as well. (Although perhaps not as intensely as I used to.)

Leaving Microsoft is a bit bittersweet for me. I’m excited by the new challenges, but I’m also going to miss all the talented and passionate people whose company I enjoyed on the Microsoft Edge team. That said, the web is nothing if not a big tent, and there’s plenty of room to work in, on, and around it. To everyone else who loves the web as I do: I’m sure our paths will cross again.

A tour of JavaScript timers on the web

Pop quiz: what is the difference between these JavaScript timers?

  • Promises
  • setTimeout
  • setInterval
  • setImmediate
  • requestAnimationFrame
  • requestIdleCallback

More specifically, if you queue up all of these timers at once, do you have any idea which order they’ll fire in?

If not, you’re probably not alone. I’ve been doing JavaScript and web programming for years, I’ve worked for a browser vendor for two of those years, and it’s only recently that I really came to understand all these timers and how they play together.

In this post, I’m going to give a high-level overview of how these timers work, and when you might want to use them. I’ll also cover the Lodash functions debounce() and throttle(), because I find them useful as well.

Promises and microtasks

Let’s get this one out of the way first, because it’s probably the simplest. A Promise callback is also called a “microtask,” and it runs at the same frequency as MutationObserver callbacks. Assuming queueMicrotask() ever makes it out of spec-land and into browser-land, it will also be the same thing.

I’ve already written a lot about promises. One quick misconception about promises that’s worth covering, though, is that they don’t give the browser a chance to breathe. Just because you’re queuing up an asynchronous callback, that doesn’t mean that the browser can render, or process input, or do any of the stuff we want browsers to do.

For example, let’s say we have a function that blocks the main thread for 1 second:

function block() {
  var start = Date.now()
  while (Date.now() - start < 1000) { /* wheee */ }
}

If we were to queue up a bunch of microtasks to call this function:

for (var i = 0; i < 100; i++) {
  Promise.resolve().then(block)
}

This would block the browser for about 100 seconds. It’s basically the same as if we had done:

for (var i = 0; i < 100; i++) {
  block()
}

Microtasks execute immediately after any synchronous execution is complete. There’s no chance to fit in any work between the two. So if you think you can break up a long-running task by separating it into microtasks, then it won’t do what you think it’s doing.

setTimeout and setInterval

These two are cousins: setTimeout queues a task to run in x number of milliseconds, whereas setInterval queues a recurring task to run every x milliseconds.

The thing is… browsers don’t really respect that milliseconds thing. You see, historically, web developers have abused setTimeout. A lot. To the point where browsers have had to add mitigations for setTimeout(/* ... */, 0) to avoid locking up the browser’s main thread, because a lot of websites tended to throw around setTimeout(0) like confetti.

This is the reason that a lot of the tricks in crashmybrowser.com don’t work anymore, such as queuing up a setTimeout that calls two more setTimeouts, which call two more setTimeouts, etc. I covered a few of these mitigations from the Edge side of things in “Improving input responsiveness in Microsoft Edge”.

Broadly speaking, a setTimeout(0) doesn’t really run in zero milliseconds. Usually, it runs in 4. Sometimes, it may run in 16 (this is what Edge does when it’s on battery power, for instance). Sometimes it may be clamped to 1 second (e.g., when running in a background tab). These are the sorts of tricks that browsers have had to invent to prevent runaway web pages from chewing up your CPU doing useless setTimeout work.

So that said, setTimeout does allow the browser to run some work before the callback fires (unlike microtasks). But if your goal is to allow input or rendering to run before the callback, setTimeout is usually not the best choice because it only incidentally allows those things to happen. Nowadays, there are better browser APIs that can hook more directly into the browser’s rendering system.

setImmediate

Before moving on to those “better browser APIs,” it’s worth mentioning this thing. setImmediate is, for lack of a better word … weird. If you look it up on caniuse.com, you’ll see that only Microsoft browsers support it. And yet it also exists in Node.js, and has lots of “polyfills” on npm. What the heck is this thing?

setImmediate was originally proposed by Microsoft to get around the problems with setTimeout described above. Basically, setTimeout had been abused, and so the thinking was that we can create a new thing to allow setImmediate(0) to actually be setImmediate(0) and not this funky “clamped to 4ms” thing. You can see some discussion about it from Jason Weber back in 2011.

Unfortunately, setImmediate was only ever adopted by IE and Edge. Part of the reason it’s still in use is that it has a sort of superpower in IE, where it allows input events like keyboard and mouseclicks to “jump the queue” and fire before the setImmediate callback is executed, whereas IE doesn’t have the same magic for setTimeout. (Edge eventually fixed this, as detailed in the previously-mentioned post.)

Also, the fact that setImmediate exists in Node means that a lot of “Node-polyfilled” code is using it in the browser without really knowing what it does. It doesn’t help that the differences between Node’s setImmediate and process.nextTick are very confusing, and even the official Node docs say the names should really be reversed. (For the purposes of this blog post though, I’m going to focus on the browser rather than Node because I’m not a Node expert.)

Bottom line: use setImmediate if you know what you’re doing and you’re trying to optimize input performance for IE. If not, then just don’t bother. (Or only use it in Node.)

requestAnimationFrame

Now we get to the most important setTimeout replacement, a timer that actually hooks into the browser’s rendering loop. By the way, if you don’t know how the browser event loops works, I strongly recommend this talk by Jake Archibald. Go watch it, I’ll wait.

Okay, now that you’re back, requestAnimationFrame basically works like this: it’s sort of like a setTimeout, except instead of waiting for some unpredictable amount of time (4 milliseconds, 16 milliseconds, 1 second, etc.), it executes before the browser’s next style/layout calculation step. Now, as Jake points out in his talk, there is a minor wrinkle in that it actually executes after this step in Safari, IE, and Edge <18, but let's ignore that for now since it's usually not an important detail.

The way I think of requestAnimationFrame is this: whenever I want to do some work that I know is going to modify the browser's style or layout – for instance, changing CSS properties or starting up an animation – I stick it in a requestAnimationFrame (abbreviated to rAF from here on out). This ensures a few things:

  1. I'm less likely to layout thrash, because all of the changes to the DOM are being queued up and coordinated.
  2. My code will naturally adapt to the performance characteristics of the browser. For instance, if it's a low-cost device that is struggling to render some DOM elements, rAF will naturally slow down from the usual 16.7ms intervals (on 60 Hertz screens) and thus it won't bog down the machine in the same way that running a lot of setTimeouts or setIntervals might.

This is why animation libraries that don't rely on CSS transitions or keyframes, such as GreenSock or React Motion, will typically make their changes in a rAF callback. If you're animating an element between opacity: 0 and opacity: 1, there's no sense in queuing up a billion callbacks to animate every possible intermediate state, including opacity: 0.0000001 and opacity: 0.9999999.

Instead, you're better off just using rAF to let the browser tell you how many frames you're able to paint during a given period of time, and calculate the "tween" for that particular frame. That way, slow devices naturally end up with a slower framerate, and faster devices end up with a faster framerate, which wouldn't necessarily be true if you used something like setTimeout, which operates independently of the browser's rendering speed.

requestIdleCallback

rAF is probably the most useful timer in the toolkit, but requestIdleCallback is worth talking about as well. The browser support isn't great, but there's a polyfill that works just fine (and it uses rAF under the hood).

In many ways rAF is similar to requestIdleCallback. (I'll abbreviate it to rIC from now on. Starting to sound like a pair of troublemakers from West Side Story, huh? "There go Rick and Raff, up to no good!")

Like rAF, rIC will naturally adapt to the browser's performance characteristics: if the device is under heavy load, rIC may be delayed. The difference is that rIC fires on the browser "idle" state, i.e. when the browser has decided it doesn't have any tasks, microtasks, or input events to process, and you're free to do some work. It also gives you a "deadline" to track how much of your budget you're using, which is a nice feature.

Dan Abramov has a good talk from JSConf Iceland 2018 where he shows how you might use rIC. In the talk, he has a webapp that calls rIC for every keyboard event while the user is typing, and then it updates the rendered state inside of the callback. This is great because a fast typist can cause many keydown/keyup events to fire very quickly, but you don't necessarily want to update the rendered state of the page for every keypress.

Another good example of this is a “remaining character count” indicator on Twitter or Mastodon. I use rIC for this in Pinafore, because I don't really care if the indicator updates for every single key that I type. If I'm typing quickly, it's better to prioritize input responsiveness so that I don't lose my sense of flow.

Screenshot of Pinafore with some text entered in the text box and a digit counter showing the number of remaining characters

In Pinafore, the little horizontal bar and the “characters remaining” indicator update as you type.

One thing I’ve noticed about rIC, though, is that it’s a little finicky in Chrome. In Firefox it seems to fire whenever I would, intuitively, think that the browser is “idle” and ready to run some code. (Same goes for the polyfill.) In mobile Chrome for Android, though, I’ve noticed that whenever I scroll with touch scrolling, it might delay rIC for several seconds even after I’m done touching the screen and the browser is doing absolutely nothing. (I suspect the issue I’m seeing is this one.)

Update: Alex Russell from the Chrome team informs me that this is a known issue and should be fixed soon!

In any case, rIC is another great tool to add to the tool chest. I tend to think of it this way: use rAF for critical rendering work, use rIC for non-critical work.

debounce and throttle

These two functions aren’t built in to the browser, but they’re so useful that they’re worth calling out on their own. If you aren’t familiar with them, there’s a good breakdown in CSS Tricks.

My standard use for debounce is inside of a resize callback. When the user is resizing their browser window, there’s no point in updating the layout for every resize callback, because it fires too frequently. Instead, you can debounce for a few hundred milliseconds, which will ensure that the callback eventually fires once the user is done fiddling with their window size.

throttle, on the other hand, is something I use much more liberally. For instance, a good use case is inside of a scroll event. Once again, it’s usually senseless to try to update the rendered state of the app for every scroll callback, because it fires too frequently (and the frequency can vary from browser to browser and from input method to input method… ugh). Using throttle normalizes this behavior, and ensures that it only fires every x number of milliseconds. You can also tweak Lodash’s throttle (or debounce) function to fire at the start of the delay, at the end, both, or neither.

In contrast, I wouldn’t use debounce for the scrolling scenario, because I don’t want the UI to only update after the user has explicitly stopped scrolling. That can get annoying, or even confusing, because the user might get frustrated and try to keep scrolling in order to update the UI state (e.g. in an infinite-scrolling list). throttle is better in this case, because it doesn’t wait for the scroll event to stop firing.

throttle is a function I use all over the place for all kinds of user input, and even for some regularly-scheduled tasks like IndexedDB cleanups. It’s extremely useful. Maybe it should just be baked into the browser some day!

Conclusion

So that’s my whirlwind tour of the various timer functions available in the browser, and how you might use them. I probably missed a few, because there are certainly some exotic ones out there (postMessage or lifecycle events, anyone?). But hopefully this at least provides a good overview of how I think about JavaScript timers on the web.

Smaller Lodash bundles with Webpack and Babel

One of the benefits of working with smart people is that you can learn a lot from them through osmosis. As luck would have it, a recent move placed my office next to John-David Dalton‘s, with the perk being that he occasionally wanders into my office to talk about cool stuff he’s working on, like Lodash and ES modules in Node.

Recently we chatted about Lodash and the various plugins for making its bundle size smaller, such as lodash-webpack-plugin and babel-plugin-lodash. I admitted that I had used both projects but only had a fuzzy notion of what they actually did, or why you’d want to use one or the other. Fortunately J.D. set me straight, and so I thought it’d be a good opportunity to take what I’ve learned and turn it into a short blog post.

TL;DR

Use the import times from 'lodash/times' format over import { times } from 'lodash' wherever possible. If you do, then you don’t need the babel-plugin-lodash. Update: or use lodash-es instead.

Be very careful when using lodash-webpack-plugin to check that you’re not omitting any features you actually need, or stuff can break in production.

Avoid Lodash chaining (e.g. _(array).map(...).filter(...).take(...)), since there’s currently no way to reduce its size.

babel-plugin-lodash

The first thing to understand about Lodash is that there are multiple ways you can use the same method, but some of them are more expensive than others:

import { times } from 'lodash'   // 68.81kB  :(
import times from 'lodash/times' //  2.08kB! :)

times(3, () => console.log('whee'))

You can see the difference using something like webpack-bundle-analyzer. Here’s the first version:

Screenshot of lodash.js taking up almost the entire bundle size

Using the import { times } from 'lodash' idiom, it turns out that lodash.js is so big that you can’t even see our tiny index.js! Lodash takes up a full parsed size of 68.81kB. (In the bundle analyzer, hover your mouse over the module to see the size.)

Now here’s the second version (using import times from 'lodash/times'):

Screenshot showing many smaller Lodash modules not taking up so much space

In the second screenshot, Lodash’s total size has shrunk down to 2.08kB. Now we can finally see our index.js!

However, some people prefer the second syntax to the first, especially since it can get more terse the more you import.

Consider:

import { map, filter, times, noop } from 'lodash'

compared to:

import map from 'lodash/map'
import filter from 'lodash/filter'
import times from 'lodash/times'
import noop from 'lodash/noop'

What the babel-plugin-lodash proposes is to automatically rewrite your Lodash imports to use the second pattern rather than the first. So it would rewrite

import { times } from 'lodash'

as

import times from 'lodash/times'

One takeway from this is that, if you’re already using the import times from 'lodash/times' idiom, then you don’t need babel-plugin-lodash.

Update: apparently if you use the lodash-es package, then you also don’t need the Babel plugin. It may also have better tree-shaking outputs in Webpack due to setting "sideEffects": false in package.json, which the main lodash package does not do.

lodash-webpack-plugin

What lodash-webpack-plugin does is a bit more complicated. Whereas babel-plugin-lodash focuses on the syntax in your own code, lodash-webpack-plugin changes how Lodash works under the hood to make it smaller.

The reason this cuts down your bundle size is that it turns out there are a lot of edge cases and niche functionality that Lodash provides, and if you’re not using those features, they just take up unnecessary space. There’s a full list in the README, but let’s walk through some examples.

Iteratee shorthands

What in the heck is an “iteratee shorthand”? Well, let’s say you want to map() an Array of Objects like so:

import map from 'lodash/map'
map([{id: 'foo'}, {id: 'bar'}], obj => obj.id) // ['foo', 'bar']

In this case, Lodash allows you to use a shorthand:

import map from 'lodash/map'
map([{id: 'foo'}, {id: 'bar'}], 'id') // ['foo', 'bar']

This shorthand syntax is nice to save a few characters, but unfortunately it requires Lodash to use more code under the hood. So lodash-webpack-plugin can just remove this functionality.

For example, let’s say I use the full arrow function instead of the shorthand. Without lodash-webpack-plugin, we get:

Screenshot showing multiple lodash modules under .map

In this case, Lodash takes up 18.59kB total.

Now let’s add lodash-webpack-plugin:

Screenshot of lodash with a very small map.js dependency

And now Lodash is down to 117 bytes! That’s quite the savings.

Collection methods

Another example is “collection methods” for Objects. This means being able to use standard Array methods like forEach() and map() on an Object, in which case Lodash gives you a callback with both the key and the value:

import forEach from 'lodash/forEach'

forEach({foo: 'bar', baz: 'quux'}, (value, key) => {
  console.log(key, value)
  // prints 'foo bar' then 'baz quux'
})

This is handy, but once again it has a cost. Let’s say we’re only using forEach for Arrays:

import forEach from 'lodash/forEach'

forEach(['foo', 'bar'], obj => {
  console.log(obj) // prints 'foo' then 'bar
})

In this case, Lodash will take up a total of 5.06kB:

Screenshot showing Lodash forEach() taking up quite a few modules

Whereas once we add in lodash-webpack-plugin, Lodash trims down to a svelte 108 bytes:

Screenshot showing a very small Lodash forEach.js module

Chaining

Another common Lodash feature is chaining, which exposes functionality like this:

import _ from 'lodash'
const array = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']
console.log(_(array)
  .map(i => parseInt(i, 10))
  .filter(i => i % 2 === 1)
  .take(5)
  .value()
) // prints '[ 1, 3, 5, 7, 9 ]'

Unfortunately there is currently no good way to reduce the size required for chaining. So you’re better off importing the Lodash functions individually:

import map from 'lodash/map'
import filter from 'lodash/filter'
import take from 'lodash/take'
const array = ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10']

console.log(
  take(
    filter(
      map(array, i => parseInt(i, 10)),
    i => i % 2 === 1),
  5)
) // prints '[ 1, 3, 5, 7, 9 ]'

Using the lodash-webpack-plugin with the chaining option enabled, the first example takes up the full 68.81kB:

Screenshot showing large lodash.js dependency

This makes sense, since we’re still importing all of Lodash for the chaining to work.

Whereas the second example with chaining disabled gives us only 590 bytes:

Screenshot showing a handful of small Lodash modules

The second piece of code is a bit harder to read than the first, but it’s certainly a big savings in file size! Luckily J.D. tells me there may be some work in progress on a plugin that could rewrite the second syntax to look more like the first (similar to babel-plugin-lodash).

Edit: it was brought to my attention in the comments that this functionality should be coming soon to babel-plugin-lodash!

Gotchas

Saving bundle size is great, but lodash-webpack-plugin comes with some caveats. By default, all of these features – shorthands for the iteratee shorthands, collections for the Object collection methods, and others – are disabled by default. Furthermore, they may break or even silently fail if you try to use them when they’re disabled.

This means that if you only use lodash-webpack-plugin in production, you may be in for a rude surprise when you test something in development mode and then find it’s broken in production. In my previous examples, if you use the iteratee shorthand:

map([{id: 'foo'}, {id: 'bar'}], 'id') // ['foo', 'bar']

And if you don’t enable shorthands in lodash-webpack-plugin, then this will actually throw a runtime error:

map.js:16 Uncaught TypeError: iteratee is not a function

In the case of the Object collection methods, it’s more insidious. If you use:

forEach({foo: 'bar', baz: 'quux'}, (value, key) => {
  console.log(key, value)
})

And if you don’t enable collections in lodash-webpack-plugin, then the forEach() method will silently fail. This can lead to some very hard-to-uncover bugs!

Conclusion

The babel-plugin-lodash and lodash-webpack-plugin packages are great. They’re an easy way to reduce your bundle size by a significant amount and with minimal effort.

The lodash-webpack-plugin is particularly useful, since it actually changes how Lodash operates under the hood and can remove functionality that almost nobody uses. Support for edge cases like sparse arrays (guards) and typed arrays (exotics) is unlikely to be something you’ll need.

While the lodash-webpack-plugin is extremely useful, though, it also has some footguns. If you’re only enabling it for production builds, you may be surprised when something works in development but then fails in production. It might also be hard to add to a large existing project, since you’ll have to meticulously audit all your uses of Lodash.

So be sure to carefully read the documentation before installing the lodash-webpack-plugin. And if you’re not sure if you need a certain feature, then you may be better off enabling that feature (or disabling the plugin entirely) and just take the ~20kB hit.

Note: if you’d like to experiment with this yourself, I put these examples into a small GitHub repo. If you uncomment various bits of code in src/index.js, and enable or disable the Babel and Webpack plugins in .babelrc and webpack.config.js, then you can play around with these examples yourself.

Why I’m deleting my Twitter account

When I first got on the Internet back in the 90’s, it felt like a cool underground rock concert. Later on, it seemed like a vast public library, maybe with a nice skate park nearby. Today it feels more like a shopping mall. The transition happened so gradually that I barely noticed it.

Hanging out with your friends at the mall can be fun. But it can also be tiring. You’re constantly surrounded by ads, cheery salespeople are trying to get you to buy stuff, and whatever you eat in the food court is probably not great for your health.

For the past few years, I’ve subsisted on a media diet that mostly came from Twitter, consisting of “snackable” news articles with catchy headlines, shareable content with wide appeal (baby koala cuddles baby cat, how cute!), and righteous outrage at whatever horrible political thing was happening that day.

Twitter was often the first thing I looked at when I picked up my phone in the morning, and the last thing I browsed late into the night, endlessly flicking my thumb over the feed in the hope that something good would pop up. The light of the smartphone was often the only thing illuminating my bedroom before I finally turned in (always much too late).

All of this content – cat pictures, articles, memes, political hysteria – came streaming into my eyeballs in a rapid and seemingly random order, forcing my brain to make sense of the noise, to find patterns in the data. It’s addictive.

But the passivity of it, and the endless searching for something good to watch, meant that for me Twitter had essentially become television. Browsing Twitter was no more edifying than flipping through channels. At the end of a long, multi-hour session of Twitter-surfing, I could barely recall a single thing I had read.

Social media as public performance

Twitter is unlike television in a few crucial aspects, though. First off, the content is algorithmically selected, so whatever I’m seeing is whatever Twitter has determined to be most likely to keep my eyes on the screen. It’s less like I’m surfing through channels and more like the TV is automatically flipping from channel to channel, reading my eye movement and facial expressions to decide what to show next.

Second, Twitter has become an inescapable part of my professional life. My eight thousand-odd Twitter followers are a badge of honor, the social proof that I am an important person in my field and worthy of admiration and attention. It also serves as a measure of my noteworthiness in comparison to others. If someone has more followers than me, then they’re clearly more important than I am, and if they have less, well then maybe they’re an up-and-comer, but they’re certainly not there yet.

(This last statement may sound crass. But any avid Twitter user who hasn’t sized someone up by their follower count is either lying to themselves, or is somehow immune to the deep social instincts that mark us as primates.)

For the kinds of professionals who go to conferences, give public talks, and write blog posts, Twitter serves as a sort of “Who’s Who,” except that everyone is ranked by a single number that gives you a broad notion of their influence and prominence.

I’m sure many of my friends from the conference and meetup scene will look at my announcement of deleting my Twitter account as a kind of career suicide. Clearly Nolan’s lost his mind. He’ll never get invited to a conference again, or at the very least he won’t be given top billing. (Conference websites usually list their speakers in descending order of Twitter followers. How else can you tell if a speaker is worth listening to, if you don’t know their follower count?)

Much of that is probably true. I used to get a lot of conference invites via Twitter DMs, and those definitely won’t be rolling in anymore. Also, anyone who wants to judge my influence by a single number is going to have a hard time: they’ll have to piece it together from blog posts and search results instead. Furthermore, my actual influence will be substantially reduced, as most of the hits to my blog currently come from Twitter.

Why I’m done with Twitter

Thing is, I just don’t care anymore. I’ve spent years pouring my intellectual and emotional labor into Twitter, and for countless reasons ranging from harassment to Nazis to user-hostile UI, platform, and algorithm choices, they’ve demonstrated that they don’t deserve it. I don’t want to add value to their platform anymore.

To me, the fact that Twitter is so deeply embedded into so many people’s professional lives is less a reason to resign myself to keep using it, and more a reason to question and resist its dominance. No single company should have the power to make or break someone’s career.

Twitter has turned a wide variety of public and quasi-public figures – from Taylor Swift to a dude who speaks at tech conferences – into brand ambassadors for Twitter, and that ought to worry us. Despite what it claims, Twitter is not a neutral platform. It’s an advertising company with a very specific set of values, which it expresses both in how it optimizes for its core constituents (advertisers) and how it implements its moderation policies (poorly).

Well, it may indeed be a bad career move for Taylor Swift to abandon her Twitter account, but for a (very) minor public figure like myself, it’s a small sacrifice to make to knock Twitter down a peg. My career will survive, and my mental health can only improve by spending less time flicking a smartphone screen into the late hours of the night.

That’s why I’m deleting my account rather than just signing out. I want my old tweets to disappear from threaded conversations, from embeds in blog posts – anything that’s served from twitter.com. I want to punch a hole in Twitter’s edifice, even if it’s a small one.

I’ve backed up my tweets so that anyone who wants to see them still can. I’m also still fairly active on Mastodon, and as always, folks can follow me via my blog’s RSS feed or contact me via email.

This isn’t me saying goodbye to the Internet – this is me saying goodbye to the shopping mall. But you can still find me at the rock concert, in the public library, and in the park.

What is Mastodon and why is it better than Twitter

Mastodon is a Twitter alternative that recently released version 2.0 and has been steadily growing over the past year. It’s also a project that I’ve been dedicating an inordinate amount of my time to since last April – helping write the software, running my own instance, and also just hanging out. So I’d like to write a bit about why I think it’s a more humane and ethical social media platform than Twitter.

Much of the discussion around Mastodon centers on the fact that the flagship instance explicitly bans Nazis. This is true, and it remains a great selling point for Mastodon, but it also kind of misses the point. Mastodon isn’t a single website run by a single company with a single moderation policy. It’s a piece of open-source software that anybody can use, which in practice means it’s a network of independent websites that can run things however they like.

There is no company behind Mastodon. There’s no “Mastodon, Inc.” Mastodon doesn’t have a CEO. The code is largely written by a 24-year old German dude who lives off Patreon donations, even though he’s a very talented web developer and could probably make a lot more money if he joined the industry. He works on Mastodon because it’s his passion.

What this means is that if someone wanted to take Mastodon’s code and build a competing service, they could do so trivially in a matter of minutes. And they do. The original instance, mastodon.social, isn’t the only server – in fact, it’s not even the biggest one anymore. There are over a thousand active instances, and it’s become easy enough that Masto.host can even create one at the click of a button.

In practice, though, these Mastodon instances don’t compete with each other so much as they form a giant constellation of interconnected communities. Users from any server can read, follow, and reply to users on another server, assuming neither of the two servers is blocking the other.

The closest analogy is email: if you use Gmail, you can still communicate with someone who uses Outlook.com and vice-versa, because they both rely on the same underlying system (email). Through its own underlying systems, Mastodon (as well as compatible software like Friendica, GNU Social, and postActiv) forms a network of independent sites referred to as the “fediverse,” or federation of servers.

Why this is better than Twitter

The problem with Twitter is that its incentives are completely misaligned with those of its users. Twitter makes its money from advertising, which means that its goal is to keep your eyes glued to the screen for as long as possible, and to convince you to interact with ads. Its goal is not to keep you safe from harassment, or to ban dangerous extremists, or to ensure your psychological well-being. Its goal is to make advertisers money by selling them an engaged audience.

This is why Twitter will never #BanTrump, even though many have called for it after he began threatening North Korea on the platform. From Twitter’s perspective, Donald Trump increases engagement. Donald Trump gets eyeballs. If Donald Trump started a nuclear war on Twitter then hey, all the better, because Twitter would get a massive boost in traffic, at least right up until the point the bombs started raining down. Twitter even uses Trump in some of its advertising, which gives you an idea of how they feel about him.

Mastodon, by contrast, isn’t run on advertising. Well, instances could add advertising if they wanted to, but I’m not aware of any that do. Most of them, including the flagship, are run on donations from their users. Others get a bit more creative: cybre.space, for instance, allows free signups for one hour each day, but if you donate you can get an instant invite. capitalism.party is an interesting experiment where every signup costs $5. social.coop is run as a co-op. The possibilities are endless, since the underlying code is open-source.

What these instances all have in common is that they’re not driven by the insatiable appetite of marketers for clicks and engagement – instead, their goal is to make as warm and hospitable a place for their users as possible. The incentives of the people who run the platform are aligned with the incentives of the users.

Ultimately, this is why Mastodon instances can implement the kinds of moderation policies that their users clamor for (including banning Nazis). Most instances only have a few dozen to a few thousand active users, and they’re often organized based on shared interests, languages, or nationalities. This means that each instance tends to be small enough and like-minded enough that they can have fairly nitpicky moderation policies (or policies that adapt to local laws and customs), and it’s not too overwhelming for a small group of sympathetic and highly-motivated admins to handle.

Privacy and respect for the user

There are a lot of other benefits to Mastodon’s lack of an advertising model. For one, as a Mastodon user you’re not subjecting yourself to the adware, spyware, and bloatware that we’ve come to expect from much of the modern web. To see what I mean, here’s a screenshot of my instance, toot.cafe, compared to Twitter.com.

creenshots of Twitter vs Mastodon, showing Twitter loading 3.48MB of JS vs 990.84KB on toot.cafe

Besides the refreshing lack of advertising on the Mastodon site (and toot.cafe’s charming purple theme), you might observe that Mastodon is loading less than a meg of JavaScript, whereas Twitter loads a generous 3.5MB. A lot of that extra heft is probably just standard web bloat, but if you have an ad blocker or tracker blocker installed, then you can see another dimension to the story.

Screenshot of Ghostery showing 4 trackers blocked on Twitter.com vs 0 for toot.cafe

According to Ghostery, Twitter.com is loading 4 separate trackers, including Google Analytics, TellApart, Twitter Analytics, and Twitter Syndication. (Those last 3 are all owned by Twitter, so who knows why they need 3 separate trackers for each.) Whereas on the Mastodon site, Ghostery found 0 total trackers.

Screenshot of uBlock origin showing 14 requests blocked for Twitter vs 0 for toot.cafe

Looking at uBlock Origin, we can see it needed to block 14 requests on Twitter.com, or 9% of the total. On the Mastodon site, though, uBlock didn’t need to block anything.

Beyond the lack of ads and trackers, though, these privacy benefits accrue to the data you share with the website itself. On Twitter, you’re handing over your tweets, browsing habits, and photo metadata to a large VC-funded company that makes no bones in its privacy policy about all the various ways it feels entitled to use and sell that data. The terms of service also make it clear that once you post something, Twitter can do whatever it wants with it.

Screenshot of https://twitter.com/en/tos#usContent starting from "By submitting, posting or displaying Content..."

A snippet of Twitter’s terms of service.

Now compare this to Mastodon. On Mastodon, image metadata is stripped by default, links show up as (wait for it) actual links instead of tracking redirects, and some instances even go so far as to specify in their terms of service that you’re not relinquishing any copyright over your content and your data will never be sold.

Screenshot of mastodon.art's guidelines, saying "All content is ⓒ each artist & cannot be distributed or used without prior permission by the respective Mastodon.ART artist."

A snippet of mastodon.art‘s terms of service.

It’s such a far cry from the way we’re used to being treated by online services, with their massive legalese-laden EULAs stripping us of the right to do anything beyond gripe at the rough way we’re being manhandled, that using Mastodon can almost feel like browsing a web from a parallel universe.

So Mastodon is a paradise, right?

I’m not going to pretend that Mastodon is devoid of moderation problems. Yes, the flagship instance bans Nazis and other malcontents, as do most of the other large instances (including my own). There are plenty of instances with their own policies, though, and there’s nothing in the software to prevent them from doing so. So if you want to use an instance that harbors Nazis, or even just libertarians or free-speech advocates, then you can certainly find them.

As you can imagine, though, a right-wing instance that brags about its tolerance toward fascists is not likely to get along with a left-wing instance that bills itself as “anticapitalist”. Thus you will find lots of instances that block each other, creating a situation where you might discover vastly different content and vastly different people depending on which instance you sign up with.

This goes beyond straightforward disagreements between the political left and right. Every so often in the Mastodon community, a serious conflict will arise between instances. Often it starts because two users on two different instances got into a fight with each other, the admins got involved, and they disagreed on how to resolve the dispute. Sometimes it’s the admins themselves who started the fight. Either way, the admins end up criticizing or disavowing each other, the public timeline gets filled with debates on who’s right or wrong, and ultimately one group of instances may decide to block or silence another group. We call this “the discourse.”

“The discourse” tends to flare up every month or so, and when it does there’s usually a lot of moaning about how much drama there is on the fediverse. This lasts for a day or two and then things go back to normal, albeit with a slightly more bifurcated community than we started with.

Discourse and disintegration

I don’t enjoy “the discourse,” and I tend to agree with folks who argue that it could be alleviated if Mastodon had better tools for resolving inter-admin conflicts. I don’t think this problem can ever be completely eliminated, though. Human beings are just naturally inclined to seek the company of those they agree with and shun those they disagree with. This has the unfortunate effect of creating filter bubbles, but it turns out human beings also have a boundless appetite for filter bubbles, as evidenced by the churches, clubs, meetups, and political parties where we seek those who are similar to us and give a cold shoulder to outsiders.

I don’t believe it’s Mastodon’s job to correct the problems caused by the right to free association. But Mastodon could improve the process of communities splitting into smaller, more harmonious networks of people with shared values and mutual tolerance for one another.

Furthermore, a lot of these disputes boil down to a difference of opinion over what constitutes harassment, abuse, hate speech, etc. So in a way, “the discourse” can be seen as a testament to the seriousness with which these subjects are treated on Mastodon. Instance admins care so much about the well-being of their users and protecting them from disturbing content, that they routinely argue and even block each other over the best way to implement it.

Now compare that situation to Twitter. On Twitter, there’s one moderation policy, and if you don’t like it: tough. Whereas on Mastodon, if you don’t like your instance’s policy, you can always switch to another one. (And there’s work in progress to make that migration easier.)

Conclusion

Mastodon is not perfect. The software is still rough in some places, the underlying protocols (OStatus and ActivityPub) are still getting hammered out at the W3C, and the community devolves into tiresome bickering more often than I’d like.

But I still have more faith in Mastodon than I do in Twitter, whose user growth has flatlined and whose profits are nonexistent, and thus will have to resort to increasingly desperate measures to satisfy its investors, who are still waiting for a sweet return on investment for all those eyeballs they bought. I expect this will mean more promoted tweets, more ways to promote tweets, and ultimately less value for Twitter’s users, as they become increasingly drowned in a sea of brand accounts trying to sell them a hamburger, fake news trying to swing an election, and bots trying to do who knows what. Meanwhile the harassment problem will never be Twitter’s main priority, despite what their CEO says, because as long as controversy and conflict are good for grabbing eyeballs, they’re good for Twitter’s bottom line.

The main reason I’m hopeful about Mastodon is that it’s an opportunity to learn from Twitter’s mistakes and to experiment with fresh ideas for improving social media. For instance, how about disabling public follower counts, since they can make us feel like we’re living in a Black Mirror episode where everyone’s self-worth is determined by a single number? (In fact witches.town already does this; every user’s number is a cheeky 666.) Or how about removing the “quote-repost” feature, since we saw the nasty dog-piling it enabled on Twitter? Or how about adding features that encourage users to log off every once in a while, so that social media doesn’t turn into an addictive slot machine?

All of these things are possible in Mastodon, because the code is open-source and the servers belong to the users. We can even tinker with these ideas at the instance level, to test how something pans out at the small scale before bringing it to a wider audience. Instead of Twitter’s one-size-fits-all approach, we can tailor social media to fit the needs of every community, with local admins who are motivated to help because they’re moderating a small group of like-minded people rather than 300 million of them.

Mastodon can feel like a return to another time, when the web was small and it felt possible to actually have an impact on the websites we use every day. But it’s also a glimpse into the post-Twitter future that we need, if we want to have control over our data, our minds, and our public discourse.

Interested in Mastodon? Check out joinMastodon.org or instances.social for help finding an instance to join. If you’re not sure, I’d recommend toot.cafe (my own), cybre.space (cyberpunk themed), mastodon.art (for artists), awoo.space (focus on safety), or for general interests: mastodon.social, icosahedron.website, or octodon.social.