Dec 17: Building offline: general architecture

This is the seventeenth post in the Fastmail Advent 2024 series. The previous post was Dec 16: Offline support now in public beta. The next post is Dec 18: Building offline: syncing changes back to the server.

Yesterday we announced full offline support for Fastmail is now available in public beta, both in our app and on the web. Today, and over the next few days, I’ll dive into some of the technical aspects about how we’re making this work.

Making Fastmail work offline has been our most popular feature request for some time (ever since we added support for custom swipe actions, our previous top request!), and we wanted to make sure our support was done right. It should just work, seamlessly, and as far as possible you should be able to do everything you can do online. Open your calendar and update an event, perhaps inviting someone using autocomplete from your contacts. Search your mail and triage it. Write a new note. Add a memo. We want it all to just work.

When you come online it should seamlessly sync these changes back to the server, and fetch any new mail and other changes.

General architecture

The Fastmail app is very cleanly separated from our server, which was a huge benefit when adding offline support. All data in the app is loaded via a JMAP API, with all UI rendering and routing happening in the app. This means the data flow looks a bit like this:

[App] ← JMAP → [Server]

This gave us a really well defined boundary on which to build the offline support. If we built something that could handle and respond to the JMAP requests directly on your device then the app would work offline, and we wouldn’t really have to change anything else in it. So the architecture we came up with looks like this:

[App] ← JMAP → [Caching layer] ← JMAP → [Server]

Looking at our new diagram, we can see that we are building two things:

1. A JMAP server that can understand the requests the client makes, fetch and write the data from/to a local store, and return a JMAP response.
2. A JMAP client that can fetch data it doesn’t have from the server, and write back changes made while we were offline.

This is actually quite a lot harder than just building a JMAP server! We will often only have partial information, and we have to transparently pass through requests for data we don’t have to the server, and gracefully handle a fallback if we are offline.

The core function our caching layer needs to perform is handling a JMAP request from the client. Each JMAP request is a sequence of method calls. Once we have locally cached data, we may be able to handle the method call entirely locally, but we may always run into one or more that we can’t.

For performance, we want to batch our method calls into a single database transaction where possible. But we can’t hold open a transaction over a network request, so we divide up the execution into phases.

First, we attempt to execute the method calls locally. If all complete, we’re done! If any require us to fallback to the server, we stop and send it everything from that point on, as we want to avoid making multiple HTTP requests, which would be slow.

If the request completed successfully, we process the responses to save any new data into our local datastore. If the request failed, we call the offline fallback methods, which may be able to still return a response to the UI.

A separate thread for the caching layer

The caching layer runs on your device, just like the rest of the app. Because JMAP requests are already asynchronous network calls from the UI, we can easily run the caching layer in a separate OS thread so it never blocks the UI thread, which could cause jank.

Our app is built using web technology, which allows us, as a small company, to build an app that runs everywhere our users are, with a single code base and feature parity across all platforms. Separate threads are represented as workers in the web API. There are three types of worker:

• Dedicated worker — this is a worker that is tied to a particular window or tab in your browser. If you have Fastmail open in multiple tabs, each would have to create its own worker.
• Shared worker — this is a worker that’s shared between tabs or windows, so no matter how many you have there’s only a single instance of this worker.
• Service worker — this is a special type of shared worker that can intercept network requests and change their response.

At first glance, a service worker seems the place to handle all of this, and this is what we tried first. However, we soon switched over to using a shared worker instead:

• The service worker is designed to be short lived and only spun up when needed, but we wanted to hold open a persistent EventSource push connection while the app is open, for instant updates. The shared worker is a better conceptual fit for this.
• We don’t need to intercept network requests, as we can just pass the JMAP request object directly to the worker using the postMessage API, avoiding some serialisation overhead.
• We ran into a bug in iOS where network requests would sometimes not be intercepted even though the service worker was registered when the app was running in the background. This meant we had to pass the request directly to the worker anyway instead of intercepting it at the network level to ensure we didn’t hit this bug.

We wanted to use a shared worker rather than a dedicated worker for more efficiency when there were multiple tabs open — we can avoid some contention and locking issues, and ensure we have a single push connection open to the server.

Storing data: IndexedDB

The web API for storing large volumes of structured data is IndexedDB. This lets you create multiple object stores (the equivalent of SQL tables), which offer simple key-value storage with ordered keys. Indexes can be automatically built based on properties in the object being stored. Transactions ensure data consistency.

The IndexedDB API was unfortunately designed just before promises became ubiquitous in the web world. This means just fetching a record from a store requires code a bit like this:

const request = store.get(id);
request.onerror = () => callErrorHandler();
request.onsuccess = () => {
    const record = request.result;
    // Do something
};

This is clunky and becomes hard to read and follow. However, we wrote one tiny little wrapper function that converts it into a promise-based API:

const _ = (request) =>
    new Promise((resolve, reject) => {
        request.onsuccess = () => resolve(request.result);
        request.onerror = () => reject(request.error);
    });

Using this function, we can rewrite the above fetch like this:

const record = await _(store.get(id));
// Do something

(Note, if the fetch has an error this will result in an exception being thrown, which is generally handled at a higher layer, so avoids that cluttering our code here at all!)

With this simple addition, I found the IndexedDB API consistent and easy to work with.

The standard object store structure

The consistency of JMAP means we can write one generic implementation and then use it to provide offline support for all our data types. For each data type (such as Calendar, Email, Contact, etc.) we create an object store to store the instances of that type.

When we create the object store we also store a single metadata object in it, using a special key (a zero-byte ArrayBuffer). This stores some important bookkeeping information, in particular the following properties:

// Do we have the full set of data from the server for this data
// type?
hasAllRecords: false,

// State string representing the current server state we have
// synced with. The store may also contain newer information, but
// that's ok as it will still get to the correct state when we
// update from the old state.
serverState: '',

// This is the highest modseq of a record in the store.
lastModSeq: 0,

// This is the highest modseq of a record that was destroyed that's
// now been removed entirely from the store; we can only calculate
// changes accurately from this point on.
highestPurgedModSeq: 0,

// This is the number of records currently marked destroyed in the
// store. We keep them there so we can calculate changes. Once we
// cross a threshold, we'll clean up old ones.
numDestroyed: 0,

A key concept here is modseq, which stands for “modification sequence”. It’s a counter we keep per account, per data type. Every time we make a change to a record in our local store we bump the sequence number and assign that as the new “updated” modseq for that record. We also store a “created” modseq on each record, which is the same as the “updated” modseq when the record is first created. These simple bookkeeping properties allow us to efficiently calculate changes, as needed for the JMAP “/changes” method.

Aside from the metadata object, every other entry in the object store is a record — an instance of that data type.

The key for each record is [account id, id], because some data types exist in multiple accounts (e.g. shared contacts and your personal contacts) and ids are only unique within an account. As far as I could see from inspecting the source, string keys are stored as UTF-16 in all major IndexedDB implementations. This is a fairly inefficient encoding, especially as we know JMAP ids can only use the base64url characters, so for efficiency we encode this data into an ArrayBuffer, making use of this fact.

The value associated with the key is the record itself — an object representing an instance of that data type, as fetched from the server. In addition, we add a few bookkeeping properties, as discussed above:

• The created modseq
• The updated modseq
• Is the record destroyed?

Each object store has an index built automatically based on the updated modseq of the records.

The modseq is used as the “state” string over JMAP. When asked for what’s changed since a particular state, we know:

• Only records with a higher modseq have changed (which we can efficiently get from the index).
• If the record’s created modseq is higher than lastModSeq, it’s new. Otherwise it’s been updated or destroyed (depending on whether the record is now destroyed). If it’s new and also destroyed, we can ignore it entirely.

Next up, local changes

In this post we looked at the basic overview of how our offline caching layer fits into our app, and the way it stores data to efficiently respond to JMAP requests. Tomorrow, we’ll dive into how it keeps track of changes the user makes while offline, so it can reconcile this with the server.