iOS 7 Mail App uses multi-folder body searches by default

This is a technical post. Regular FastMail users subscribed to receive email updates from the FastMail blog can just ignore this post.

We’ve recently been testing out the Mail application in iOS 7 and found that there’s been a couple of significant changes, especially when you search your email. If you run an IMAP server with users that connect using the iOS Mail application, you might be interested in what we found. The two main ones are:

1. Rather than searching the current folder, by default, it searches
   all folders. To do that, it opens many IMAP connections at once for
   searching each folder concurrently. We’re not sure on the upper
   bound on the number of connections it will make, but saw at least 10
   in one case.
2. Rather than searching the Subject/To/Cc/From fields, by default, it
   searches all those fields as well as the message body.

Both of these changes are actually great for the user experience, but they create potentially large headaches for IMAP server maintainers.

Clients making multiple IMAP connections at once isn’t new, but the number of potential IMAP connections an iOS client might now make is large. Some administrators limit the total number of IMAP connections a user can make at once. They might have to raise this or iOS might start returning unexpected results.

The bigger concern is the body searching. The search string iOS now sends is:

tag UID SEARCH RETURN (ALL) (OR FROM “Foo” (OR SUBJECT “Foo” (OR TO “Foo” (OR CC “Foo” BODY “Foo”)))) NOT DELETED

IMAP search semantics suggest that a body search is supposed to be a pure sub-string search (BODY : Messages that contain the specified string in the body of the message). Depending on your IMAP server, your message bodies may or may not be indexed in a way that allows sub-string searching. If they’re not, then a BODY search is potentially very expensive, requiring every message to be opened and searched individually. Doing that simultaneously across many folders might generate a sudden and significant IO load on a server.

Our plan at FastMail is to detect iOS clients, and convert all searches into FUZZY searches. This causes matches to be done on “terms” rather than pure sub-strings, but allows us to use our xapian powered index which should make matching and fetching results much, much quicker.