Mobile email/teleflip.com blocked (updated)

Update 11-Oct-2007: teleflip.com have contacted us and changed over to using SSL IMAP connections rather than screen scraping web pages, a much better approach to retrieving emails. Thanks teleflip.

I just noticed that one of our web servers was getting noticeably more loaded than the others. After some investigation, I tracked the problem down to a site called teleflip.com. Basically it appears they provide a way to get and send email on your cellphone in the US. A nice idea, but it seems the way they implement at least the “get” part is horrible and unacceptable.

Rather than using standard email protocols like POP or IMAP which are designed to download emails, especially IMAP which lets you download just the important meta-data for emails such as From address, Subject, Date, etc, and then get the parts of an email you actually want, they seem to be logging into our web interface and paging one page at a time through users mailboxes, some of which are 1000’s of pages long.

In fact from our logs, I see that they appear to be logging in for only 8 different users, but appear to be generating over 1,000,000 web requests a day! This is absolutely insane behavior. And to top it off, rather than using a User-Agent field that identifies them as a bot type site, they’re trying to hide themselves as a standard version of Firefox using “Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050518 Firefox/1.0.4”.

Unfortunately because of this activity, I’ve had to block all the teleflip.com IPs (64.74.168.128/25). I don’t intend to unblock them, because using the web interface to scrape email data is completely the wrong approach. They should be using POP or IMAP, and if they had some real sense, they’d use IMAP IDLE to actually get notification of when new emails arrived so they could push just those new emails to your phone.

Until teleflip.com improve their systems, we recommend that you do NOT use them.

For users looking for the best possible mobile email experience, we recommend using ChatterEmail with a Palm Treo. ChatterEmail is a full IMAP client on a phone, including IDLE support to handle push email events. For Windows Mobile users, we recommend looking at FlexMail. It’s not quite as powerful as ChatterEmail, but seems to be the best email client for Windows Mobile phones at the moment.