Emails and dates/times

There’s often some confusion with people regarding emails and the dates/times (datetime) shown on the email. The main problem is that there’s actually two main different datetimes associated with every email. There’s the sent datetime and the received datetime.

In many cases, these values are sufficiently close (a few seconds) that it doesn’t really matter which you see or use. However in some cases (eg delayed email, or email retrieved from another server to FastMail via a Pop Link) the datetimes can be significantly different, causing some confusion.

Mostly this revolves around that emails on the Mailbox screen show and are sorted by the received datetime, but if you view the headers of an email, the “Date:” field that most people see is actually the sent datetime. In fact historically, the
received datetime is something that is created and stored internally in the email server itself when the email is delivered, it’s not visible as a field in the email headers at all.

Because the received datetime is only stored internally in the email server separate to the email itself, a number of things can cause this information to be lost, such as restoring emails from a backup, copying emails from one server to another, POP retrieving them from the server, etc.

To avoid the confusion this was causing, we decided to try and come up with a more consistent way of representing the internal date that would be stored with the email itself.

We decided to do this by noting that when an email is delivered to one of our email servers, the final hop in the email "Received:" headers shows this delivery step as something like this:

Received: from compute1.internal (compute1.internal [10.202.2.41])
     by store123m.internal (Cyrus v2.3.9) with LMTPA;
     Sat, 13 Oct 2007 04:27:43 -0400

The final part of that header shows the exact time the email was delivered to the mailbox (as per the definition of Trace fields in RFC2822). By using the time in the final “Received:” header, we basically have something that is constant in the email itself as something to reference when an email is moved, or restored from backup, or POPed from another server. This does have one potentially negative side-effect related to using IMAP to APPEND messages to a mailbox, but this is both a relatively rare case, and also we still believe the consistency of using the last “Received:” as the received datetime reference outweighs the potential negative.

At the same time we did this, we also wanted to allow users the option of when they used POP Links to retrieve emails from another server to use either the datetime the email was actually downloaded from the POP server as the received datetime, or use the “Date:” header on the email (eg sent datetime) as the received datetime. The way we decided to do this was by allowing a special “X-DeliveredInternalDate:” header. If that header is present, then it should be a datetime value in the standard RFC format, and that value will be used as the received datetime instead of looking at the datetime in the last “Received:” header. This provides flexibility in setting the received datetime, but at the same time it ensures that the received datetime won’t be lost by internal movements in and out of the server. The use of this is controlled on the Pop Links screen on each Pop Link by the “Received date” options which can be set to either “When POP occurs” or “Message sent date”.

This setup has been in place for about 6 months now and worked very well, giving a good balance of consistency and flexibility. The only issue that has really surfaced is that some programs use IMAP to APPEND emails with no “Received:” headers at all (eg Bynari Connector).