All About ARC - an experimental standard for authenticating forwarded emails
Post categories
Deliverability and Operations
As email passes through servers from sender to receiver, ARC (an experimental standard) tracks its authentication state. Read about how we use ARC at Fastmail, and how ARC can be used to benefit everyone.
In the fight against fraudsters and scammers, an important tool is being able to trust that the sender is who they say they are (DMARC) and that the message didn’t get modified along the way (ARC). Let’s look in more detail about ARC, what it is, how we use it, and why it’s important to the future of email.
What is ARC?
We have written before about ARC, an experimental system to enable recipients to evaluate a chain of senders in a secure and trusted way.
The stated purpose of ARC is as follows…
“ARC preserves email authentication results across subsequent intermediaries (“hops”) that may modify the message, and thus would cause email authentication measures to fail to verify when that message reaches its final destination.” - http://arc-spec.org/.
ARC builds on the established DMARC standard. DMARC allows domain owners to publish a policy stating how email:
- Either was sent directly from a listed server IP address and verified using the SPF standard, or
- Has not been modified since it was sent by checking it has a valid DKIM signature for that domain.
DMARC fails to handle cases where mail doesn’t flow directly from sender to recipient. This is common with forwarding services and mailing lists.
ARC is an experimental standard and is intended to enable a better understanding of these DMARC failures and to build in support for these indirect mail flows.
Fastmail’s involvement in building the ARC experimental standard
We believe in collaborating with other providers to produce and support common email standards through organizations like the IETF and M3AAWG.
At the Prague IETF99 hackathon in 2017, our CEO Bron Gondwana started work implementing ARC in open source as part of the Mail::DKIM Perl module and added it to our mail authentication software, Authentication Milter. This was built upon with additional work by John Levine and Valimail, to make sure Mail::DKIM passes the ARC test suite.
That’s where I come in. Hi, I’m Marc, and I work at Fastmail doing specialist work on email authentication using standards like ARC, DKIM, DMARC and BIMI (more on that in another post). Subsequently, I updated the Authentication Milter ARC handler with the existing DKIM, SPF, and DMARC handlers to provide an ARC-aware DMARC validator.
After the October 2018 M3AAWG Meeting in New York, we spent a day with a number of other providers to make sure our ARC implementations could talk to each other properly, and the results of our testing showed that mail passing through our systems would meet the requirements for the ARC standard.
We are now not only validating and adding ARC headers, but we are also actively using them to authenticate mail that flows between trusted systems.
Why ARC is important
In short, without ARC, mailing lists and mail forwarders are often broken when it comes to authentication.
Mailing lists re-send mail from their own servers, and will usually modify that mail, for example, by adding footers to include unsubscribe links and changing headers to ensure replies to mail are returned to the mailing list, not the original sender.
Prior to ARC, when Fastmail received an email from a Topicbox mailing list, it would check the sending server against the From address in the mail. This would fail, because the mail came from a Topicbox server and not the sender’s server as it expected. It would also check that the mail had not been modified, which would fail again, because Topicbox added footers to the message.
DMARC checks for mailing lists often result in a fail result, necessitating workarounds such as more lenient application of DMARC checks for known mailing lists, or re-writing From headers.
How ARC works
With ARC, we can improve on this, or more correctly, with ARC, we are now able to determine who modified the message and caused the authentication to fail, and can see what the authentication state of the message was before it was modified.
As we can see who modified a message, we are able to make policy decisions based on the state of the message as it was when it was received by a trusted third party. What ARC doesn’t help with is discovering how or why a message was modified, so we still need to make sure the list of services we trust is carefully curated.
When a message comes into Topicbox, an Authentication-Results header is added; this shows the state of the email when it arrived at Topicbox, including the sending server (SPF results), and cryptographic signature (DKIM results); given that the message came in from a legitimate source these will both pass.
Topicbox alters the message, removes the original DKIM signatures (as these will now fail), and in the case of domains with strict DMARC policies, will also rewrite the From headers (but more on that later).
Topicbox then re-signs the message as it leaves; this shows the state of the now-altered message when it leaves Topicbox.
The message arrives as Fastmail, and once again is evaluated for Authentication-Results. This time, the result is a DMARC fail, as both SPF and DKIM now fail. However, as the message has a valid ARC chain, we can check back over that chain. We find signatures from Topicbox (who we trust) and are able to extract the passing SPF and DKIM results from that chain.
Although the DMARC policy check failed, ARC has shown that the message passed SPF and DKIM when it arrived at Topicbox, and as we trust Topicbox to “do the right thing,” we can override the DMARC policy fail and allow the message to pass through.
So, what’s the header rewriting all about? ARC is an experiment and is not currently widely deployed. As a workaround, Topicbox will not send out an email that it knows would fail DMARC for any domain with a quarantine or reject policy. It will change the From header to be from a special Topicbox domain instead.
Fastmail maintains a list of trusted forwarding and mailing servers, and we perform these ARC actions and checks for those as we did for Topicbox. For forwarders not on our trusted list, or forwarders who do not use ARC, we cannot use ARC to evaluate the path of the mail, and these are more likely to run into DMARC policy issues.
The future of ARC
As the list of providers deploying ARC grows, and as trust between providers is established, these workarounds will become less necessary. While some of the large providers are already on board with ARC, the challenge includes adding smaller providers and mail forwarders,
The ARC Community Sealers Project is building a list of trusted ARC sealers which providers can use to bootstrap their trusted list.
An industry-wide group of interested parties has been created to discuss the issues with ARC trust, issues relating to email authentication in general, and how to improve the handling of these challenging indirect mail flows.
Fastmail is a part of this group, and we continue to work with others in the email industry to make the ecosystem better for everyone.
Support email innovation by creating a Fastmail account today! Start a free trial and download our mobile apps during your trial.