Web/IMAP/POP frontend proxies changed to nginx

A while back, we changed our frontend IMAP/POP proxy from perdition to nginx. Perdition uses a traditional unix “one process per connection” model to manage the proxying of IMAP/POP requests. Because of the long lived nature of IMAP connections, perdition was using over 8,000 processes on each of our frontend machines. Even with Linux 2.6 and the O(1) scheduler, the machines were beginning to struggle with the large number of processes, creating a sluggish feeling to IMAP connections.

Instead of a process per connection, nginx uses a small fixed process pool and non-blocking code with epoll (on linux) to provide much higher scalability. At the time we first looked at nginx, it only supported HTTP proxying, but we realised the underlying architecture would be a good one for IMAP/POP proxying as well. With that in mind, we contacted the author of nginx (Igor Sysoev who we were already familiar with due to mod_accel) to implement an IMAP/POP proxy in nginx. We agreed to pay him for this, and to allow the code to be included in the regular nginx distribution. Over the next couple of months he implemented it, and after some testing and bug fixing, we rolled it out to our frontend proxy servers in Sep 2005. The results for us were dramatic. Load on our frontend servers dropped dramatically, and IMAP/POP responsiveness improved noticeably.

Because web connections aren’t as long lived as IMAP connections, we stayed with Apache for our frontends for a while longer. However we’ve now switched over to using nginx for our frontend web proxy as well, which has also allowed us to increase the keep-alive timeout for HTTP connections to 5 minutes, which should result in a small perceptible improvement when moving between pages.

The net result of all this is that each frontend proxy server currently maintains over 10,000 simultaneous IMAP, POP, Web & SMTP connections (including many SSL ones) using only about 10% of the available CPU on
3.20GHz Netburst Xeon based CPUs.