Scaling Our Infrastructure for 4+ Billion Emails in a Single Day

Scaling Our Infrastructure for 4+ Billion Emails in a Single Day

This post first appeared on the Twilio SendGrid blog.

Any marketer or business owner from small to large enterprises know the importance of Black Friday in the United States as the busiest shopping day of the year. As you prepare for this, Twilio SendGrid prepares to scale up our infrastructure to manage billions of emails reliably, quickly and securely. The hard work of our talented teams has paid off–demonstrated by our ability to support our customers sending a total of 2 trillion emails by March of this year.

To provide a clearer understanding of our scalable systems, on Black Friday 2019 we processed 4.1 billion emails and this Cyber Monday we processed 4.2 billion emails (46% more than 2018) and processed up to 315 million emails/hour (burst rate) into inboxes all around the world! To give you a better understanding of this scale, you can get to the moon and back 4 times in fewer yards than the number of emails we sent on Black Friday.

A common question I hear from SendGrid customers is how fast will my emails get delivered to the inbox? The short answer is usually very fast and in the order of seconds, but the long answer may surprise you.

Calculating “Time to First Attempt”

Some email service providers (ESPs) advertise their time to inbox, but don’t share how it’s calculated. There are many factors that affect this time–some that the ESP controls and some that the mailbox providers do (such as what happens after email processing and delivery).

To keep us honest and remove any confusion, we have come up with a new terminology for this metric called, “Time to First Attempt” or TFA. This terminology is defined as the time it takes from when we receive an email API request to when we first attempt to deliver the email to the mailbox provider server.

This system has enabled us to achieve a median TFA of 1.9 seconds over the last 3 months.

Here at Twilio SendGrid we take pride in innovating and optimizing our proprietary Mail Transfer Agent (MTA). While many email delivery services rely on third-party vendors to supply their MTAs, we are in full control of our MTA. Controlling our MTA has empowered us to be scalable, reliable, and agile with the ability to handle billions of emails expeditiously.

The Journey Begins

Now let’s dig in deeper and look at the journey an email takes from the time the sender initiates an email API or SMTP to the recipient’s mailbox. Although this journey takes only a few seconds, there are many steps involved when an email request is launched.

The first step is for us to receive your requests as fast as possible so we can start processing and delivering. Believe it or not, location matters. Although the proliferation of computers have shortened long distances throughout the world, milliseconds matter to us!

Hence we have established endpoints in different parts of the world to handle emails that originate from various parts of the globe. We cover the entire world by distributing these endpoints in different parts of the U.S., Europe and Asia.

Twilio SendGrid’s Secret Sauce

The next journey for an email is authentication, processing, and delivery–that’s where Twilio SendGrid’s secret sauce resides. After checking the user’s permissions and right by authentication, emails are built/processed and channeled to our MTA for delivery and relaying to the recipient mailbox providers like Gmail, Outlook, iCloud etc.

Every mailbox provider has a different connection and throughput setting for transferring mail to their servers. These settings are not published and must be established as baseline recommendations by monitoring and adjusting based on signals from the mailbox provider.

Twilio SendGrid monitors connection and throughput settings regularly and makes adjustments to ensure optimal delivery. Mailbox providers can still limit connection and throughput based on the reputation of you, the sender.

Twilio Sendgrid process both internal and mailbox provider signals so we can proactively, at times temporarily throttle your mail to confirm it is not abusive, while protecting your reputation and deliverability from mailbox provider detection and throttling.

Other factors such as incorrect email addresses, full mailboxes, and spam reports are controlled by mailbox providers and can result in bounces, deferrals, and blocks which can cause delays in email delivery time. We can control the burst rate, processing, and relaying your email to the mailbox providers efficiently; however, the rest of the journey is the job of mailbox providers.

Mailbox Provider Filtering

Mailbox providers protect their customers’ inboxes and cater to their customer preferences by doing their best to filter unwanted or spam emails. Sometimes this filtering might even cause legitimate emails to get filtered erroneously.
As a sender you can maximize your sending reputation yourself by:

1. Authenticating your sender identify by setting up your SPF, DKIM and DMARC
2. Exercising email hygiene by using an email validation service
3. Cleaning your contact list regularly to remove chronically unengaged recipients.
4. Using best practices for choosing what content to include in your emails
5. If using more than 1 dedicated IP, segmenting your transactional vs. marketing IP(s) and domains

As an email infrastructure provider, we provide: excellent uptime 99.999%, fast burst rate, geolocation processing, optimized scalable MTA, mailbox provider feedback loop, and handle all of the low-level protocols to help you with the efficient and prompt delivery of your messages.

As a sender, you must build trust and a good reputation with the mailbox providers by following best practices. Since we understand that email delivery can be complicated and not everyone is an expert in deliverability, we offer deliverability consultation services to maximize your delivery.

Sara Saedinia is a product manager on the Email Pipeline and Delivery teams. Sara also works cross-functionally with other teams to bring new products and features to meet customers' needs. In her free time, she enjoys creating sculptures and ceramics with hands or on the wheel.