Where Do Text Messages Come From?

Where Do Text Messages Come From?

“…and when you click “SEND”, the API Stork scoops up your text message in its beak and flies a great distance to drop the message off to your recipient.”

That doesn’t seem accurate. To get to the bottom of where texts come from we talked to Ben Stein, who is the man leading the messaging team here at Twilio. He sorted out the stork debate.

 
Listen to our interview with Ben Stein below
or find the Twilio Radio podcast on iTunes.

Why are you such a fan of SMS?

There’s so many things that point to SMS as not being a great way to communicate, but it turned out to be the most popular form of communication on the planet. It’s the only app that’s installed on every single phone ever. Email is not installed on every phone, but SMS is.

And there’s something about short messaging and the ability for businesses to communicate using SMS, to address everyone in the world globally, that is really, really powerful. The response rates that you see, the conversion rates that you see, the open and read rates that you see, they were staggering ten years ago and they’re staggering to me still today. And you just don’t see that with any other medium. The companies who adopt messaging, specifically SMS, and do it well, just see so much success. And the thing that drives me in my work is making customers successful.

What are the open rates for SMS?

All of them.

Pretty darn close to all of them, right? It’s almost silly to talk about open rates because you just read them. The conversion metrics vary based on the company and the call-to-action. But I guess if you were to compare to something like email, which is a common comparison (I don’t think it’s a fair comparison, but it’s common) it absolutely dwarfs email. You may see a conversion rate of 2% in your email and 40% in your text message.

That said, it is a very, very personal form of communication, and it needs to be treated as such. You’re making someone’s pocket vibrate. That’s a lot of power that companies have: the ability to make people’s pocket vibrate. It’s almost bizarre when you think about it. That is what SMS gives to you. When treated with respect you can really get some pretty powerful results.

One other thing about SMS is, in general, people respond. You send somebody a text, they’re generally going to write back, which is not something that you had in the email world previously. People will actually just respond. Even if it’s just a PIN code, people just respond, “Cool, thanks.” People write back. That’s something that I’ve seen for a long time but we’re kind of watching businesses slowly start to realize that people do respond back, that it’s a two-way conversational medium, that people message each other and they’ll message your application and your business in the exact same way.

How does a text message get from one phone to another phone?

The protocol for SMS is called SMPP. Not to be confused with SMTP, which is the mail protocol. SMPP is a store and forward protocol. The way it will typically work is you have a client, the ESME (External Short Messaging Entity), which is going to connect up to the server which is the SMSC (Short Message Service Center). A persistent connection is made between those two endpoints. Text messages and other control commands flow bidirectionally: messages coming in, messages going out, and something called delivery receipts, which tell you the status of a message.

It’s not necessarily an inelegant protocol, and it technically is extensible, but you’re limited by adoption. In order to have this lowest common denominator communication all around the world, you need to use the features which are supported by all the servers and all of the handsets.

Generally, you’re limited in the protocol. For example, you want to stick to 160 characters. There are neat features like concatenation, where you can make messages much, much longer by concatenating them together. However, that’s not supported by every handset by every carrier. Especially when you look globally, support really varies. If you want to keep that consistent user experience, you want to stay lowest common denominator.

What about MMS?

MMS is a completely different protocol than SMS. With the Twilio API, you can send a text message and you can also attach an image. And most of the world thinks of texting a picture as just being a text message with an attachment. It’s kind of like, “In email I attach a file, so in SMS I attach a file.”

Under the cover that’s just not the case at all. It’s a completely different protocol. There’s actually two different protocols typically: MM4 and MM7. They are HTTP protocols. If you think of SOAP web-services and SMTP, the mail protocol, these are the protocols that are used for sending binary content — pictures, but also videos, vCards, really any type of binary content.

So one nice thing about the Twilio API is developers don’t care. You’re sending a text, you have a “media” parameter, and that’s it. Under the covers, we actually will switch to a completely different protocol. There’s just nothing the same about them.

What’s the carriers’ role in all this?

There are over a hundred carriers in the US. People usually think there are about four of them, but there are many. And there are roughly a thousand around the world. When you think about Twilio’s real value prop, it is providing connectivity into all of these carriers all around the world.

The person who’s holding the phone, they have a direct contract with their carrier and in order to get a text message to appear on that phone the message has to get routed from Twilio to Verizon’s network, and then from Verizon down to that handset.

Remember, before Twilio, before all of this was available to web developers, carriers have been able to interact and send messages to one another for quite a while. It’s important for carriers to know where a subscriber is, what carrier they’re on, and also things like “Are you roaming?”

If I go to London, I still get my phone calls and messages. The carrier infrastructure and the work that the carriers have done building out the phone network is incredible. We really take it for granted. We take for granted that you have a phone number and that you can move from carrier to carrier and keep your number. We take for granted that you can be addressed globally by, plus or minus, everyone in the world who is on a different carrier. There’s a lot of things that the standards bodies and operators around the world have done over the years that are really quite remarkable. We just take it for granted that a text message shows up.

What is Twilio’s API and what does it give you?

Twilio has a bunch of different APIs. For the most part, these APIs give our customers the ability to build communications into their apps. We do it with voice, video, and messaging. Specifically, within telecom messaging, it’s going to be SMS and MMS.

We have a REST API and we work pretty hard to keep it as simple as possible. In general, to , from , and body are all you really need. You can add media if you want to add binary content, but that’s pretty much all you need to send your first text message. We take that incoming HTTP request, authenticate it, parse all the parameters, and then convert it into the SMPP protocol because we have to route it to carriers all around the world.

There’s a lot of work that goes on under the covers. I’ll give you an example: character encoding. Developers use our API and they’ll send us Unicode, as they should. UTF-8 everywhere. They take it for granted that the whole world works that way. That’s just absolutely not the case when you look at twenty years of telecom messaging. There’s eight or so different character encodings that SMS sees. Again, these are the servers that speak the SMPP protocol and they’re not going to be unicode. They’re all different types of character sets. One of the very first things that we need to do is correctly convert everything we get from a developer — even emojis — into the correct character encodings.

Then we figure out where and how to route that message and what it’s going to cost. Then we actually deliver that message. Then we provide status callbacks to our developers in the form of webhooks. A developer can register an endpoint and say, “Hey, when this message changes status from queued to sent  or failed , send me a notification via a webhook.” (You could also query your API to get the message disposition.)

The developer experience is all REST HTTP. It feels great. But under the covers, it’s all telecom protocols.

Sort your SMPP from your SMTP and get your EMSE connected to your SMSC because we’re going to want to text you to find a spot to meet up at SIGNAL so we can talk about unicode and which emoji you encode the most frequently. Cool? Cool. (psst…I send the burrito emoji all the time).

Ben and a ton of other developers from the Messaging team will be at SIGNAL, breaking down what you need to know about SMS, and all the acronyms that history has burdened us with. If you’d like to register, use the promocode RADIO for 20% off.