Twilio’s Customer-Obsessed Innovation Department And Their AI Assistants

Twilio’s Customer-Obsessed Innovation Department And Their AI Assistants

The 16-Year-Old Pizza Box

People at Twilio love to tell the story of the pizza box. Cofounders Jeff Lawson, John Cooke, and John Wolthius, brainstorming Twilio’s early product model after a dinner, scribbled their to-do list on the back of a pizza box.

“We talk about it as the way to really get started and have that entrepreneurial ‘Builder’ spirit,” says Emily Shenfield, one of Twilio’s Principal Developer Educators on Twitch.

The pizza box that launched Twilio is framed in the company’s San Francisco office.

It has been 16 years since the pizza box sketch. Twilio – a contemporary of Uber, Stripe, and Shopify – grew and gained in complexity along with the companies it served. Now, the company has 5,700 employees and a myriad of products, offering a comprehensive customer engagement platform that allows businesses to reach customers on channels like SMS, Email, and WhatsApp while monitoring the resulting data via Segment. But the days of the pizza box still echo throughout the minds of the company’s employees.

Small, goes the startup credo, is agile. The now-larger company is still inspired by the image of three people in a room, sketching a postprandial plan to change an industry. Employees name promotion codes after the pizza box, reference it in speeches and blog posts, and build brands with it in their charter.

The Emerging Technology & Innovation team relies on “pizza box sketches” and tight circles of customer feedback to come up with products for Twilio Alpha.

Emerging Technology & Innovation

Twilio’s newest brand is called Twilio Alpha, and it is run by a new team called Emerging Technology & Innovation (ETI). ETI’s mission is to guard Twilio’s position as an industry leader by brainstorming and deploying new ideas with the agility of a small startup. But according to Braden Becker, the Senior Director of ETI, the driving force of the team isn’t ideas.

“It's about developer and customer obsession,” says Becker. “So, grounding all of the work that we're doing in the needs of our customers and in the developer community where Twilio has really strong roots. We want to make sure that we're looking at our customers' future problems. What are the challenges they're going to encounter with communications, with customer engagement, with customer data in the future?”

Becker is one of sixteen people currently on the team, consisting of engineers, product managers, designers, and business development professionals. Their goal isn’t simply to research the future, but to build concrete prototypes for customers to engage with.

“We want to make sure that our work is getting shipped early to our customers and we're also shipping often,” says Becker.

ETI was founded with the philosophy of “Better Together,” a phrase meant to indicate not only unity with customers, but cooperation across the organization, particularly between Twilio’s Communications and Segment Business Units.

“We really wanted to showcase how we can unlock value for the customers to bring these two things together,” says Dominik Kundel, Senior Manager of Product Management and founding member of the team. “The idea was: we sit outside of these two business units so that we can think about brand new products that are not necessarily coming from only the lens of Segment or only the lens of Communications.”

Currently, ETI seeks to investigate how AI can serve as a go-between for communications tools and the large amounts of customer data stored and sorted by Twilio Segment. Their most recent creation: AI Assistants.

“Autonomous agents are going to fundamentally change how we're going to have customer engagement,” says Kundel. “We believe there's a lot of power in that.”

AI Assistants

The product is exactly what it sounds like: a way for businesses to add friendly digital agents to their customer interfaces. Twilio’s interface is extremely straightforward. A few text boxes allow you to input how your chatbot should interact and what its goals should be, and then you click a button to launch it. I watched a recording of Christopher Brox, engineer, demoing AI Assistants on Twitch. 

“I want to build an AI Assistant where I can say, ‘I’m too busy, talk to my Assistant,’” Brox says. He clicks a few fields, including one that will include Segment data in personalizing the assistant’s responses. Then, he customizes the agent’s personality.

“You are a helpful sales development rep to schedule demos of Twilio AI Assistants,” he types. “Customers will reach out to you to learn about Twilio AI Assistants.”

In another field called Knowledge, he adds context about AI Assistants in the form of the product’s launch website. This site gets scraped and added to the Assistant’s context. Under Tools, he connects the assistant to his calendar API so that it can schedule appointments for him. The whole process takes about ten minutes, including Brox’s explanations.

The result is a simple but effective interaction that schedules events on Brox’s calendar. Later in the demo, Kundel uses the same platform to create an assistant that talks like a pirate and tells dad jokes.

AI Assistants allows you to customize the style of the agent’s diction, for example asking it to act as a “Pirate who likes to tell dad jokes.”

According to Christopher Lintz, ETI’s Software Architect, this interaction is the work of a number of Large Language Models (LLMs) that work with the indicated tools and knowledge to produce the desired interaction.

“We aspire to have a constellation of LLMs that are not just cost efficient for Twilio, but that are answering with the highest quality response for a particular task,” says Lintz.

Under the hood, AI Assistants is based on bleeding edge research put into practice by ETI’s team of architects and engineers. In addition to the LLMs, tools, and knowledge that make up the Assistant’s interactivity, the team has also focused on building a form of infinite memory, where the Assistant is able to recall information from its interactions with the customer and take actions accordingly. The platform optionally uses Segment to store information that occurs in a conversation that may be needed later. For example, an Assistant may store information about a user’s flight and city, and later use that to help a customer locate their baggage in a seamless interaction.

An example of an AI Assistant helping with lost bags from another one of Brox’s demos.

“Everything is still ‘early stages’ and changing,” says Andrew Hitti, the Director of Software Engineering at ETI, “A big challenge for our team is continuing to experiment and understanding new paradigms and hopefully defining those new paradigms.”

“The industry moves that fast,” agrees Lintz, “There's plenty of research. We hope to be part of the research as well, given our unique position and this amazing opportunity of being integrated into one of the most popular communication and customer engagement platforms on planet Earth.”

Transparency in a world of black boxes

AI Assistants includes a simulator for testing interactions, which Brox uses extensively in his demo. Notably, the simulator allows developers to delve deeper into how the assistant “thinks,” showing the logic and sources behind its responses.

The simulator exposes the “thought process” behind the Assistant’s responses.

“You will see when that assistant is reaching out to a tool, and what in the prompt led it to say that this tool should be executed to fetch more information, and which knowledge source it's pulling from when it gives you an answer,” says Hitti. “I love that part about the product and that transparency because it lets you know it's working, it lets you trust that it's working, and then it lets you get more comfortable allowing this to take on more complex workloads. I think that transparency is critical.”

Transparency within the product is as important to the engineering leadership as it is to product and business leaders, who have the challenge of presenting evolving prototypes as a part of a larger brand of polished products.

“We just try to be really clear upfront on the expectations of where we're at,” says Kat McCormick Sweeney, ETI’s Director of Business Development. “And I think it benefits both sides.”

ETI plans to continuously publish their progress and be clear with customers about their products and roadmap.

“It's really important to us to make sure that the work we do gets published, it gets put out there,” says Becker. “We're sharing what we're learning with the other people within Twilio and then in the community, broader community, our customers, the Developer Network, and the industry at large.”

Getting Involved

Kundel describes ETI as a sort of “compass” for Twilio, and the team itself uses customers as their guiding light.

“We're trying to talk to as many customers as possible, and we're trying to understand what matters most to your business when it comes to customer engagement? What are you looking to automate and why, which areas?” says McCormick Sweeney.

The engineering team is particularly excited to see Developers take hold of AI Assistants and begin to build.

“Get on the waitlist and get your hands on it,” says Hitti. “Don't sleep on challenging your assumptions with how powerful this may or may not be… A lot of people have agendas around how they evaluate this technology. But I would tell everyone to have your firsthand experience using these and let that form what your opinion is.”

Whether your idea is small or large, and whether it fits on the back of a pizza box or not, the Emerging Technology and Innovation team is eager to see developers build. You can get on the waitlist for AI Assistants here, and read about how Twilio Alpha fits into Twilio’s larger AI ecosystem here.