Generate Art with DALL·E 2 and Twilio Serverless via SMS

Time to read: 4 minutes

January 30, 2023

Written by

Lizzie Siegle

Twilion

Reviewed by

Niels Swimberghe

Twilion

pikachu drinking boba header image that also has twilio messaging and openAI logos and says

AI-generated art is having a moment! In November 2022, OpenAI released their DALL·E API in public beta. Read on to learn how to build a serverless SMS chatbot using OpenAI's DALL·E 2, Twilio Programmable Messaging, the Twilio Serverless Toolkit, and Node.js.

sms gif example with pikachu holding boba and a tennis racket, a panda with mickey ears cycling in front of the golden gate bridge

You can test this chatbot out yourself by texting a prompt (like in the GIF above) to +12407633723.

Do you prefer learning via video more? Check out this TikTok summarizing this tutorial in under three minutes!

Dall·E

DALL·E is a deep learning model developed by OpenAI to generate digital images from natural language descriptions, called "prompts". Here, I provide a prompt of "impressionist version of pikachu playing tennis".

art generated in browser of pikachu playing tennis

You can test out DALL·E in the browser here.

The DALL·E model was updated to DALL·E 2 in April 2022 with the ability to create more realistic images, added support for more prompt styles, and in-painting. DALL·E 2 uses a diffusion model with CLIP image embeddings.

To use DALL·E in a Node.js application, you must use OpenAI's DALL·E API, which automatically uses the latest version of DALL·E 2.

Prerequisites

A Twilio account - sign up for a free Twilio Account here
A Twilio phone number with SMS capabilities - learn how to buy a Twilio phone number here
OpenAI Account – make an OpenAI Account here
Node.js installed - download Node.js here

Get Started with OpenAI

After making an OpenAI account, you'll need an API Key. You can get an OpenAI API Key here by clicking on + Create new secret key.

Save that API key for later to use the OpenAI client library in your Twilio Function.

Get Started with the Twilio Serverless Toolkit

The Serverless Toolkit is CLI tooling that helps you develop Twilio Functions locally and deploy them to Twilio Functions & Assets. The best way to work with the Serverless Toolkit is through the Twilio CLI. If you don't have the Twilio CLI installed yet, run the following commands on the command line to install it and the Serverless Toolkit:

npm install twilio-cli -g
twilio login
twilio plugins:install @twilio-labs/plugin-serverless

Afterward, create your new project and install our lone package openai:

twilio serverless:init dalle-img-generation-sms --template=blank
cd dalle-img-generation-sms
npm install -s openai

Set an Environment Variable with Twilio Functions and Assets

Open up your .env file for your Functions project in your root directory and add the following line:

OPENAI_API_KEY=YOUR-OPENAI-API-KEY

Replace YOUR-OPENAI-API-KEY with the OpenAI API Key you took note of earlier. Now, you can access this API Key if you'd like to do so in your code with context.OPENAI_API_KEY.

Make a Twilio Function with JavaScript

Make a new file in the /functions directory called dalle.js containing the following code:

const { Configuration, OpenAIApi } = require("openai");

exports.handler = async function (context, event, callback) {
  const configuration = new Configuration({
    apiKey: context.OPENAI_API_KEY
  });
  const openai = new OpenAIApi(configuration);
  const inbMsg = event.Body.trim();
  const imgData = await openai.createImage({
    prompt: inbMsg,
    n: 1,
    size:"1024x1024"
  });
  const dataURL = imgData.data.data[0].url;
  const twiml = new Twilio.twiml.MessagingResponse();
  const message = twiml.message(); 
  message.body(`here's an image pertaining to ${inbMsg}!`);
  message.media(dataURL);
  return callback(null, twiml);
};

This code makes an async function that will handle incoming messages. The function

Creates a Configuration object (from the openai package) passing in an object containing the apiKey property.
Creates an OpenAIApi object passing in the configuration.
Retrieves the body of the incoming message and stores it in the inbMsg variable.
Calls the openai.createImage function to use one of their language models to generate an image based on the inbMsg prompt.
Creates a Twilio Messaging Response object which responds with some text and media pointing to the image generated by DALL·E 2. This twiml is passed into the callback function which will write it to the HTTP response.

In order to specify your image generation, you need to pass in a configuration object containing three properties: prompt, n, and size. prompt is the only required property, representing a text description of the desired generated image. n is an optional property that defaults to one. It's the number of images to generate and must be between one and ten. Lastly, the optional size property is the size of the generated images and must be either 256x256, 512x512, or 1024x1024.

For more details about creating images and other ways to work with images using Dall·E, check out OpenAI's documentation here.

You can view the complete source code on GitHub here.

Configure the Function with a Twilio Phone Number

To deploy your app to Twilio, run twilio serverless:deploy from the chatgpt-sms root directory. You should see the URL of your Function at the bottom of your terminal:

Using the Twilio CLI, you can update the phone number using the Phone Number SID of your Twilio phone number. You can see it in the Twilio Console under Properties and it begins with "PN".

twilio phone-numbers:update {PHONE_NUMBER_SID|E164} \
  --sms-url {your Function URL ending in /dalle}

If you don't wish to configure your Twilio phone number using the Twilio CLI, you can grab the Function URL corresponding to your app (the one that ends with /dalle) and configure a Twilio phone number with it as shown below: select the Twilio number you just purchased in your Twilio Phone Numbers console and scroll down to the Messaging section. Paste the link in the text field for A MESSAGE COMES IN webhook making sure that it's set to HTTP POST. When you click Save, it should look like this!

Twilio Messaging configure phone number with webhook URL

The Service is the Serverless project name, environment provides no other options, and Function Path is the file name. Now take out your phone and text a question or prompt to your Twilio number.

text saying "pikachu drinking boba" and the reply is a picture generated by dall-e 2 of pikachu drinking boba

What's Next for Twilio Serverless and DALL·E?

The development possibilities offered by DALL·E and Twilio are endless! If you're a .NET developer, you can generate images with DALL·E using .NET here. You can edit images, merge images, and more. Let me know what you're working on with OpenAI and DALL·E–I can't wait to see what you build.