Building a Voice-Based Pizza Ordering Service with Twilio, OpenAI, and Google Maps

January 21, 2025
Written by
Eluda Laaroussi
Contributor
Opinions expressed by Twilio contributors are their own
Reviewed by

Building a Voice-Based Dominos Pizza Ordering Service with Twilio, OpenAI, and Google Maps

Introduction

In this project, we'll develop a phone service enabling users to place an order for Dominos pizza by calling us, using the menu of their nearest store to select items, and processing the payment with their bank card. Additionally, we'll utilize a Large Language Model (LLM) to interpret user requests into Dominos' official item codes.

Architectural diagram of Twilio x Dominos.

Prerequisites

- An active Twilio account ( sign up for a free account if you don't have one).

- An OpenAI platform account.

- A Twilio phone number with phone call support.

- A Google Cloud account with a project and billing enabled for address processing.

Gathering Order Information

We'll utilize Twilio Studio to set up the user flow for when calls come in. This graphical tool provides a simple, intuitive way to orchestrate the interactions without dealing with TwiML codedirectly.

Begin by creating a new flow in Twilio Studio and clicking Create new flow. Then, name it "Dominos Flow" and select the Start from scratch option. After clicking Next, you’ll be brought to your newly created Studio Flow.

After you're done following this tutorial, you can confirm your final Flow is correct by importing the expected Flow JSON to your Twilio console.
Flow containing entrypoint, split, get_name, get_address and get_food.

Step 1: Create the Entry Point

Start with a Gather Input on Call block named entrypoint. This block greets callers and prompts them to begin the ordering process. Configure it to say: Welcome to the Domino's Pizza ordering system. Press 1 to place an order.

Limit the input to one digit (number_of_digits: 1), set a timeout of 5 seconds, and ensure it stops gathering input after the user responds. On keypress, connect this block to the next block, split.

Step 2: Verify User Input

Add a Split Based On block named split. This block checks if the input from entrypoint equals "1" using the variable {{widgets.entrypoint.Digits}}. If it matches, transition to the next block, get_name.

Step 3: Collect the User’s Name

Create another Gather Input on Call block named get_name. This block prompts the user with: Your name, please?

Configure it to stop gathering once the input is received. On speech, connect this block to the get_address block.

Step 4: Collect the Delivery Address

Add a Gather Input on Call block named get_address to collect the user's delivery details. Set the message to: What is your delivery address, including city, state, and ZIP code?Use the numbers_and_commands speech model for accurate address input. Connect this block to get_food on speech.

Step 5: Collect the Food Order

Create the final Gather Input on Call block named get_food. This block asks the caller: What do you want for today?

Ensure it gathers the input and transitions to the next step in your flow, such as get_price.

Final Connections

Connect the blocks in this order:

  • entrypoint → split (on keypress)
  • split → get_name (on match)
  • get_name → get_address (on speech)
  • get_address → get_food (on speech)

Calculating Order Price

Creating a Twilio Serverless Function

Before proceeding to the payment steps, validate the order and compute its price using the Dominos API.

Note that the Dominos API is not officially available to third parties, is rate-limited, and accepts requests without authentication. Exercise caution to avoid exceeding the request limit.

We'll create a custom JavaScript function within Twilio Functions enabling us to leverage NPM packages for our custom logic. For now, set up a service with a user-friendly URL name, like "dominos", and create a corresponding function named "/price". This function will be accessible through a URL format like https://dominos-[random_numbers].twil.io/price.

This walkthrough splits the code blocks in sections to explain what each part does. If you're ever lost during this guide, you can always refer to the final code for the /price function

Validating the Delivery Address

The Dominos API requires precise address entries. However, users typically won't provide exact matches, and speech recognition can be imperfect, leading to potential errors.

Google Maps Autocomplete API.

To mitigate this, employ the Google Maps Places API to convert inexact addresses into the required format.

In your Google Cloud project console, locate "Places API" , enable it. Once enabled, you should be presented with an API key; ensure you copy this for the next step.

Head over to Twilio Functions Console, navigate to Environment Variables, and define a variable GOOGLE_MAPS_PLACES_API_KEY, pasting in the copied key.

Next, in the Dependencies section, add the Google Maps package by copying the following into the Module box: @googlemaps/google-maps-services-js. Leave the Version box blank and click Add to add the dependency..

Now, in your /price function, replace everything inside the exports.handler function with the following code to handle address conversion. Comments are provided for clarity.

const { Client } = require("@googlemaps/google-maps-services-js");
const placesClient = new Client({});
async function validateAndFindNearestAddress(addr) {  
	/* 
	Initialize the request parameters for Google Maps Autocomplete API.
	This will take the user's input and attempt to find valid address matches.
	*/
	const autocompleteRequest = {  
		params: {  
			input: addr,
			types: "address",
			key: process.env.GOOGLE_MAPS_PLACES_API_KEY,
		},  
	};
	try {  
		/* 
		Send the autocomplete request to Google Maps API.
		If no predictions are returned, throw an error to indicate the address was invalid.
		*/
		const autocompleteResponse = await placesClient.placeAutocomplete(autocompleteRequest);
		if (!autocompleteResponse?.data?.predictions?.length) {  
			throw new Error("Could not find a valid or approximate match for the address.");  
		}
		/*
		Extract the place ID from the first prediction. 
		This ID is used to fetch detailed information about the address.
		*/
		const placeId = autocompleteResponse.data.predictions[0].place_id;
		/*
		Use the place ID to retrieve detailed address components, 
		such as street number, route, city, region, and postal code.
		*/
		const placeDetails = await placesClient.placeDetails({  
			params: {  
				place_id: placeId,
				key: process.env.GOOGLE_MAPS_PLACES_API_KEY,
				fields: ["address_components"],
			},  
		});  
		/* 
		Extract specific components from the address details. 
		If a component is missing, default values are provided to avoid errors.
		*/
		const components = placeDetails.data.result.address_components;
		const streetNumber = components.find((comp) => comp.types.includes("street_number"))?.long_name
							 || "0000";
		const route = components.find((comp) => comp.types.includes("route"))?.long_name || "";  
		const city = components.find((comp) => comp.types.includes("locality"))?.long_name || "";  
		const region = components.find((comp) => 
						comp.types.includes("administrative_area_level_1"))?.short_name || "";  
		const postalCode = components.find((comp) => comp.types.includes("postal_code"))?.long_name || "";
		/* 
		Return the address in a standardized format.
		This ensures the address meets the requirements of downstream APIs.
		*/
		return {  
			street: `${streetNumber} ${route}`.trim(),  
			city,  
			region,  
			postalCode,  
		};  
	} catch (error) {  
		/* 
		Handle errors gracefully by logging them and returning null. 
		This prevents the application from crashing due to invalid inputs.
		*/
		console.error("Error finding nearest valid address:", error);  
		return null;  
	}  
}

Extracting the Order Metadata

Typically, a user will request something like "I want a medium-sized chicken BBQ pizza", which needs translating into a Dominos recognizable code like "CHICKEN10".

For this translation, use an LLM like GPT 4o Mini or any preferred LLM (e.g., Gemini, LLama, Claude).

Go to your OpenAI Platform page, obtain your API key, and add it to a new environment variable: OPENAI_API_KEY. Then, install the OpenAImodule by navigating to the Dependencies section and entering openai as Module and clicking Add.

Add the following code snippet below the existing code in module.exports. This code will interpret user input into a compatible Dominos code:

const OpenAI = require("openai");  
const client = new OpenAI({  
	apiKey: process.env.OPENAI_API_KEY,
});
async function getOrderDetails({ menu, prompt }) {  
	/* 
	Define a prompt for the AI model that includes the store's menu and the user's request.
	The goal is to extract a valid item code based on the user's description.
	*/
	const chatCompletion = await client.chat.completions.create({  
		messages: [{  
			role: "user",  
			content: `
Menu for this store:  
${menu}
User wants: ${prompt}
Output the most qualified CODE for what the user wants.  
Must be from the menu. DO NOT CHOOSE ITEMS THAT REQUIRE OPTIONS.  
OUTPUT NOTHING BUT THE CODE. NO QUOTES. ALWAYS OUTPUT SOMETHING,  
NEVER RETURN EMPTY.`, 
		}],  
		model: "gpt-4o-mini",
	});
	/* 
	Check the response from the AI model. If the response is empty, return null.
	Otherwise, extract the item code from the response content.
	*/
	if (!chatCompletion.choices[0].message.content) return null;  
	return {  
		code: chatCompletion.choices[0].message.content,  
	};  
}

The menu input is a string representation capturing menu items and their attributes, which is retrieved from the Dominos API.

Getting the Nearest Store

Upon acquiring a formatted delivery address, query the Dominos API to yield a list of proximate stores. Implement a simple algorithm to determine the closest outlet.

Then, install the Dominos module by navigating to the Dependencies section and entering dominos as Module and clicking Add. Next, add this snippet within exports.handler:

const dominos = await import('dominos');
Object.assign(global, dominos);

Note: Direct requires won't work due to CommonJS module limitations.

To change the dominos country settings, use:

const { canada, useInternational } = require("dominos/utils/urls.js");
useInternational(canada);

Here's how to calculate the nearest store:

function getClosestStore(stores) {  
	/* 
	Initialize variables to track the closest store.
	Set a high initial distance to ensure the first valid store is selected.
	*/
	let storeID = 0;  
	let distance = 100; 
	/* 
	Iterate through the list of stores to find one that is:
	1. Capable of processing online orders.
	2. Open for delivery.
	3. Closest to the user.
	*/
	for (const store of stores) {  
		if (store.IsOnlineCapable && 
			store.IsDeliveryStore && 
			store.IsOpen && 
			store.ServiceIsOpen.Delivery && 
			store.MinDistance < distance
		) {  
			distance = store.MinDistance;  
			storeID = store.StoreID;  
		}  
	}
	/* 
	Return the closest store's ID.
	If no stores match the criteria, default to the first store in the list.
	*/
	return storeID || stores[0].StoreID;  
}

Fetching the Order Price

These utilities allow creating, validating, and pricing a Dominos order.

For a comprehensive process that constructs an order, include this code in the exports.handler function:

async function createOrder({ address, fullName, phone, prompt }) {
    // Validate and format the user's address
    const formattedAddress = new Address(await validateAndFindNearestAddress(address));
    // Retrieve nearby Dominos stores
    const nearbyStores = await new NearbyStores(formattedAddress);
    // If no stores are found, return null to indicate failure
    if (!nearbyStores.stores.length) return null;
    // Create a customer profile using the provided details
    const customer = new Customer({
        address: formattedAddress,
        firstName: fullName.split(" ")[0],
        lastName: fullName.split(" ")[1] || "",
        phone,
        email: "dummy@email.com",
    });
    // Determine the closest store and initialize an order
    const storeID = getClosestStore(nearbyStores.stores);
    const order = new Order(customer);

This leverages the functions discussed to revalidate addresses and ascertain the nearest store ID, then proceeds to retrieve the store's menu.

const menu = await new Menu(storeID);

Next, translate it for our LLM, only passing essential fields:

Avoid uploading the entire menu to the LLM as the excessive data and tokens surpass allowed limits.
/* 
Prepare the menu in a compact format for the AI model.
Each item is represented by its name and code to minimize unnecessary details.
*/
const itemDetails = await getOrderDetails({  
	menu: JSON.stringify(Array.from(
		Object.values(menu.menu.variants).map(item => ({
			name: item.name,
			code: item.code,
		}))
	)),
	prompt, // The user's input describing their order.
});

Finally, utilize this data to generate and validate a Dominos item object:

/* 
If the AI model fails to provide a valid item code, stop further processing.
*/
if (!itemDetails) return null;
/* 
Create a new item object using the returned item code and add it to the order.
This step links the user's request to a specific item in the store's menu.
*/
const pizza = new Item(itemDetails);  
order.storeID = storeID;  
order.addItem(pizza);
try {  
	/* 
	Validate the structure of the order.
	Calculate the total price based on the item's details and store pricing.
	*/
	await order.validate();  
	await order.price();  
} catch {  
	/* 
	If validation or pricing fails, return null.
	This ensures invalid orders are not processed further.
	*/
	return null;  
}
return order;
}

Utilize this function with parameters from the handler, utilizing stored event data:

const order = await createOrder({  
	address: event.address,  
	fullName: event.name,  
	phone: event.phone,  
	prompt: event.prompt,  
});
/* 
If order creation fails, set a 404 status code and return an error response.
*/
if (!order) {  
	response.statusCode = 404;  
	return callback(null, response);  
}
/* 
If order creation succeeds, prepare a response containing all relevant order details.
This includes the total cost, estimated wait time, and delivery address.
*/
response.statusCode = 200;  
response.body = {  
	currency: order.formatted.Currency,  
	estWaitMinutes: order.formatted.EstimatedWaitMinutes,  
	priceAmount: order.formatted.AmountsBreakdown.customer,  
	orderID: order.products[0].code,
	storeID: order.storeID,  
	address: {  
		street: order.address.street,  
		city: order.address.city,  
		region: order.address.region,  
		postalCode: order.address.postalCode  
	}  
};
return callback(null, response);

Return the currency, price, order ID, store ID, and address. This prevents redundancy in recalculating this information, thus improving cost efficiency. Additionally, return the estimated wait time, which can be added as user feedback within your flow as an exercise.

Trying it Out

Save the function with Ctrl/CMD + S and Deploy All. Return to Twilio Studio.

get_price block.

Incorporating the function into your flow is straightforward. Add a Run Function block, selecting your dominos service and the /price function. Name this block get_price. Then, connect the get_food block to the get_price block using the User Said Something transition.

If you encounter issues seeing the function, try refreshing the Studio page. You might also want to visit the Function Logs page to see potential errors.

Within the Function Parameters section of the get_price block, inject the details to the function: name, address, phone, and order, through parameters with this syntax:

prompt: {{widgets.get_food.SpeechResult}}  
name: {{widgets.get_name.SpeechResult}}  
address: {{widgets.get_address.SpeechResult}}  
phone: {{trigger.call.From}}

Utilize a Say/Play block to notify the caller about their order cost:

You will be charged {{widgets.get_price.parsed.body.priceAmount}} {{widgets.get_price.parsed.body.currency}}.

Connect the block to the get_price block using the Success transition.

Say/Play block connected to get_price.
Generic Pay Connector architecture diagram.

To manage payments within our app, implement PCI compliance for your Twilio account. Access Twilio's Voice Settings, click Enable PCI Mode, agree to Terms of Service, and save changes.

In Voice > Manage > Pay Connectors, install the Generic Pay Connector. Though specific connectors exist (e.g., for Stripe), the Generic Pay method enables direct transaction handling with Dominos.

Note that the Pay Connector facilitates DTMF keypad input for user financial details.

You are advised to use test cards such as 5555555555554444 for MasterCard, with any combination for expiry date, CVV, and Zip Code. A LIVE deployment presents advanced boundaries outside this walkthrough's scope, focusing on PCI security compliance and payment detail tokenization.

Creating a New Twilio Service Route

Making /place-order public.

Establish a new function within the Twilio Dominos service titled /place-order. This route must be Public to be accessed from our Flow. Therefore, in a production setting, you must ensure strict security practices to ensure that it can only be called from Twilio's webhooks, but that's outside the scope of this guide.

This walkthrough splits the code blocks in sections to explain what each part does. If you're ever lost during this guide, you can always refer to the final code for the /place-order function.

Since building for a production level setting is beyond this guide, a mock function will simulate token generation:

function generateTokenId() {  
	return `tok_${[...Array(32)].map(() => Math.floor(Math.random() * 16).toString(16)).join('')}`;  
}

Implement order creation with subtle variations for handling segmented data components. Include this inside exports.handler:

const dominos = await import('dominos');  
Object.assign(global, dominos);
/*
This function creates a new Dominos order based on the provided user details and order code.
It initializes a customer object, assigns an address, and adds the desired item to the order.
The order is validated and priced before returning.
*/
async function createOrder({ street, city, region, postalCode, fullName, phoneNumber, orderCode, storeID }) {  
	const customer = new Customer({  
		address: new Address({  
			street,
			city,
			region,
			postalCode,
		}),  
		firstName: fullName.split(" ")[0], // Extract the first name
		lastName: fullName.split(" ")[1] || "", // Extract the last name, default to empty if not provided
		phone: phoneNumber,
		email: "dummy@email.com", // Placeholder email for demo purposes
	});
	const order = new Order(customer);  
	/*
	Add the desired item to the order using the provided item code. 
	Store ID is assigned to ensure the order is linked to the correct location.
	*/
	const pizza = new Item({ code: orderCode });  
	order.storeID = storeID;  
	order.addItem(pizza);  
	try {  
		await order.validate(); // Check the order structure and contents
		await order.price(); // Calculate the total price based on the menu and item details
	} catch {  
		return null; // If validation or pricing fails, return null to indicate the error
	}
	return order; // Return the completed order object
}

Now, create an order object utilizing event parameters (we'll later ensure setting up your flow to dispatch data into this function):

/*
Create a new order using parameters received from the event.
Each parameter corresponds to user-provided information, such as address, name, and order details.
*/
const order = await createOrder({  
	street: event.parameters.street,
	city: event.parameters.city,
	region: event.parameters.region,
	postalCode: event.parameters.postal_code,
	fullName: event.parameters.fullname,
	phoneNumber: event.parameters.phonenumber,
	orderCode: event.parameters.order,
	storeID: event.parameters.storeid,
});
Pass in address fields separately, as Pay connectors are limited to transmitting string data only, disallowing JSON or other complex data types.

Returning a <Pay> Response

Twilio forwards an HTTP POST request to your function, formatted like:

{
	"transaction_id": "transaction_id",
	"cardnumber": "4111111111111111",
	"cvv": "123",
	"expiry_month": "08",
	"expiry_year": "27",
	"description": "pizza",
	"amount": "10.0",
	"currency_code": "USD",
	"postal_code": "94111",
	"method": "charge",
	"parameters": {
		"customer_defined_key_1": "customer_defined_value_1",
		"customer_defined_key_2": "customer_defined_value_2"
	}
}

If it's a tokenize request, respond with:

{
	"token_id": "tok_abcdef0123456789abcdef0123456789",
	"error_code": null,
	"error_message": null
}
The tokenize procedure means securing a token from the payment processor based on provided user payment details, effectively facilitating future payments without repeated data inputs. This practice, however, requires an active gateway and is unimplemented here. See Twilio's guide.

For charge operations, we will return an object with the following schema: charge_id, parameters, error_code and error_message:

{
	"charge_id": "ch_abcdef0123456789abcdef0123456789",
	"parameters": {
		"gateway_key_returned_1": "gateway_value_returned_1",
		"gateway_key_returned_2": "gateway_value_returned_2",
		"gateway_key_returned_3": "gateway_value_returned_3"
	},
	"error_code": null,
	"error_message": null
}

We can use the Twilio Response class to create such objects. Add this to your code:

/*
This response is part of Twilio's payment workflow. 
It handles both success and failure scenarios for order creation.
*/
const response = new Twilio.Response();
if (!order) {  
	response.setStatusCode(404); // Set status to indicate the order creation failed
	response.setBody({ error: "Order creation failed" }); // Provide an error message
	return callback(null, response); // Return the response and stop further execution
}

Construct a payment card through request data:

/*
Create a payment card object using the user's payment details.
This includes the card number, expiry date, CVV, and postal code.
*/
const card = new Payment({  
	amount: order.amountsBreakdown.customer, // The total cost for the user
	number: event.cardnumber, // The credit card number
	expiration: `${event.expiry_month.padStart(2, "0")}/${event.expiry_year}`, // Expiry in MM/YYYY format
	securityCode: event.cvv, // CVV (security code)
	postalCode: event.postal_code, // User's postal code for verification
	tipAmount: 0, // Optional: Tip amount, set to 0 here
});
/*
Add the payment card to the order's payments list.
This links the payment method to the current order.
*/
order.payments.push(card);

Deliver relevant outputs:

/*
Process the payment request and deliver the appropriate response.
This includes simulating delays for testing and handling both charge and tokenize methods.
*/
try {  
	await new Promise(res => setTimeout(res, 1500)); // Simulate a processing delay
	if (event.method === "charge") {  
		// Respond with a successful charge result
		response.setStatusCode(200);
		response.appendHeader('Content-Type', 'application/json');  
		response.setBody({  
			charge_id: "ch_abcdef0123456789abcdef0123456789", // Mock charge ID for testing
			parameters: null,  
			error_code: null,  
			error_message: null,
		});  
	} else if (event.method === "tokenize") {  
		// Respond with a successful tokenization result
		const tokenId = generateTokenId();  
		response.setStatusCode(200);
		response.appendHeader('Content-Type', 'application/json');  
		response.setBody({  
			token_id: tokenId,
			error_code: null,  
			error_message: null,
		});  
	} else {  
		// Handle invalid payment methods
		response.setStatusCode(400);  
		response.setBody({ error: "Invalid method" });  
	}
	return callback(null, response);
} catch (err) {  
	/*
	If an error occurs during payment processing, return a failure response.
	This ensures the user is notified of the issue.
	*/
	response.setStatusCode(403);  
	response.appendHeader('Content-Type', 'application/json');  
	response.setBody({  
		error_code: "PAYMENT_FAILED",  
		error_message: "Payment processing failed."
	});
	return callback(null, response);
}

Note: The order placement process is omitted, given we can't fully validate the payment using test card info alone and will result in a failed order due to incorrect data.

For a real-world deployment, implement comprehensive security measures and practice stringent handling of sensitive card details.

Save the function and click the Deploy All button to deploy your new Function.

Adding a Pay Connector

Context menu in a code editor with options to copy URL, edit, delete, and rename a file function.

Copy your new /place-order URL by clicking on the three dots next to your Function name beneath the Functions section and then clicking Copy URL..

Screenshot of Twilio test connector mode settings showing endpoint URL and instructions.

Head to the installed modules page and find your connector here. Paste the endpoint into the ENDPOINT URL field on the Generic Pay connector page.

co

Return to Twilio Studio, refreshing the instance to surface the Capture Payments block. Integrate it into the flow, ensuring that the " Pay Connector" property is set to "Default" or matches the pay connector's " UNIQUE NAME," and set the charged amount using:

{{widgets.get_price.parsed.body.priceAmount}}

Approve card types like VISA and MASTERCARD (you may include additional preferences) and select YES for Request Postal Code?.

Inject the following parameters to correspond with the connector configuration:

order: {{widgets.get_price.parsed.body.orderID}}  
phonenumber: {{trigger.call.From}}  
storeid: {{widgets.get_price.parsed.body.storeID}}  
fullname: {{widgets.get_name.SpeechResult}}  
street: {{widgets.get_price.parsed.body.address.street}}  
city: {{widgets.get_price.parsed.body.address.city}}  
region: {{widgets.get_price.parsed.body.address.region}}

Integrate Say blocks for both positive (success) and negative (failure) scenarios. Deploy the changes by hitting Publish.

When you test your application out, make sure that the billing ZIP code you enter matches the delivery postal code, ohterwise the order will be invalid. As an exercise, you could separate the billing ZIP code and the delivery postal code so they don't have to match.

Conclusion

Here's a demo of a phone call with the completed service: https://vimeo.com/1030106262?share=copy.

Screenshot of a team collaboration app showing a voice call in progress with chat messages and contacts listed.

By following this walkthrough, you've established a comprehensive voice-based service for Dominos, bolstered by Twilio and interfaced with both AI-powered natural language processing and Dominos' own systems. This project showcases integrating robust APIs and cutting-edge AI to make interactions feel natural and intuitive, enhancing user satisfaction and achieving seamless experiences across communication channels.

Next Steps

  • Collect and handle tips.
  • Let users choose preferred stores.
  • Implement "Press 2" to track orders.
  • Use GPT to dynamically describe menu items.
  • Utilize parsed wait times to inform users in natural units of time.
  • Extend the system to manage multiple items like drinks and sides.
  • Explicitly inform users of the store processing their order and seek confirmation.
  • Handle errors such as incorrect orders, out-of-stock items, or unavailable nearby stores.
  • Use GPT to interpret diverse responses, such as name and address, because users might say "my name is X" rather than just giving a name.

Enjoy your streamlined pizza ordering experience!