Solve your Culinary Conundrums with Twilio, OpenAI and Java

Solve your Culinary Conundrums with Twilio, OpenAI and Java

We've all been there - it's been a long day, you're tired and hungry, so you open the fridge and see… stuff. Just a random collection of ingredients. Deciding what to make can be more effort than cooking.

My tiny fridge

In this post I'll show how you can marinate a list of ingredients in OpenAI's Chat and Completions APIs to suggest a more-or-less plausible recipe (it depends on how plausible your ingredients are). So that you don't have to leave your kitchen, add a dash of Twilio's SMS API and serve directly to your cell phone.

You can follow along with this tutorial or grab the completed code from my GitHub repo.

Before we start

The code in this post is in Java. For your development environment you will need:

• a recent version of Java installed. My preferred way to install Java is with SDKMAN. If you're not familiar with it, read about how to use SDKMAN here.
• a Java IDE. I use IntelliJ which has a free version but other IDEs will work fine.
• an application to allow localhost URLs to be used by Twilio. I will use ngrok.

To use the OpenAI API you will need:

• An OpenAI account, sign up here if you don't already have one.

To use Twilio's API you will need:

• A Twilio account. Sign up here.

How it works

We'll buy a phone number from Twilio and configure it with an SMS Webhook URL pointing to the application we write. Ngrok will let us use a localhost URL for this while we're developing. Part of the webhook request from Twilio will be the content of the message, which we will modify to make a prompt for OpenAI. The response from OpenAI will be sent back to our application and passed to the Twilio API to send as an SMS. We'll use the OpenAI Chat API, but the code on GitHub includes an example using the Completion API as well.

Cooking up some code

To handle webhooks we will need to make an app that can respond to HTTP requests, for which I would recommend Spring Boot. Head over to the Spring Initializr to generate a project. It's important to check that Java, Maven and JAR packaging are selected, and that the "Web" dependency is added. You can set the other options as you wish to use Gradle or any other Spring dependencies, but the code in this post will reflect the settings embedded in the link above so if you choose something different you may need to change your build to match.

Download and unzip the project then open it in your IDE.

First add the dependencies that this project needs. In the root directory of the project there is a pom.xml which already has some dependencies from the Spring Boot template in the <dependencies> section. Next to these add the dependencies for OpenAI and Twilio APIs:

[code in context on Github]

At this point you might need to tell your IDE to load Maven changes so that it downloads the dependencies.

At this point there is a single class in the project, in src/main/java/com/example/culinaryconundrum. You won't need to edit that file but it does have a main method in it from which you can run the app.

The webhook handler

In the same package as the existing class, create a new class,call it GptRecipe, and then add these contents:

[full code on GitHub]

This code will not compile just yet, but it's a good starting point. Here's a quick summary:

• @RestController (line 1) tells Spring to scan this class for methods that can handle HTTP requests.
• @PostMapping("/gpt-recipe") (line 14) is a method to handle POST requests to that URL. On the following lines you can see what parameters will be extracted from Twiliio's webhook request.
• String prompt (line 20) adds some extra text to the list of ingredients to help GPT generate a nice output. You can get a lot of variety depending on how you engineer this prompt so do experiment with asking it to rhyme or speak in the style of Nicolas Cage or whatever you feel like.
• chatCompletionRequest (line 22) is a multiline statement that builds a request which is executed by the openAiService on the line below. I've used the settings in the code and left most options as default, which I found gets decent results. Feel free to read up and experiment though - they are documented here. maxTokens dictates the length of the reply (in fact the prompt and the reply are counted) so if long recipes are cut off then you can adjust this.
• @Autowired (line 7) uses Dependency Injection to ask Spring to supply an OpenAI client and a class that will call the Twilio API. Neither of these classes exists yet so it's a good time to create them right now.

The OpenAI client

Create another new class in the same package as the others called OpenAiServiceFactory. Put these contents in:

[full code on GitHub]

By marking this class as a @Component and the method that returns the OpenAiService as a @Bean, Spring will be able to fix one of the three errors in the previous code.

I only made this a separate class so that I could override the default HTTP timeout (of 10 seconds) which I found was sometimes necessary, and reuse this code in a couple of places later on.

The @Value("${openai.api-token}") will cause Spring to look for that value in src/main/resources/application.properties. That file will be empty right now but you will need to get an OpenAI API Key and add it to that file in the format:

The Twilio client

Time for another new class. Same place as before, name the file  TwilioSmsService.java, and add the following content:

[full code on GitHub]

This class pulls in your API Key and Secret from the application.properties file so you will need to add them there. As well as initialising the client, this class exposes the method to send an SMS which was used in the GptRecipe class.

Code complete

That is all the code you'll need for this build - three new classes with about 50 lines of code. You could modify the code right now to test the OpenAI section without using Twilio but I'll jump ahead to configuring Twilio so that you can text the recipe bot.

Twilio account setup

You will need to buy a phone number from Twilio before heading to the configuration page for your new phone number and configuring it to use your app.

Navigate to the Twilio Console and click on the phone number you want to use for this app. Scroll down to the Messaging Configuration section.

For the URL you will need to create your own - this is where ngrok steps in to help by creating public URLs that can be connected to localhost servers. You'll need this because Twilio won't be able to see your laptop on the internet. Download and install ngrok then set up a tunnel to localhost:8080 with:

8080 is Spring Boot's default port. You can start up your app by running the main method in CulinaryConundrumApplication through your IDE. If you configure Spring to use a different port then you'll need to change how you start ngrok accordingly. ngrok's output looks like this:

For the a message comes in section, copy the https forwarding URL and paste that into the Twilio console's URL field along with the gpt-recipe endpoint. Set it to an HTTP POST method and save the phone number configuration as seen below:

Test the Twilio and OpenAI Spring Boot app

Send a text with what you can see in your fridge and you will get a recipe back. My fridge (pictured above) was pretty uninspiring but this sounds nice!

The skull and crossbones on my jar of chilli sauce is telling me that one tablespoon is probably a little too much, but I can adapt. Don't have honey? No problem. Text in again and add "no honey" or whatever other refinement you like to get another recipe.

OpenAI models

In this post I've used the gpt3-3.5-turbo model from their list of available models. This is a decent choice as it is fast and relatively cheap and gets excellent results. But what if you don't want excellent results? For comparison I have another class on GitHub which uses the text-davinci-003 model.

For the most part gpt-3.5 outperforms that model and is cheaper, but If you have particularly adventurous cooking tastes davinci will happily give you a recipe including snails and soil, whereas gpt-3.5 will just tell you that those aren't safe for consumption. It's up to you, but I've included it so you can compare and experiment with the different models yourself (probably don't cook the snail/soil omelette though, please?)

You can download and add that to your project and change the webhook URL that Twilio uses from /gpt-recipe to /davinci-recipe. You will see in that code how to use the Completion API which is slightly different to the Chat API used by gpt-3.. Bear that in mind when designing your prompt - you will probably get better results if you tweak the prompt depending on what model you use.

Wrapping up

This small project has glued together the OpenAI and Twilio APIs to solve your cooking quandary. If it's your first time using either or both of them then congratulations! Where you take it next is up to you. A slight change to the prompt and you could have a book recommender or a pocket dad joke generator. Or make it more complex and build a product recommendation service for work – you can include as much context as you need in the prompt to get some really sophisticated results.

Do bear in mind as always that Large Language Models like GPT can be very plausible and hugely entertaining to play with and use, but all they do is produce words that seem to fit with the prompt. There is no guarantee of factual accuracy (or in this case tastiness) so use your judgement and be kind.

Get in touch with me: mgilliard@twilio.com if you have any questions about this post or anything else about Twilio or Java. I'd love to hear from you and I can't wait to see what you build.

Matthew is a Developer Evangelist at Twilio specialising in Java and other JVM languages. Previously he was a developer working on public cloud infrastructure, and before that a teacher of English as a foreign language.