Analyze Texts using IBM Watson and Twilio Functions with Node.js

Time to read: 4 minutes

June 18, 2020
Written by
Lizzie Siegle
Twilion
Dave Nugent
Contributor
Opinions expressed by Twilio contributors are their own
Reviewed by
Diane Phan
Twilion
Phil Nash
Twilion

IBM with Dave

The IBM Watson supercomputer is perhaps best known for beating two quizmasters on Jeopardy, but it can do so much more, such as diagnose cancer, fight crime, and more. Watson also provides APIs and tools to help developers build intelligent models and apps. This post will use deep learning to extract relevant information such as texts, like entities, keywords, categories, sentiment, emotion, and more from a text message with Watson Natural Language Understanding (NLU) and Twilio Functions.

This post was written with IBM developer advocate Dave Nugent for a joint workshop.

updated text gouda baseball ibm sentiment

 

 

 

Prerequisites

To follow along with this post, you need three things:

Now let's get started with IBM Watson!

Make a Watson Natural Language Service

To make our Watson NLU service, go to your IBM Cloud dashboard and click the blue Create Resource button in the top right corner.

Watson NLU service

Next, type "natural language" in the search bar that says Search the catalog and click enter.

IBM cloud
 

You should see quite a few options. Select Natural Language Understanding.

NLU search

The lite plan should automatically be selected. Click the blue create button.

create free tier

Whoop whoop! You've just made your free Watson Natural Language Understanding service!

To get our required credentials, click the Navigation Menu in the top left corner.

navigation menu

Select Resource list from the dropdown menu followed by your Natural Language Understanding resource.

Keep this page with your API key and URL (which are unique to you) open--we'll use those in the next step!

NLU credentials

Call your Watson NLU Service from a Twilio Function

One underrated Twilio product (in my opinion) is Twilio Functions. Functions is a serverless environment for Node.js applications. It allows developers to get up-and-running quickly.

Make your way to the Configure section of your Twilio Functions console. To add your IBM Watson NLU apikey and url copied from the last section as environment variables, click the red plus button near Environment Variables and name the two values as "watson_apikey" and "watson_url". Add your corresponding values accordingly. With our environment variables set, it's time to import the node modules required for this project.

Click the red plus button near Dependencies. Add ibm-watson as a package. Feel free to use a * in the version field to indicate the latest version and then click Save at the bottom!

configure Function

Nice! Now we can access our IBM Watson credentials in any of our Twilio Functions. Let's make one now.

Make a Twilio Function

Above the Configure button on the left, select Manage and make a new Function by clicking the red plus symbol and then select "+ Blank" to make a blank Function. Give your Function a name like "IBM Watson SMS analysis" and a path like "/ibm".

Function name, path

Nice! We have a Twilio Function.

Analyze Texts with Watson Natural Language Understanding

Replace all the code already in your Function with

const NaturalLanguageUnderstandingV1 = require('ibm-watson/natural-language-understanding/v1');
const { IamAuthenticator } = require('ibm-watson/auth'); 
exports.handler = function(context, event, callback) {
    let twiml = new Twilio.twiml.MessagingResponse();
    console.log(event.Body);
  const analyzeParams = {
    'text': event.Body,
    'features': {
      "sentiment": {},
      "categories": {},
      "concepts": {},
      "entities": {},
      "keywords": {}
    }
  };
  const naturalLanguageUnderstanding = new NaturalLanguageUnderstandingV1({
    version: '2020-06-12',
    authenticator: new IamAuthenticator({
      apikey: context.watson_apikey,
    }),
    url: context.watson_url,
  });
  naturalLanguageUnderstanding.analyze(analyzeParams)
    .then(analysisResults => {
        const score = analysisResults.result.sentiment.document.score;
        const label = analysisResults.result.sentiment.document.label;
        let categories = analysisResults.result.categories;
        let keywords = analysisResults.result.keywords;
        categories = categories.filter(item => item.score > 0.8).map(item => item.label).join(', ').replace(/\\|\//g,', ');
        categories = categories.substring(1).split(','); //remove initial comma
        keywords = keywords.reduce((a,v,i) => {return a + (i===0?"":", ") + v.text}, "");
        const msg = "We detected " +label + 
            " sentiments, and identified the categories " +
            categories + " relating to keywords " + keywords;
	    twiml.message(msg);
	    callback(null, twiml);
    })
    .catch(err => {
      callback(null, 'Error: ' + err);
    });
};

That's a lot of code! Let's walk through it.

We import Watson NLU and authentication at the top with

const NaturalLanguageUnderstandingV1 = require('ibm-watson/natural-language-understanding/v1');
const { IamAuthenticator } = require('ibm-watson/auth');

Then in exports.handler we make a Twilio Messaging Response and make the analyzeParams variable that tells Watson what to do. Here, we tell Watson to analyze the inbound SMS message providing us information about sentiment, categories, concepts, entities and keywords. They are initialized as empty objects to return the default options for those categories. You could add parameters if you want to limit the number of responses, for example.

 const twiml = new Twilio.twiml.MessagingResponse();
    const analyzeParams = {
        'text': event.Body,
        'features': {
        "sentiment": {},
        "categories": {}
        }
    };

You could receive less data by changing or removing these properties in features.

Next, we instantiate a natural language understanding API client and authenticate it with our API Key. The Watson library provides an authenticator object IamAuthenticator we can use to do that. This is where we access our environment variables by appending the variable names to context so "watson_apikey" is referenced with context.watson_apikey.

const naturalLanguageUnderstanding = new NaturalLanguageUnderstandingV1({
        version: '2020-06-12',
        authenticator: new IamAuthenticator({
            apikey: context.watson_apikey,
        }),
        url: context.watson_url,
    });

For more information on IAM authentication, check out the npm module page for Watson.

Lastly, we call the Watson NLU API with the parameters we built up. This returns a promise whose results, when resolved, helps build a response message containing a sentiment score, label, and categories.

naturalLanguageUnderstanding.analyze(analyzeParams)
    .then(analysisResults => {
        const score = analysisResults.result.sentiment.document.score;
        const label = analysisResults.result.sentiment.document.label;
        let categories = analysisResults.result.categories;
        let keywords = analysisResults.result.keywords;
        categories = categories.filter(item => item.score > 0.8).map(item => item.label).join(', ').replace(/\\|\//g,', ');
        console.log(`keywords: ${JSON.stringify(keywords)}`);
        keywords = keywords.reduce((a,v,i) => {return a + (i===0?"":", ") + v.text}, "");
        categories = categories.substring(1).split(','); //remove initial comma
        const msg = "We detected " +label + 
            " sentiments, and identified the categories " +
            categories + " relating to keywords " + keywords;
	    twiml.message(msg);
	    callback(null, twiml);
    })
    .catch(err => {
      callback(null, 'Error: ' + err);
    });
};

Tada! You can hit this Function with an HTTP request with a tool like Postman, but let's configure a Twilio phone number to analyze texts using IBM Watson.

Configure your Twilio Phone Number with a Twilio Function

If you don't have a Twilio number yet, go to the Phone Numbers section of your Twilio Console and search for a phone number in your country and region, making sure the SMS checkbox is ticked.

buy a number

In the Messaging section of your purchased number, in the A Message Comes In section, set the dropdown menu to Function instead of Webhook and then on the right select your Function from the larger dropdown menu, as shown below. Hit Save.

messaging configuration phone number

Whip out your phone and text your Twilio number to see IBM Watson analyze your text! Feel free to play around with what Watson analyzes and send different text messages.

updated text gouda baseball ibm sentiment

 

You can view this on Dave's GitHub.

What's Next for Analyzing Texts with Watson

With Watson, you can quickly analyze and infer different tidbits much better from the texts. How is someone feeling? Do they need an animal gif even if they don't outright say so? You can perform sentiment analysis with TensorFlow.js, in a phone call, and more. Let us know online or in the comments what you're building!

Related Posts

Related Resources

A newspaper article

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
An Open book

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.
User group reactions

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.