Asynchronous JavaScript: Using Promises With REST APIs in Node.js

Time to read: 12 minutes

July 30, 2020

Written by

Maciej Treder

Contributor

Reviewed by

AJ Saulsberry

Contributor

Asynchronous JavaScript: Using Promises With REST APIs in Node.js

In JavaScript, like many programming languages, asynchronous processing can be used to handle tasks that take a long time to run or are potentially unresponsive. The JavaScript Promise object provides a way of monitoring their state and serves as a placeholder and container for the data they’ll eventually return — or not.

Often it doesn’t matter when a Promise returns data and the members of a collection of Promises can be resolved independently without regard to timing. A function can perform asynchronous tasks that don’t depend on the data from, or the success of, other asynchronous tasks.

But sometimes business rules or program design require that asynchronous actions are dependent on other asynchronous operations. For example:

Establishing a remote database connection must be preceded by loading configuration information from the file system.
Performing a REST API call to a weather API must be preceded by obtaining the user’s location from a user’s mobile device.
Confirming shipment of goods ordered online must be preceded by a successful payment.

Using the Promise object .then method is often enough for two or three dependent actions. But as your action chain gets longer your code starts to grow into the pyramid of doom. A large stack of nested calls quickly becomes difficult to read and debug, entombing a developer’s time.

Fortunately, the introduction of the async and await keywords in the ECMAScript 2017 specification provided JavaScript with a major improvement handling dependent asynchronous tasks. This post will demonstrate how you can utilize async and await to build an action chain that’s more flexible and easier to read and debug.

Understanding the async and await keywords

The async and await keywords are a form of syntactic sugar for JavaScript Promises, making sequences of asynchronous tasks wrapped in Promises easier to write, read, and debug. They’re a perfect fit if you want to hold your program execution until some asynchronous action finishes.

The async keyword can be used only in a function declaration. It tells the JavaScript runtime environment (V8, Node.js, or Deno) that it should wrap a function body in a Promise object. The Promise will be returned by the function instead of the return statement value. The return value will be used in the resolution of the Promise when that happens asynchronously.

The await keyword can be used only with the Promise object and within a function declared as async. The await operator informs the JavaScript runtime environment that it should hold up program execution until the associated Promise is settled: fulfilled by returning data or rejected. When an await expression is resolved it returns a value provided through the resolution of the Promise it is awaiting. When the Promise is rejected it throws the rejected value as an exception.

The caller of an await expression can continue processing, using the pending state of the Promise as a placeholder until the Promise is settled. If the Promise is fulfilled it’s converted to a resolved Promise.

This means you can write a function that uses a number of await expressions to perform asynchronous tasks with the confidence that the resolved Promises and their data will be used in the order they’re written. If a Promise is rejected the rejected value can be processed by the function’s error handling block.

This post will show you a practical example of using these keywords in a common programming scenario: processing the results of REST API calls that are dependent on the successful completion of previous calls.

Prerequisites

You’ll need the following tools to accomplish the task in this tutorial:

Node.js and npm (The Node.js installation will also install npm.)
Git
Visual Studio Code (or another IDE or code editor)

Twilio account – although it’s not required for this tutorial, readers of this post can receive an additional $10 credit on their Twilio accounts when they sign up with this link.

To get the most out of this post you should also have a basic working knowledge of JavaScript. An understanding of the JavaScript callback loop, event queue, and Promises will be very helpful. See the other posts in this series listed in the Additional resources section if you need to get sharp on those concepts.

There is a companion repository containing the complete source code for the project available on GitHub.

Understanding the case study

The case study project for this post is a simple Node.js program to determine the “best” movie by Quentin Tarantino, based on review scores. To accomplish the task the code will retrieve data from REST API mockups on GitHub.io.

You can emulate the process of retrieving and manipulating the sample data by performing GET calls with your web browser:

Navigate to https://maciejtreder.github.io/asynchronous-javascript/directors/ to see the list of JSON objects representing the directors. Quentin Tarantino has id 2.
Navigate to https://maciejtreder.github.io/asynchronous-javascript/directors/2/movies/ to get the list of Tarantino’s movies.
See the reviews for Inglourious Basterds by navigating to https://maciejtreder.github.io/asynchronous-javascript/movies/5/reviews.
Add up all the scores and divide by the number of scores to get the average review score.

This case study is also used in other posts in this series on asynchronous JavaScript. See the Additional resources section at the bottom of this post for a complete list.

What you’ll learn

In this tutorial you’ll learn how to use the await keyword to wait synchronously for the results of asynchronous actions, like making calls to REST APIs. You’ll see how to pair the await keyword in the body of functions with the async keyword in the function declaration.

You’ll also see how the await keyword can be used in a for…of loop using the for await…of syntax.

Addressing time delays associated with asynchronous actions is an important consideration for user interface design. In this command-line application you’ll see how you can use the ora library to provide the user with an indication the program is working while it’s waiting for an asynchronous action to be completed.

You will also see how to handle Promises rejections in try…catch blocks within functions that use the async…await mechanism.

Setting up the project

You can use the code in the companion repository to set up the Node.js project for this tutorial or you can start from scratch.

The companion repository which includes the code used for the previous posts in the series on asynchronous JavaScript.
If you clone the companion repository you’ll be able to easily compare the asynchronous techniques presented in this post to other approaches, including RxJS Observables, Promises, JavaScript callbacks.
If you start from scratch the project you build will contain only the code from this tutorial, simplifying the project structure.

To clone the companion repository, execute the following command-line instructions in the directory where you would like to create the project root directory:

git clone https://github.com/maciejtreder/asynchronous-javascript.git
cd asynchronous-javascript
git checkout step19
npm install
cd async-await

To create the project from scratch, execute the following command-line instructions in the directory where you’d like to place the project root directory:

mkdir async-await
cd async-await
git init
npx license mit > LICENSE
npx gitignore node
npm init -y
git add -A
git commit -m "Initial commit"
npm install esm ora node-fetch

You can learn more about initializing Node.js projects in this post by Twilio's Phil Nash.

Making multiple await-ed statements asynchronous

The await keyword can only be used within functions declared with the async keyword. One way to do this in a program that uses the await keyword multiple times is to wrap the entire program or module in an anonymous function that includes the async keyword, then invoke the function immediately after the anonymous function. This will cause the anonymous function to be executed immediately and the await-ed code to be executed asynchronously.

The following code block shows the syntax:

(async () => {
   //your code goes here
})();

Isn’t JavaScript wonderful? 🙄

The project in this tutorial uses this technique. As you work with the code, keep in mind that statements including the await keyword are paired at the highest, or outermost, level with a single anonymous function declared with the async keyword.

Building the application

Open the project in the IDE or code editor of your choice. Visual Studio Code is usually a good move.

Create a fetch.js file in the async-await folder and place the following JavaScript code in it:

import fetch from 'node-fetch';

(async () => {
   const directors = await fetch('https://maciejtreder.github.io/asynchronous-javascript/directors').then(response => response.json());
  
   console.log(directors);
})();

The above code uses the fetch method imported from the node-fetch library. The fetch method returns a Promise object that represents the response of the GET request made to the URL passed as a parameter.

That promise is chained to response.json() with the response => lambda operator to retrieve the response body rather than the whole response, which includes headers, a status code, and other information not relevant to accomplishing the task.

Run the code by executing the following command-line instruction in the async-await directory:

node -r esm fetch.js

In the console output you should see the same list of directors that’s available at the https://maciejtreder.github.io/asynchronous-javascript/directors static page:

  { id: 1

In this array you can easily find the entry containing Quentin Tarantino and retrieve his ID. Replace the code in the fetch.js file with the following:

import fetch from 'node-fetch';

const directorToCheck = 'Quentin Tarantino';
(async () => {

   const directors = await fetch('https://maciejtreder.github.io/asynchronous-javascript/directors')
      .then(response => response.json());
   const directorId = directors.find(director => director.name === directorToCheck).id;

   console.log(directorToCheck + ' id = ' + directorId);
})();

The new code adds a constant directorToCheck and uses that constant in an array .find method to get the value of .id for a specific director.

Re-run the program and verify that Quentin’s ID has been found:

Quentin Tarantino id = 2

Using the async iterator

Once you have Quentin Tarantino’s ID, you can retrieve the list of movies directed by him, and subsequently the reviews of those movies.

Find the following statement in the fetch.js file:

console.log(directorToCheck + ' id = ' + directorId);

Replace it with the following code:

const movies = await fetch('https://maciejtreder.github.io/asynchronous-javascript/directors/' + directorId + '/movies').then(response => response.json());

let reviewPromises = [];
movies.forEach(movie => {
   reviewPromises.push(
       fetch('https://maciejtreder.github.io/asynchronous-javascript/movies/'+ movie.id +'/reviews')
       .then(response => response.json())
       .then(reviews => { return {title: movie.title, reviews: reviews}})
   );
});

console.log(reviewPromises);

Run the program. In the output you should see the reviewPromises array, which contains Promise objects:

Promise { <pending> }

The above code retrieves a list of Tarantino movies and saves the request results in the movies constant. Then it instantiates the reviewPromises array, which is fed with data from within the movies.forEach loop. During each iteration of the loop, the request to the https://maciejtreder.github.io/asynchronous-javascript/movies/{id}/reviews static API is performed. A Promise representing that request result is chained with two functions:

response => response.json() to get the response body
reviews => { return {title: movie.title, reviews: reviews}} to associate the retrieved reviews with the related movie title

These modified Promises are put into the reviewPromises array.

Now the question is “What to do with such an array?”

The answer is simple: you need to have the reviews of all movies before you determine which movie is the best. In other words, you need to wait until all Promises resolve. There are two ways you can carry this out.

The first option is to use the Promise.all method, which combines Promises passed as a parameter and returns one Promise object representing all of them. In general terms, it converts the Array<Promise> to Promise<Array>.

Because reviewPromises is an AsyncIterator you have the option of using the for await…of construction to iterate through the Promises in the array and await their results. (Remember, that’s done with the await keyword in the outermost, anonymous, function.)

Replace the console.log(reviewPromises); statement in the fetch.js file with the following JavaScript code:

let moviesRating = [];
  
for await (let reviewsSet of reviewPromises) {
   let aggregatedScore = 0;
   reviewsSet.reviews.forEach(review => aggregatedScore += review.rating);
   let averageScore = aggregatedScore / reviewsSet.reviews.length;
   moviesRating.push({title: reviewsSet.title, score: averageScore});
}

The above code iterates through the Promises available in the reviewPromises, synchronously waits for the Promise to resolve, and passes the resolved value to the loop body under the let reviewsSet variable. Within the loop, an average score is calculated by adding up the scores from each review and dividing the sum by the number of reviews. The average score is put together with the movie title into the moviesRating array.

Once you’ve got the average score of each movie, you can determine which one is the best by sorting the array and displaying the first element.

Insert the following code below the for await loop:

const best = moviesRating.sort((movie1, movie2) => movie2.score - movie1.score)[0].title;
console.log('The best movie by ' + directorToCheck + ' is: ' + best);

Re-run the program and verify the output:

The best movie by Quentin Tarantino is: Inglourious Basterds

Be aware that the for await…of iterator will synchronously wait for a Promise to resolve. It won’t apply the loop body for the nth Promise until the previous n-1 Promise resolves.

For example, you have an array of promises: A, B and C. A resolves in 1 second, B in 4 seconds, and C in 2 seconds. The execution flow of the for await…of will be:

After 1 second: first loop execution with the value from promise A
After 3 seconds: second loop execution with the value from promise B
After 0 seconds: third loop execution with the value from promise C

Notice that it doesn’t matter that Promise C resolves before B, but the order of the elements in the array (A, B, C) does matter. The second loop is resolved in 3 seconds because 1 second of it’s elapsed time occurs while the first loop is being processed. The third loop resolves immediately after the second loop because the Promise C has already been resolved by the time the second loop is completed.

This is the great benefit of async/await: there is a predictable order to events, based on the order of the Promises A, B, and C in the array. There is also efficiency because the Promises are in pending status before the for await…of loop is invoked: while Promise A is proceeding toward resolution Promise B is “being worked on” and while Promise B is being processed Promise C has been resolved.

A similar rule applies to the Promise.all method, with the difference that a Promise returned by the Promise.all resolves when the last (slowest) Promise passed to it resolves. Using the Promise.all method in place of the for await…of construct will take exactly the same elapsed time in this case study.

Improving the user interface

Network APIs can take some time to respond — or may not respond at all. It’s a good idea to provide a user interface indicator to inform the user that the program is working and isn’t hung while the user and the program are both waiting for the REST API call results.

Import ora into the fetch.js file and instantiate the spinner by inserting the following code below the first import statement:

import ora from 'ora';

const directorToCheck = 'Quentin Tarantino';
const spinner = ora('The best movie by ' + directorToCheck + ' is...').start();

Replace the console.log('The best movie by ' + directorToCheck + ' is: ' + best); statement with the spinner.succeed method invocation:

spinner.succeed(best + "!");

This code will inform the program’s users that the code is running, even if it’s taking a noticeable length of time. The statements that let the user know when there are errors come next.

Catching exceptions

The solution you’ve built thus far is fully functional. But sometimes issues occur, especially when your code interacts with remote resources.

The JavaScript try…catch statement is intended for specifying what to do in response to errors in a designed code block and you can use it to wrap the code in the anonymous asynchronous function and handle possible errors like request timeouts or unexpected API responses

Modify the code in fetch.js so that the code currently inside the anonymous function body is wrapped in a try block and add the catch block shown above the end of the anonymous function body. The code below shows the result of these modifications and is the final form of the program:

import fetch from 'node-fetch';
import ora from 'ora';

const directorToCheck = 'Quentin Tarantino';
const spinner = ora('The best movie by ' + directorToCheck + ' is...').start();
(async () => {
   try {
       const directors = await fetch('https://maciejtreder.github.io/asynchronous-javascript/directors').then(response => response.json());
       const directorId = directors.find(director => director.name === directorToCheck).id;

       const movies = await fetch('https://maciejtreder.github.io/asynchronous-javascript/directors/' + directorId + '/movies').then(response => response.json());

       let reviewPromises = [];
       movies.forEach(movie => {
           reviewPromises.push(
               fetch('https://maciejtreder.github.io/asynchronous-javascript/movies/'+ movie.id +'/reviews')
               .then(response => response.json())
               .then(reviews => { return {title: movie.title, reviews: reviews}})
           );
       });

       let moviesRating = [];
  
       for await (let reviewsSet of reviewPromises) {
           let aggregatedScore = 0;
           reviewsSet.reviews.forEach(review => aggregatedScore += review.rating);
           let averageScore = aggregatedScore / reviewsSet.reviews.length;
           moviesRating.push({title: reviewsSet.title, score: averageScore});
       }
  
       const best = moviesRating.sort((movie1, movie2) => movie2.score - movie1.score)[0].title;
      
       spinner.succeed(best + "!");

   } catch(error) {
       spinner.fail(error);
   }
})();

While the code is running and waiting for responses from the APIs the spinner will display a spinner. If the code executes successfully, which includes getting appropriate responses from all three API calls, the succeed method will display the “best” movie by Quentin Tarantino, preceded by a ✔ (green checkmark). If there’s a runtime error, including an API call failure, you’ll see a 🗙(red cancelation “x”) followed by the error message.

Testing the completed app

Run the program by executing the following command-line instruction, as you have been doing:

node -r esm fetch.js

You should see the spinner, followed closely by the correct results (if everything is working correctly).

There are a number of ways you can experiment with error handling, such as turning off your network connection. You can also alter one of the API URLs slightly. For example, change the “j” in “”javascript” to an “l” (changing the metaphorical reference from hot liquid coffee to liquid hot rock). When you run the program again it should invoke the code in the catch block.

Have fun. And see you at the movies!

Summary

In this post you saw how to use the async…await construct to make a synchronous sequence of related asynchronous REST API calls in the same function and how to handle the Promise objects returned by the fetch method. The project included the for await…of construction to iterate through a collection of objects like an array of Promises using the AsyncIterator. You also saw how to add user interface features to a command-line interface project to keep the user “in the loop” while long-running asynchronous processes are executing. Finally, you learned how to wrap asynchronous code in a try…catch block to handle the error conditions raised by rejected Promises.

Additional resources

The following posts are the previous steps in this series of posts walking you through the various aspects of asynchronous JavaScript:

Asynchronous JavaScript: Understanding Callbacks – Learn the fundamentals of asynchronous processing, including the event loop and callback queue.

Asynchronous JavaScript: Introduction to JavaScript Promises – Learn how Promises work and how to use them in your own projects.

Asynchronous JavaScript: Advanced Promises with Node.js – Learn advanced features of Promises and how to use them with the Node.js runtime engine.

You might also want to learn about RxJS (ReactiveX JavaScript), which is a better asynchronous tool for some programming situations:

Asynchronous JavaScript: Introducing ReactiveX and RxJS Observables – Learn to use RxJS, the JavaScript implementation of the ReactiveX framework.

Asynchronous JavaScript: Using RxJS Observables with REST APIs in Node.js – Learn to use Observables with REST APIs, one of their primary applications.

There are also 3rd-party resources that are essential references for JavaScript developers. Here are a few:

MDN web docs: Javascript – The Mozilla Developer Network provides a comprehensive JavaScript reference site, with tutorials and reference information.

Node.js Docs – If you’re writing server-side JavaScript, the Node.js reference documentation is an essential resource.

RxJS – The site for learning resources and reference information for RxJS, a JavaScript implementation of the observer, iterator patterns along with functional programming with collections.

Which Operator do I use? – A helpful tool for choosing the best Observables operator for a desired action.

Want to have some fun while you sharpen your programming skills? Try Twilio’s video game:

TwilioQuest – Defeat the forces of legacy systems with this 16-bit style adventure.

Maciej Treder is a Senior Software Development Engineer at Akamai Technologies. He is also an international conference speaker and the author of @ng-toolkit. You can learn more about the author at https://www.maciejtreder.com. You can also contact him at: contact@maciejtreder.com or @maciejtreder on GitHub, Twitter, StackOverflow, and LinkedIn.

Gabriela Rogowska contributed to this post.

Related Resources

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.

Asynchronous JavaScript: Using Promises With REST APIs in Node.js

Related Posts

Related Resources

From APIs to SDKs to sample apps

The latest ebooks, industry reports, and webinars

Twilio's developer community hub