Add Holiday Themed Filters and Overlays to Your React Video Chat App with Twilio's DataTrack API

Time to read: 11 minutes

December 14, 2020
Written by
Ashley Boucher
Twilion
Reviewed by
Diane Phan
Twilion

holiday.png

This article is going to teach you how to use the Twilio Programmable Video DataTrack API to add shareable holiday themed filters and overlays to your React video chat app. You’ll be provided with some holiday themed overlays as you carry out the tutorial, but you can apply this knowledge to make any type of filter or overlay!

You won’t be building a new video chat app from scratch, but adding to a basic video chat app that already exists instead.

Prerequisites

Before moving forward, make sure you have the following accounts and tools:

Project set up

Clone the repository

Your first step is to clone the existing video chat app from GitHub. Alternatively, if you followed along with the Build a Custom Video Chat App article, you can use the code you wrote for that project. If that’s the approach you take, then feel free to jump to the next section: Add and publish a DataTrack for the local participant.

git clone https://github.com/ahl389/video-chat-base.git

Once you’ve cloned it successfully, change your working directory to the new video-chat-base folder:

cd video-chat-base

If you explore this new directory, you’ll find two folders at the root level: frontend and backend.

The frontend folder houses the React video chat app that you’ll be working with in this tutorial.

The backend folder contains the code needed to generate an Access Token. An Access Token grants authorization to users of your video chat app. This backend uses Twilio Functions and it will need to be running alongside your frontend in order to test the app.

Configure the backend

In order to run the backend, you’ll need to make sure you have both the Twilio CLI and a CLI plugin, the Twilio Serverless Toolkit, installed. To install these, run the following commands:

npm install twilio-cli -g
twilio plugins:install @twilio-labs/plugin-serverless

Then, change your working directory to the backend folder:

cd backend

Inside the backend folder is a file called .env.example. Change the name of this file to .env (without the .example extension).

Open this file and you’ll find three environment variables with placeholder values. You’ll need to replace these placeholder values with your actual values. To find your Account SID, head to your Twilio Console. It will be on your dashboard. Next, visit the API Keys section of the Console to generate an API key and collect the values for the API Key SID and API Secret.

Once you’ve updated your .env file, you can start a local server for your backend by running the following command from the root of the backend folder:

twilio serverless:start

You’ll see a response from the CLI that contains details about the Functions service you’re now running locally, including the endpoint you’ll fetch from in the React app. This endpoint will look like http://localhost:3000/token.

Screenshot of CLI showing token endpoint

If you’re running your local server on a port other than 3000, your endpoint will look slightly different.

Hang onto the URL for this endpoint. You’ll need it in the next step.

Configure the frontend

With your backend fully configured and running, it’s time to switch gears to the frontend, where the rest of your work will take place.

First, in your command prompt, navigate to the frontend directory. From the frontend directory, install all dependencies:

npm install

Next, before you can dive into building, you need to update the React app to reflect the endpoint you received in the previous section.

Look for the file named App.js inside the frontend/src folder.

On line 26 of this file, at the beginning of a method called joinRoom(), you should see some code that looks like this:

const response = await fetch(`{your-endpoint}?identity=${this.state.identity}`);

Change this line so it’s fetching from your endpoint. If you’re running the backend locally on PORT 3000, then you’re updated line will be:

const response = await fetch(`http://localhost:3000/token?identity=${this.state.identity}`);

Get to know the frontend

Now that your app is configured, take a moment to learn about the different components and how they work together:

The App component is the top most component in your app. This component controls what the user sees when they land at your app and it handles the user driven actions of joining and leaving the room. It has one child component: Room.

The Room component is the container for all the unique participants in the video room. It also listens for new remote participants coming or existing remote participants leaving. It can have one or more child Participant components.

Next, the Participant component manages the given participant’s audio and video tracks. Each of these tracks are represented via child Track components.

Finally, the Track component is responsible for attaching and rendering the track it receives as props.

To see how it all works together, start the app by running the following command from the frontend directory (make sure your backend is still running):

npm start

If your backend is on PORT 3000, you’ll be prompted to run your React app on a different port. Press the y key on your keyboard to confirm. When you’re done exploring the app you can stop your server by pressing CTRL + C.

Add and publish a DataTrack for the local participant

As of right now, the app only utilizes two tracks from each user: their audio track and their video track. Whenever a user joins the video chat, their audio and video tracks are automatically published and they are also automatically subscribed to by all the other users. This publication and subsequent subscription is how all users connected to the video room can see each other and hear each other right away.

Twilio Programmable Video offers a third type of track: a DataTrack. This track allows users to share data to other users via a track that must be published and subscribed to just like audio and video tracks. It’s this type of track that will be used to send information about the video filters to each participant.

Your first order of business is to give your app access to the DataTrack API by importing LocalDataTrack from the Twilio Programmable Video JavaScript SDK.

Open the App.js file inside the frontend/src folder.

At the top of the file you’ll see a few imports. Change line 4 so LocalDataTrack is also being imported:

const { connect, LocalDataTrack } = require('twilio-video');

Now jump down to the joinRoom() method that you visited earlier.

When a user of the app clicks the Join Room button on the UI, this method is called. joinRoom() fetches the Access Token from your backend, and then connects to a video room called cool-room. This room name could be anything, and typically, in a production app, it would not be hard coded.

After the connection is complete, Twilio returns a room object that contains lots of data about the video room, including its participants. The local participant, i.e. the user who clicked the Join Room button, is available on the room.localParticipant key.

In order to give the local participant a data track, you can create one after they connect to the room and then publish it.

Below the room variable assignment, on line 33 of App.js, inside the joinRoom() method, add the following code:

      const localDataTrack = new LocalDataTrack();
      await room.localParticipant.publishTrack(localDataTrack);

This code creates a new instance of Twilio’s LocalDataTrack class, and then publishes it for the local participant. Now, when the Room component renders, this new track will be available on the local participant.

Here is the code for the entire joinRoom() method, with the new lines highlighted:

  async joinRoom() {
    try {
      const response = await fetch(`https://localhost:3000/token?identity=${this.state.identity}`);
      const data = await response.json();
      const room = await connect(data.accessToken, {
        name: 'cool-room',
        audio: true,
        video: true
      });

      const localDataTrack = new LocalDataTrack();
      await room.localParticipant.publishTrack(localDataTrack);

      this.setState({ room: room });
    } catch(err) {
      console.log(err);
    }
  }

Listen for remote participants’ DataTracks

So far you’ve added code that creates and publishes DataTracks for new participants as they join the video room.

The other people in the room don’t have access to the new participants’ DataTracks yet, though.

Whenever someone publishes a track, an event is emitted. Remote participants can listen for this event, and get access to the new track in the event listener callback.

Open up the Participant.js file inside frontend/src. This file contains all the code for the Participant component. On any given instance of the app, there will be one Participant component rendered for every person in the video room.

Look at the componentDidMount() method starting at line 18 of Participant.js.

componentDidMount() is a special React lifecycle method that is called only when the component mounts initially, and not again. For that reason, it’s a good place to make network requests or add event listeners.

This method already creates an event listener on the Participant components that are not representing the local participant. These event listeners are listening for the trackSubscribed event, which is fired whenever a remote participant joins and all the other participants are automatically subscribed to their audio and video tracks.

You’ll want to add an additional event listener inside that if statement that will listen for the trackPublished event. This event will fire whenever a remote participant connects to the room and publishes a track (this could be any track, but in this case, it’s the publication of the DataTrack upon connection that will cause the event to fire).

Pass data over the DataTrack

So far you’ve created and published a new DataTrack for every new participant that joins the video room. You’ve also created event listeners that wait for these new DataTracks and then add them to your app.

With the track publications and subscriptions in place, your next step is to learn how to actually send and display received data via the DataTrack.

The goal for this tutorial is to allow participants to select video overlays that can be seen by everyone on the call. These filters/overlays are built with HTML (via React components) and CSS.

When a user selects the overlay they want to display, a new Filter component will be attached to their instance of the app.

Their choice of filter, represented by a string with the filter’s name, will be sent over their DataTrack and received by the other running instances of the app (their friends!). At that point, the remote instances of the app will rerender and, using the received filter name, the appropriate Filter component will be attached for all remote participants.

Time to make it happen. You’ll import the filters first, then build out some supporting components, and then learn how to pass and receive the data.

The filters

Take a look inside the frontend/src folder. You’ll see an additional folder called filters. This folder includes three pre-written filters/overlay components that you can import into your app. These are holiday themed and include:

  • A green to red gradient overlay
  • Twinkling festive lights
  • Snowfall (I learned how to make the snowfall effect for this component from a tutorial by Rahul Arora at W3Bits).

To make these filters work locally, you’ll need to create two supporting components: Filter and FilterMenu.

Start with FilterMenu. This component is responsible for showing a list of available filter options to the local participant.

Create a new file inside frontend/src called FilterMenu.js. Copy and paste the following code into this file:

import './App.scss';

function FilterMenu(props) {
  const filters = ['None', 'Snowfall', 'GreenRed', 'Twinkle'];

  return (
    <div className="filterMenu">
      { 
        filters.map(filter => 
          <div className={`icon icon-${filter}`} 
            onClick={ () => props.changeFilter(filter) }>{filter}
          </div>
        )
      }
    </div>
  );
}

export default FilterMenu;

This is a functional React component that maps over an array of filter names and returns a <div> element with an onClick attribute for each one. When the user clicks on any of these filter names, a method called changeFilter(), which the component receives as props, will be called.

Save and close this file. You’re now going to import and attach this component to the Participant component.

Revisit the Participant component

Open Participant.js inside frontend/src. First, import the FilterMenu component up top:

import React, {Component} from 'react';
import './App.scss';
import Track from './Track';
import FilterMenu from './FilterMenu';

Next, jump down to the render() method.

You’re going to add some code inside the render() method that will conditionally render the FilterMenu component, dependent on whether this participant is the local participant.

Update your render() method to reflect the highlighted lines:

  render() {
    return ( 
      <div className="participant" id={this.props.participant.identity}>
        <div className="identity">{ this.props.participant.identity}</div>
        {
          this.props.localParticipant
          ? <FilterMenu changeFilter={this.changeFilter} />
          : ''
        }
        
        { 
          this.state.tracks.map(track => 
            <Track key={track} filter={this.state.filter} track={track}/>)
        }
      </div>
    );
  }

Take note of two things in these changes: first, a component method called changeFilter() is being passed as props to the FilterMenu component. Second, a filter prop was added for the Track component that contains the Participant component’s filter state.

To add the changeFilter() menu to the component, copy and paste the following code directly above the render() method:

  changeFilter(filter) {
    const dataTrack = this.state.tracks.find(track => track.kind == "data");
    dataTrack.send(filter);
    this.setState({ filter: filter });
  }

This method receives the name of the filter to change to. It finds the DataTrack among the participant’s three tracks (audio, video, and data). Then, for the magical part, it sends the name of the filter over the DataTrack, using the Twilio method send(). You’ll add the code to receive this filter name in just a little while.

Then, after sending the filter name, this method updates the component’s filter state to the new filter.

To round out this component, jump up to the constructor method, update the state object to include a filter state key:value pair. The value, initially, will be None. Also, bind your new changeFilter() method to the JavaScript keyword this:

  constructor(props) {
    …

    this.state = {
      tracks: nonNullTracks,
      filter: 'None' 
    }

    this.changeFilter = this.changeFilter.bind(this);
  }

Create the Filter component

Create a new file inside frontend/src called Filter.js. Inside this file, copy and paste the following code:

import './App.scss';
import * as componentMap from './filters';

function Filter(props) {
  const Filter = componentMap[props.name]
  return <Filter/>
}

export default Filter;

This code imports all the available filters from the frontend/src/filters folder. It then creates a functional React component called Filter that receives props (the name of the filter). Based on the name passed as props, this component will dynamically render the correct filter component.

Edit the Track component

At this point you’ve learned how to publish and subscribe to new DataTracks. You’ve learned how to send messages over a DataTrack. You’ve also added a filter menu that’s available to every participant and implemented a way for users to change their filter.

Your final steps are going to be rendering the filter for the local participant, and receiving and displaying that filter for all the remote participants.

Open the Track.js file inside frontend/src.

At the top of the file, import the Filter component:

import React, {Component} from 'react';
import './App.scss';
import Filter from './Filter';

Update the constructor method to create a new filter state for the Track component:

  constructor(props) {
    super(props)

    this.state = {
      filter: ''
    }

    this.ref = React.createRef();
  }

This new filter state is initialized as an empty string.

Now take a look at the componentDidMount() method. Right now this method checks to make sure the track object isn’t null. If it’s not, it attaches the track to the DOM. This works for audio and video tracks, but the attach() method doesn’t apply to DataTracks.

Likewise, the componentDidMount() method is where you’ll add the event listener that’s fired whenever a message is sent over the DataTrack. But this listener only applies to DataTracks and not to audio or video tracks.

Replace your componentDidMount() method with the following code to reflect these constraints:

  componentDidMount() {
    if (this.props.track) {
      if (this.props.track.kind !== 'data') {
        const child = this.props.track.attach();
        this.ref.current.classList.add(this.props.track.kind);
        this.ref.current.appendChild(child)
      } else {
        this.props.track.on('message', message => {
          this.setState({filter: message})
        });
      }
    } 
  }

Now, for the final step, rendering the filter!

Update the Track component’s render() method with the highlighted lines:

  render() {
    return (
      <div className="track" ref={this.ref}>
        {
          this.props.track && this.props.track.kind === 'data'
          ? <Filter name={this.state.filter || this.props.filter} />
          : ''
        }
      </div> 
    )
  }

With these changes, if the given track is a DataTrack, it will attach the Filter component, and will pass to it the filter name (as represented by either the filter state, for remote instances of the app, or the filter props, for local participants).

Congratulations, your video app participants can now add and share video filters/overlays amongst themselves!

Test out the filters and overlays on the video chat app

Save and close all the files you’ve been working with.

Make sure your backend from the first part of this tutorial is still running.

Then, open a new command prompt tab or window and navigate to the root of your React project (video-chat-base/frontend).

Run npm start to start your local server. If your backend is running on PORT 3000, you’ll be prompted to start your React server on a different port. Press y to continue.

Once your app is running, head to your browser and visit localhost:3000 (or whatever your port is).

You’ll see the app’s lobby. Enter your name in the field and click Join Room.

Open a new browser tab and load your app there as well. Enter a different name in the field and join the room. For each “user”, select a filter and see it displayed for all participants.

GIF showing filters

Conclusion

This article taught you how to implement the Twilio Programmable Video DataTrack API inside a React App to share holiday themed video overlays among all participants of the app.

I hope you had fun! If this was exciting to you, check out some of the other projects you can build with the DataTrack API, like adding Snack Chats to your app. Let me know on Twitter what you’re building!

Ashley is a JavaScript Editor for the Twilio blog. To work with her and bring your technical stories to Twilio, find her at @ahl389 on Twitter. If you can’t find her there, she’s probably on a patio somewhere having a cup of coffee (or glass of wine, depending on the time).

 

 

Related Posts

Related Resources

A newspaper article

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
An Open book

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.
User group reactions

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.