Building your own Twitch Plays Pokemon with JavaScript, Twilio Programmable Chat and Socket.io

Time to read: 8 minutes

December 10, 2015
Written by
Sam Agnew
Twilion

Screen Shot 2015-12-10 at 12.28.39 PM

t0m2SHFN0CvBu.gif
In early 2014, a phenomenon called Twitch Plays Pokemon took over the Internet. It was a social experiment on Twitch.tv which allowed viewers to play Pokemon Red Version together using Twitch’s chat and it was awesome. We were all on the edge of our seat for several weeks as over 120,000 people around the world played together simultaneously to guide Red on his Pokemon quest. There is even still an active community for the channel despite the initial run of the game being completed.

With the new Twilio Programmable Chat API coming out, I knew I had to work on my own version of this in the browser with a custom chat app and no dependence on streaming software. I’ve had a ton of fun building this and now I’m going to show you how to as well. If you want to skip to just playing Pokemon with a bunch of people, you can check it out here.

Throughout the rest of this post, we will recreate this experience by building “Twilio Plays Pokemon” using the new Programmable Chat API for the in-browser chat and Socket.io to sync a server side gameboy emulator across multiple browser clients. Socket.io has an example of something like this up already called Weplay. We will be using some of their code for the gameboy emulation setup and then implementing the chat ourselves. You can get started with this repository.

gMECDVV4VUAmc.gif

Acquiring our starters

Instead of Bulbasaur, Squirtle and Charmander, we will need to set up several dependencies to begin this adventure:

  • You’ll need to create a free Twilio account in order to run the code for the chat application we are building.
  • You’ll need to have Node.js and npm installed. I currently have Node v4.0 and npm v2.14.2 on my machine.
  • You’ll also need to install Redis if you don’t already have it.
  • node-canvas: Follow these directions for your operating system in order to install.

Run the Redis server now because we will be using it later. You can do this in the background on the default port with the following command:

redis-server &

To start off, grab the code from this repository. You can grab this from a ZIP file or using git with the following commands in your terminal. You might have to enter your GitHub credentials to do this:

git clone https://github.com/sagnew/Twilio-IP-Messaging-Plays-Pokemon.git
cd Twilio-IP-Messaging-Plays-Pokemon
git checkout original-game

Now you have the code you need to get the Gameboy emulator running. This is an updated version of the example Socket.io has on their website.

Get all of our dependencies installed by running this command from the root directory of the project:

npm run deps

If you are using OSX and had trouble with the npm install step for weplay-emulator due to problems with the dependencies on Cairo, you may need to update to Xcode 7 and have the new command line tools. Alternatively, entering cd /usr/local/lib && sudo ln -s ../../lib/libSystem.B.dylib libgcc_s.10.5.dylib may work as a temporary fix.

Leaving Pallet Town

Now that everything is installed, let’s get it running to make sure it works. We should be able to run the emulator and see the title screen of the game in a web browser if everything is installed correctly.

There are 2 other important processes you’ll need to run and they all exist in different repositories. I compiled them together and updated the dependencies so that you can get everything running quickly.

Each directory contains the code for different processes that we will need to run together for everything to work:

  • weplay/ – A directory containing the process that sets up the socket connections between the web app and the emulator.
  • weplay-emulator/ – A directory containing the process that will run the Game Boy emulator and paint every frame to a node-canvas. The raw data of each frame will be emitted as an image to the client. This is how we can have a client side Game Boy “emulator” that is actually just an
tag.

You’ll have to set an environment variable to have a path to a ROM of a Pokemon game before beginning everything:

export WEPLAY_ROM="/path/to/gameboy/rom"

If you don’t have access to a ROM of Red Version, you can test the same functionality with any other Game Boy game. I’d recommend this awesome open source game called Tuff that was made in 2014.

Now run the web app so you can see if everything works:

npm start

When you visit http://localhost:3000 you should see this:

Screen Shot 2015-11-24 at 4.23.59 PM.png

Earning your first badge

So we have the game up and running in your browser, but you can’t play it yet because we haven’t written the code for receiving chat commands. In order to use Twilio Programmable Chat to implement the chat on the client side, we will need to have a server that generates tokens. Think of this portion as a battle with Brock in Pewter City to earn the Boulder Badge, but our reward will instead be an access token for the Programmable Chat API.

First, you’ll need to create an API key and a service SID. You can create an API key and a secret here. When you generate an API key pair at the URLs above, your API Secret will only be shown once – make sure to save this in a secure location, or possibly your ~/.bash_profile.

A service instance provides a shared scope for all the messages, users, and data in our Programmable Chat application. It’s like a new database for all your app’s data. You can create one here, which is also where you can grab the SID for it that we are about to use.

You’ll need to set the following environment variables as well:

export TWILIO_ACCOUNT_SID="INSERT-ACCOUNT-SID-HERE"
export TWILIO_API_KEY="INSERT-API-KEY-SID-HERE"
export TWILIO_API_SECRET="INSERT-API_SECRET-SID-HERE"
export TWILIO_IPM_SERVICE_SID="INSERT-IPM-SERVICE-SID-HERE"

Now it’s time to add some code to the server side Express app to generate a token that authenticates the client. From the root directory open up index.js and add the following imports to the beginning of the file:

var browserify = require('browserify-middleware');
var mustache = require('mustache-express');
var express = require('express');
var app = express();
var AccessToken = require('twilio').AccessToken;
var IpMessagingGrant = AccessToken.IpMessagingGrant;

And add this route to generate tokens to the end of the file:

/*
Generate an Access Token for a chat application user - it generates a random
username for the client requesting a token, and takes a device ID as a query
parameter.
*/
app.get('/token', function(request, response) {
  var appName = 'TwilioChatDemo';
  var identity = request.query.identity;
  var deviceId = request.query.device;
  // Create a unique ID for the client on their current device
  var endpointId = appName + ':' + identity + ':' + deviceId;

  // Create a "grant" which enables a client to use IPM as a given user,
  // on a given device
  var ipmGrant = new IpMessagingGrant({
    serviceSid: process.env.TWILIO_IPM_SERVICE_SID,
    endpointId: endpointId
  });

  // Create an access token which we will sign and return to the client,
  // containing the grant we just created
  var token = new AccessToken(
    process.env.TWILIO_ACCOUNT_SID,
    process.env.TWILIO_API_KEY,
    process.env.TWILIO_API_SECRET
  );
  token.addGrant(ipmGrant);
  token.identity = identity;

  // Serialize the token to a JWT string and include it in a JSON response
  response.send({
    identity: identity,
    token: token.toJwt()
  });
});

On the client side, we will send an AJAX GET request to this route in order to receive a token to authenticate with the Programmable Chat API.

Facing the Elite Four with our client side code

Now that you can authenticate, let’s add chat functionality to our client side code using Twilio Programmable Chat. The client side JavaScript for our app is in the client/ directory, with most of the code residing in app.js.

Let’s create a module for functions specific to our chat app. We will use this code
in app.js. To start off, add a couple of global variables that will be used in several chat related functions with the IP Messaging API. In the client/ directory create a file called chat.js and add the following:

// We will be using jQuery to manipulate DOM elements for the chat.
var $ = require('jquery');
// Manages the state of our access token we got from the server
var accessManager;

// Our interface to the Programmable Chat service
var messagingClient;

// The object we are going to export as this module
module.exports = {
};

Now add a utility function for “printing” a message to the chat. This function will be called whenever a client receives a message and will display the message in the chat box using jQuery. Add this to the module.exports object right after line 10:

 printMessage: function(msg, by) {
    var p = $('<p>').text(msg);
    if (by) {
      p.prepend($('<span class="message-by">').text(by + ': '));
    } else {
      p.addClass('server');
    }
    $('.messages').append(p);
  },

Next we want to grab a token from the server and initialize an Programmable Chat application. Let’s create a function called initializeChat that will request a token from our server, initialize our global variables and then call a utility function that we will write in the next step. Add this after printMessage:

  // Connect to the Twilio Programmable Chat API and set up the chat app
  initializeChat: function(identity) {

    // Get an access token for the current user, passing a username (identity)
    // and a device ID - for browser-based apps, we'll always just use the
    // value "browser"
    $.getJSON('/token', {
      identity: identity,
      device: 'browser'
    }, function(data) {
      // Initialize the Programmable Chat client
      accessManager = new Twilio.AccessManager(data.token);
      messagingClient = new Twilio.IPMessaging.Client(accessManager);

      this.setupChannel();
    }.bind(this));
  },

Now let’s write the functions setupChannel and setListeners. setupChannel will retrieve the “general” channel or create on if it doesn’t exist and then call the setListeners function that will print new messages to the chat whenever a message is received in the channel. These will go right after the initializeChat function:

  setupChannel: function() {
    // Get the general chat channel, which is where all the messages are
    // sent in this application
    var promise = messagingClient.getChannelByUniqueName('general');
    promise.then(function(channel) {
      this.channel = channel;
      if (!this.channel) {
        // If it doesn't exist, let's create it
        messagingClient.createChannel({
          uniqueName: 'general',
          friendlyName: 'General Chat Channel'
        }).then(function(channel) {
          this.channel = channel;
          this.setListeners();
        }.bind(this));
      } else {
        this.setListeners();
      }
    }.bind(this));
  },

  // Set up channel after it has been found
  setListeners: function() {
    var onMessageAdded = function(newMessage) {
      this.printMessage(newMessage.body, newMessage.author);
    }.bind(this);

    // Join the general channel
    this.channel.join().then(function(channel) {
      // Listen for new messages sent to the channel
      channel.on('messageAdded', onMessageAdded);
    });
  },

Now add a function called join that executes after a user enters their name. It will have a call to the new initializeChat function we just made. You can add this after the other functions we just wrote:

  // Function that will be called in order to join the chat with a given username.
  // This calls initializeChat after
  join: function(username) {
    $('body').addClass('joined');
    $('.input').addClass('joined');
    $('.input input').attr('placeholder', 'type in to chat').blur();

    // If the Programmable Chat channel doesn't exist yet, initialize it.
    if(!this.channel) {
      this.initializeChat(username);
    }
  }

After all of that, your code in chat.js should look like this.

It’s time to navigate the code that already existed for the client side app so we can use these chat functions to hook into the game. In client/app.js, first add the following line to the top of the file with the other require statements so we can use our chat module:

var chat = require('./chat');

Now we have to allow users to send messages to the chat and also check to see if a message that a user submits is a keyword for input to the game. If the message is a keyword, we’ll use Socket.io to emit which command was entered. This will access the process running the emulator to send that as button input to the game. At the end of app.js in the client directory, add the following code to send messages to the channel whenever the form is submitted:

$('.input form').submit(function(ev) {
  // First prevent the form from being submitted
  ev.preventDefault();

  // Create an array of strings corresponding to GameBoy buttons to test against input
  var gbButtons = ['left', 'right', 'up', 'down', 'a', 'b', 'start', 'select'];
  var enteredText = input.val();

  // Do nothing if no text was entered.
  if ('' === enteredText) {
    return
  }

  // Remove the message that was sent from the form input
  input.val('');

  // Check to see if the user has already joined the chat
  if (joined) {
    if (gbButtons.indexOf(enteredText.toLowerCase()) !== -1) {
      chat.channel.sendMessage(enteredText);
      socket.emit('move', enteredText);
    } else {
      chat.channel.sendMessage(enteredText);
    }
  } else {
    // If the user hasn't joined, then join using their entered text as a username
    nick = enteredText;
    joined = true;
    chat.join(enteredText);
  }
});

Note that if a user has already joined, we send a message. If a user has not joined the channel yet, we call the join function.

Restart your server again by running npm start and you should now have a fully functional Twitch Plays Pokemon clone with Twilio IP Messaging.

IPMPlaysPokemon.gif

Congratulate yourself for making it this far and making an awesome Twitch Plays Pokemon clone. You are now as rad as this “10 year old” kid who has 10 badges even though there are only 8 of them in Kanto.

gary-oak-10-badges.jpg

Finishing your quest to catch ‘em all

We may be done with the basic functionality, but that’s no reason to end your adventure. If you want to keep playing around with this you can add onto the chat app in many different ways.

You could try:

  • Filtering out the chat messages to not allow bad content to go through. You could have it stop users from saying “Gary Oak” or whatever you named your in-game rival because everyone knows he’s not a nice guy.
  • Adding authentication to verify users’ identities. Right now our chat app works more like IRC in which anyone can use any screen name.

 

Diploma_RB.png
I can’t wait to see what amazing things you do with the new Programmable Chat API. You might even be able to make your Twitch Plays Pokemon clone go viral by adding some novel features. If you have any questions or want to show off what you built, feel free to reach out to me.

Related Posts

Related Resources

A newspaper article

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
An Open book

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.
User group reactions

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.