Create, Innovate, and Make Art with Machine Learning

Time to read: 3 minutes

March 29, 2020
Written by
Lizzie Siegle
Twilion

Make Art with Machine Learning

One thing I love about programming is the chance to get creative. There are many cool APIs that help developers make art. This post will go over how to generate text from another text or image, recognize celebrities' faces, colorize photos and videos, and perform neural style transfer using the machine learning library DeepAI.

the Scream edited pikachu image

What is DeepAI?

DeepAI is a platform that aggregates guides such as Separate sampling: its effects and one potential solution, as well as research, data science, a glossary, a newsletter, job listings, and more to increase AI accessibility to both consumers and developers. This post will look at some of its machine learning models and APIs that can help you get creative juices flowing.

They offer language support for JavaScript, Python, Ruby, and C#, and you can also hit the APIs with a cURL request.

Prerequisites

To follow along with this post, make an account on DeepAI.org and take note of your API key which can be found on your dashboard.

DeepAI Dashboard

Then install the DeepAI npm module. At the top of a .js file (mine is called deepai.js) add the following lines:

const fs = require('fs');
const deepai = require('deepai');
deepai.setApiKey('REPLACE-WITH-YOUR-DEEPAI-API-KEY');

Now it's time to start building art with machine learning.

Generating fiction with DeepAI

I love reading and writing, especially fiction. DeepAI's text-generator API lets you generate text according to an input string, also known as seed text in natural language processing (NLP.)

Below is an example of JavaScript code using DeepAI's text-generator API to complete an unfinished paragraph from Harry Potter and the Sorcerer's Stone (complete book in .txt form found on GitHub here). I saved it to an assets folder locally. You can save either one, but this post uses the unfinished paragraph.

(async function () {
    resp = await deepai.callStandardApi("text-generator", {
        text: fs.createReadStream("assets/hp11st2ps.txt"),
    });
    fs.writeFile('assets/gentext.txt', JSON.stringify(resp), (err) => {
        //throw error
        if (err) throw err;
        //file saved
        console.log('image saved!');
    });
    console.log(resp);
})();

Given the first one-and-a-half sentences of Harry Potter, the returned text is

"Mr. and Mrs. Dursley, of number four, Privet Drive, were proud to say that they were perfectly normal, thank you very much. They were the last people you'd expect to be involved in  ̈the \"first-ever\" team meeting. They said it was a great opportunity for you to spend your entire time in hockey, and they even took the time to meet with this guy who had already done a lot to help me improve my game as a coach..."

In addition to a locally-hosted file, you could generate text from a URL, text string, and other forms as well.

Generating text from an image

Something else you can do with DeepAI is generate text from an image with their Neural Talk 2 API. The code below takes in this image of Detective Pikachu dancing

 

happy Pikachu Dance image

and outputs "a teddy bear sitting on top of a table."

Description of Pikachu image

Why yes, that is an extremely cute prediction summary of Pikachu, but it's not quite the most accurate! Here's the code:

(async function () {
resp = await deepai.callStandardApi("neuraltalk", {
        image: "https://www.syfy.com/sites/syfy/files/2019/05/untitled.png",
    });
    console.log(resp);
})();

Recognizing Celebrities in an image

DeepAI also has a Celebrity Recognition API that recognizes famous people from images.

Shaq and Kobe

This JavaScript code classifies (and misclassifies) this above image of Shaq and Kobe via URL. 

(async function () {
resp = await deepai.callStandardApi("celebrity-recognition", {
        image: "https://pbs.twimg.com/media/BhxWutnCEAAtEQ6.jpg",
    });
    console.log(JSON.stringify(resp));
})();

It is fairly confident one of them is Kobe (and it classified another image of Kobe as well), but even more confident that Shaq is Joe Fraiser.

Kobe recognized output

Alternatively you could instead pass in images hosted locally, from a website, and more.

Colorizing Images

Perhaps a more useful API DeepAI offers is image colorization. This allows you to add color to old or historic pictures or video and perform photo restoration as well, like on this black-and-white image of San Francisco Chinatown.

black-and-white SF Chinatown image

Given this JavaScript code and that image, DeepAI returns this URL to the colorized output image (shown below.)

(async function () {
resp = await deepai.callStandardApi("colorizer", {
        image: "https://www.terragalleria.com/images/black-white/us-ca/usca9303-bw.jpeg",
    });
    console.log(resp);
}();

colorized output of B&W SF Chinatown photo

Performing Neural Style Transfer

Last but not least, the Fast Style Transfer API lets you generate art using deep learning. It quickly performs neural style transfer which is an optimization technique that takes in two images: one content image and one style image (ie. a famous painting). The transfer combines the two pictures together and returns an output image with the content of the content image and the style of the style image.

side-by-side Pikachu and the Scream

Given these two images above (including the Scream by Edvard Munch on the right), I added them to a folder called assets and then the code below returns this link containing the output image below.

(async function () {
 resp = await deepai.callStandardApi("fast-style-transfer", {
        content: fs.createReadStream("assets/avatar.png"),
        style: "https://cdn.kastatic.org/ka-perseus-images/6a6151155fbde50cec7b9616661c1448d3374fa7.jpg",
    });
    console.log(resp);
})();

That code outputs the first image in the style of the second:

Lizzie + Pikachu in style of the Scream

Now that's neural style transfer!

Go forth and make art with code

If you're stuck at home, now is the perfect time to experiment with machine learning libraries like DeepAI and to creatively make art, whether generating text from other text or image, recognizing celebrities' faces, colorizing photos or videos, or performing neural style transfer. Again, DeepAI lets you do all these tasks and more with JavaScript, Python, Ruby, C#, or a cURL request. Let me know what you're building in the comments or online.

Related Posts

Related Resources

A newspaper article

Twilio Docs

From APIs to SDKs to sample apps

API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
An Open book

Resource Center

The latest ebooks, industry reports, and webinars

Learn from customer engagement experts to improve your own communication.
User group reactions

Ahoy

Twilio's developer community hub

Best practices, code samples, and inspiration to build communications and digital engagement experiences.