Add a Little AI to Your Love Letters This Valentine’s Day

Time to read: 2 minutes

February 14, 2017

Written by

Twilion

Add a Little AI to Your Love Letters This Valentine’s Day

tQT7Fjh9VmiIHe6rW8JUWcOrGP2_Jg7bnUDWnQpIVjT5AZRfXjxKynunxm5TtWuIuAL_AHozRoXXZXbbkcgmjH900ABMN8kmm8JE4CKhlcJTqtSOt_FNlmg0tZgbOhxrQUM94W4u

Happy Valentine’s Day everyone! There is a lot of love floating around today and you might be looking for a way to tell that special someone exactly what you think of them through the power of prose.

If Shakespeare isn’t your thing, Michael Jordan (of Gifted) is back with another artificially intelligent hack that will help you say all the right things today.

You can give it a try right now by texting a picture to (267) 422-7066.

Love Letters from Her

After a successful holiday season, Michael was asked by many people if there would be a Valentine’s Day rendition of Gifted. He immediately began to brainstorm ways in which he could update the artificial intelligence text messaging service to write love letters.

Michael turned to Hollywood for inspiration and rediscovered the critically acclaimed 2013 film, Her. At the beginning of the movie, Joaquin Phoenix’s character is working as a letter writer. Michael described it as “some parody of a startup you’d see on SNL.” But the scene also gave him a thought, “I wonder if you can automate that?”

What Shall Thee Write?

Michael admits he tried the Shakespearean sonnet route. But it was not personal enough and most of the results seemed like nonsense. Building on past projects, he decided to use photo tags from user generated content tag count for which he already had data. For example, “boy” being mentioned 900,000 times.

With the help of an nltk tokenizer (splitting text into relevant parts of speech) and tf-idf (term frequency-inverse document frequency), Michael was able to analyze the top 150 computer vision tags and tried mad lib style passages.

Still, mad libs were not exactly what he was aiming for. Michael ultimately wrote a thoughtfully crafted sentence for each of the 150 tags. A mechanical engineer by trade, Michael laughed as he expressed how much fun he had showing an amorous side and surprise that he knocked it out in just 15 minutes.

Using the Python Pillow library and Clarifai, when you send an mms to Gifted’s V-day phone number, Gifted will return a personalized poem on top of the image with content based upon their surroundings in the given photograph.

The app architecture hasn’t changed from Michael’s last project however, he couldn’t help but reiterate that using AWS Lambda Twilio makes connecting everything in the app easier. Rather than having to deal with the passing back and forth of media, S3 returns rich media right there!

Below is some of the code that helped Michael bring these love letters to life.

"""
This code was used to figure out the most popular 150 computer vision tags (this is an overly complex way to do it, but has more potential uses than a simple frequency counter)
"""

vectorizer = TfidfVectorizer(use_idf = True)
X = vectorizer.fit_transform(corpus)
idf = vectorizer.idf_
corpus_dict = dict(zip(vectorizer.get_feature_names(), idf))

empty_corpus = []
for i in corpus_dict:
    tfidf_score = corpus_dict[i]
    list_value = [i, tfidf_score]
    empty_corpus.append(list_value)
print 'corpus: ', sorted(hope, key=lambda x: x[1], reverse=False)[0:150]

""""
Part of Speech Tagger
"""

from nltk import word_tokenize
file = open('corpus_unique.txt')
unique_corpus = file.read() 
corpus_tokenize = word_tokenize(unique_corpus)
corpus_pos = nltk.pos_tag(corpus_tokenize)

""""
Lambda Function
"""

def photo_filter(img):
    url_open = urllib2.urlopen(img)
    img_file = io.BytesIO(url_open.read())
    im = Image.open(img_file)
    im_new = im.resize((1080,1080))
    im1 = im_new.filter(ImageFilter.GaussianBlur(radius = 10))
    im_draw = ImageDraw.Draw(im1)
    font = ImageFont.truetype("Custom_Font.ttf", 40)
    photo_quote = im_draw.text((50,60), quote, (241, 177, 170, 100), font = font)
    return im1

def photo_tagger(image):
    app = ClarifaiApp('app_id','app_secret')
    result = app.tag_urls(urls = [image], model = 'aaa03c23b3724a16a56b629203edc62c')
    name_list = []
    for num in range(20):
        name_list.append(result['outputs'][0]['data']['concepts'][num]['name'])
    return name_list

def lambda_handler(event, context):

    # User Info
    message = event['body']
    from_number = event['fromNumber']
    pic_url = event['image']
    num_media = event['numMedia']

    if num_media != '0':
        twilio_pic = urllib2.Request(pic_url, headers={'User-Agent': "Magic Browser"})
        
        #Use clarifai to tag the image
        name_list = photo_tagger(pic_url)
        
        #Create love letter quote
        quote_list = []
        quote_dict = {'Clarifai_Tag' : 'Quote'}
        for name in name_list:
            if quote in quote_dict:
        	quote_list.append(quote_dict[quote])

        #Build Quote
        full_quote = "\n\n".join(quote_list)
        first_name = message
        second_name = 'Happy Valentines Day!'
        quote = first_name + '\n\n\nWhen I look at this picture I think of...\n\n' + full_quote + '\n\n\nxoxo\n\n' + second_name
        
        #Add quote to photo
        im = photo_filter(twilio_pic)