Process Incoming Emails with Laravel and SendGrid Inbound Parse

Time to read: 5 minutes

June 26, 2020

Written by

Michael Okoko

Contributor

Reviewed by

Marcus Battle

Twilion

Process Incoming Emails with Laravel and SendGrid Inbound Parse

Sending emails from your Laravel app is a common requirement, but sometimes, you want to receive and process the replies programmatically as well. SendGrid lets you process inbound emails with Inbound Parse. Inbound Parse allows you to provide a webhook URL to process all incoming emails for a domain or subdomain. Subdomains are recommended as they don’t affect your regular domain emails.

In this tutorial, we will be building a blog application where users can comment on a post by replying to a transaction email.

Prerequisites

To complete this tutorial, you will need:

Composer and the Laravel installer on your machine.
An authenticated domain on SendGrid. You can follow the guide here to set one up.
Ngrok installed (for creating a tunnel on your localhost)

Create Migrations and Models

To get started, create a new Laravel project in your preferred location (I am naming mine sg-inbound), switch to the folder and install the sendgrid PHP library.

$ laravel new sg-inbound
$ cd sg-inbound
$ composer require sendgrid/sendgrid

Next, visit the API Keys page on your SendGrid dashboard and click on the Create API Key button to generate a new API key. Save the generated key as the value for your SENDGRID_API_KEY in the application .env file.

SENDGRID_API_KEY="YOUR_SENDGRID_API_KEY"

This application needs a Post and Response model to wrap the blog’s posts and comments respectively. Create these models as well as their migration files by running the commands below:

$ php artisan make:model -m Post
$ php artisan make:model -m Response

Next, update the Post’s migration file in the database/migrations folder to include the required tables by replacing its content with the code below:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

class CreatePostsTable extends Migration
{
        public function up()
        {
            Schema::create('posts', function (Blueprint $table) {
                $table->id();
                $table->string("title");
                $table->mediumText("body");
                $table->timestamps();
            });
        }

        public function down()
        {
            Schema::dropIfExists('posts');
        }
}

Similarly, replace the contents of the Response’s migration file with the following code:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

class CreateResponsesTable extends Migration
{
        public function up()
        {
            Schema::create('responses', function (Blueprint $table) {
                $table->id();
                $table->unsignedBigInteger("post_id");
                $table->string("email");
                $table->mediumText("body");
                $table->timestamps();

                $table->foreign("post_id")
                    ->references("id")
                    ->on("posts");
            });
        }

        public function down()
        {
            Schema::dropIfExists('responses');
        }
}

Next, modify the generated models to reflect the one-to-many relationship between Posts and Responses (i.e, a post can have many responses but a response belongs to a single post) by updating the Post model app\Post.php as shown:

<?php

namespace App;

use Illuminate\Database\Eloquent\Model;

class Post extends Model
{
        protected $fillable = [
            'title',
            'body'
        ];

        public function responses() {
            return $this->hasMany(Response::class);
        }
}

In the same way, add the belongsTo relationship to the Response model app/Response.php to tie it to a single Post.

<?php

namespace App;

use Illuminate\Database\Eloquent\Model;

class Response extends Model
{
        protected $fillable = [
            'post_id',
            'email',
            'body'
        ];

        public function post() {
            return $this->belongsTo(Post::class);
        }
}

With your migrations and relationships set up, run the migrate command to generate the databases. Make sure that you have set a database in the .env file.

$ php artisan migrate

Add Dummy Posts with Seeds

A factory will be needed to generate sample posts using sample data from the Faker library, which are in turn saved to the database using seeders. To create such a factory for the Post model, run php artisan make:factory PostFactory and replace the contents of the generated file (i.e database/factories/PostFactory.php) with the code block below:

<?php
use Faker\Generator as Faker;
/** @var \Illuminate\Database\Eloquent\Factory $factory */

$factory->define(\App\Post::class, function(Faker $generator) {
        return [
            'title' => $generator->sentence(),
            'body'=> $generator->text(1000)
        ];
});

Next, generate a seeder for the posts table with php artisan make:seed PostTableSeeder. Update the newly created seeder file (i.e database/seeds/PostTableSeeder.php) to generate a couple of posts and save them.

<?php

use Illuminate\Database\Seeder;

class PostTableSeeder extends Seeder
{
        public function run()
        {
            factory(\App\Post::class, 5)->create();
        }
}

Make Laravel aware of the PostsTableSeeder by updating the run method in DatabaseSeeder.php as shown in the code below:

public function run()
        {
            $this->call(PostTableSeeder::class);
        }

Now that your seeders are in place, run php artisan db:seed to populate the database.

Set Up App Routes

Open the web routes file at routes/web.php and replace its content with the code below.

<?php

use Illuminate\Support\Facades\Route;

Route::get('/', 'PostsController@index')->name('index');
Route::get('/posts/{id}', 'PostsController@show')->name('view-post');
Route::post('/posts/receive-email-response', 'PostsController@receiveEmailResponse');
Route::get('send-email', 'PostsController@sendMails');

The code above registers the following URLs on the application

GET /: Renders a list of published blog posts
GET /posts/:id: Renders a single post and all of the post responses/comments.
POST /posts/receive-email-response: The webhook handler for processing POST requests from SendGrid
GET /send-mail: The route to stimulate sending out reply-able emails. Ideally, you may want to send out such emails each time a new post is published instead

You will notice that the routes reference a PostsController which doesn’t exist yet. Run the artisan command for creating a new controller in your project directory.

$ php artisan make:controller PostsController

The command will generate a new file at app/Http/Controllers/PostsController whose content is just an empty PHP class declaration. Let’s go over the methods specified in the route file before implementing them.

index: Uses Laravel pagination to load the blog posts and renders them
show: Renders a single blog post, as well as all responses to the post.
receiveEmailResponse: The webhook handler which receives POST requests (from SendGrid), parses and extracts the data it needs from the request payload, and creates a new response in the database. It then sends back a 200 OK response back so SendGrid knows we have successfully received the payload.
sendMails: A fill-in method to send out emails to an array of addresses which can then be replied to comment. It works by appending the post ID to the email address so that they are in the form (replies+{POST_ID}@example.com). That way, we can extract the ID when replies come back in and process it accordingly.

Open the PostsController file and replace its content with the code below:

<?php
namespace App\Http\Controllers;

use App\Post;
use App\Response;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Log;
use SendGrid\Mail\From;
use SendGrid\Mail\Mail;
use SendGrid\Mail\To;

class PostsController extends Controller
{
        public function index() {
            $posts = Post::paginate(15);
            $data = [
                "posts" => $posts
            ];
            return view("posts", $data);
        }

        public function show($id) {
            $post = Post::findOrFail($id);
            $data = [
                "post" => $post
            ];
            return view("post", $data);
        }

        public function receiveEmailResponse(Request $request) {
            $from = $request->input("from");
            $to = $request->input("to");
            $body = $request->input("text");

            preg_match("#<(.*?)>#", $from, $sender);
            preg_match("#<(.*?)>#", $to, $recipient);
            $senderAddr = $sender[1];
            $recipientAddr = $recipient[1];

            // extract the number between "+" and "@" in the email address, this would be the post ID
            preg_match("#\+(.*?)@#", $recipientAddr, $postId);
            if ($post = Post::find((int)$postId[1])) {
                $comment = Response::create([
                    'email' => $senderAddr,
                    'post_id' => $post->id,
                    'body' => $body
                ]);
                Log::info("Create response: " . $comment->toJson(JSON_PRETTY_PRINT));
            }

            // in any case, return a 200 OK response so SendGrid knows we are done.
            return response()->json(["success" => true]);
        }

        public function sendMails($id) {
            $post = Post::findOrFail($id);

            $mails = [
                "test@example.com",
            ];
            $subject = "SG Inbound Tutorial: ".$post->title;
            $from = "replies+".$post->id."@inbound.michaelokoko.com";
            $text = "Reply to this email to leave a comment on " . $post->title;

            $mail = new Mail();
            $sender = new From($from, "SG Inbound Tutorial");
            $recipients = [];
            foreach ($mails as $addr) {
                $recipients[] = new To($addr);
            }
            $mail->setFrom($sender);
            $mail->setSubject($subject);
            $mail->addTos($recipients);
            $mail->addContent("text/plain", $text);
            $sg = new \SendGrid(getenv('SENDGRID_API_KEY'));
            try {
                $response = $sg->send($mail);
                $context = json_decode($response->body());
                if ($response->statusCode() == 202) {
                    echo "Emails have been sent out successfully!";
                }else {
                    echo "Failed to send email";
                    Log::error("Failed to send email", ["context" => $context]);
                }
            } catch (\Exception $e) {
                Log::error($e);
            }
        }
}

NOTE: Remember to include your own email address in the $mails array (in the sendMails method) and update the $from domain when testing so you can receive the email also.

Set up Views

The index and show methods in the preceding section referenced some view templates (i.e posts and post), but before implementing that, create a layout template file in resources/views to hold code and CSS styles common to both of our views.

$ touch resources/views/layout.blade.php

Paste the code block below into the layout.blade.php file:

<!DOCTYPE html>
<html lang="{{ str_replace('_', '-', app()->getLocale()) }}">
<head>
        <meta charset="utf-8">
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <title>SG Inbound</title>
        <link href="static/normalize.min.css" rel="stylesheet" />
        <link href="https://fonts.googleapis.com/css?family=Nunito:200,600" rel="stylesheet">
        <style type="text/css">
            .container {
                max-width: 48rem;
                margin-right: auto;
                margin-left: auto;
                color: #1c336a;
            }
            a,a:visited {
                color: #426bcd;
            }
            .responses {
                background-color: #e5f6f6;
                margin-top: 24px;
                padding: 12px;
            }
            .responses > div {
                padding: 12px 0;
            }
            .responses > div:not(:last-child) {
                border-bottom: 1px solid #7fd4d3;
            }
        </style>
</head>
<body>
<div class="container">
        <div class="content">
           @yield('content')
        </div>
</div>
</body>
</html>

Next, create the actual views by running the command below:

$ touch resources/views/{posts,post}.blade.php

The command uses bash expansion to create both posts.blade.php and post.blade.php at the same time. Paste the code below in To posts.blade.php to render a list of published posts.

@extends('layout')
@section('content')
@foreach($posts as $post)
        <h3 class="links">
            <a href="{{route('view-post', $post->id)}}">{{$post->title}}</a>
        </h3>
@endforeach
@endsection

Similarly, paste the following in post.blade.php to render an individual post as well as all of the responses to it.

@extends('layout')
@section('content')
<h3>
        {{$post->title}}
</h3>
<div>
        {{$post->body}}
</div>
<div class="divider"></div>
<div class="responses">
        @foreach($post->responses as $response)
            <div>
                {{$response->body}}
            </div>
        @endforeach
</div>
@endsection

Disable CSRF for Webhook Route

Laravel requires every incoming POST request to provide an extra token to protect the application from cross-site request forgery (CSRF), but we need to disable this for our webhook handler (since SendGrid won’t know what CSRF token to send). To achieve this, add the webhook route to the $except property of the VerifyCsrfToken middleware (app/Http/Middleware/VerifyCsrfToken.php).

<?php

namespace App\Http\Middleware;

use Illuminate\Foundation\Http\Middleware\VerifyCsrfToken as Middleware;

class VerifyCsrfToken extends Middleware
{
        /**
         * The URIs that should be excluded from CSRF verification.
         *
         * @var array
         */
        protected $except = [
            "/posts/receive-email-response"
        ];
}

Configure Inbound Parse

Start the Laravel server with php artisan serve and open Ngrok tunnel on the server port with ngrok http 8000. Note the generated Ngrok Forwarding URL as you will be needing it shortly.

Next, head to your Inbound Parse settings page on SendGrid and click on the “Add Host & URL button”. Select your preferred domain and specify a subdomain if applicable.

NOTE: In production, you really should use a subdomain, as SendGrid will trigger your webhook for every email that is sent to the domain (or subdomain) you provided.

Also, paste in the Ngrok Forwarding URL you copied earlier combined with the endpoint /posts/receive-email-response into the Destination URL field and save the information using the “Add” button.

Add Host & URL to SendGrid Inbound Parse

Simulate Automated Emails

With your Laravel server running, visit http://localhost:8000/posts/send-email/1 to send out emails to the specified recipients. Each reply will be processed and added as a response to the post whose ID is 1.

Conclusion

With this post, we have explored how to process inbound emails from our Laravel application using SendGrid. The complete source code is on Github and if you are looking to explore further, you could:

Lookup the official Inbound Parse Documentation
Notify your users of new blog posts with Twilio Notify
Learn how to Send Bulk Emails in PHP using Twilio SendGrid

Michael Okoko is a software engineer and computer science student at Obafemi Awolowo University, Nigeria. He loves open source and is mostly interested in Linux, Golang, PHP, and fantasy novels! You can reach him via: