How to Use Queueing in Laravel

How to Use Queueing in Laravel

What is queuing

In the world of web applications, where user interactions trigger a multitude of actions, the efficient handling of tasks becomes a pivotal concern. Task queuing is a technique that plays a crucial role in optimising the processing of these tasks. In this tutorial, we will delve into the fundamentals of task queuing, its significance in web development, and the essential concepts associated with it.

Prerequisites

To follow along with this tutorial, you will need the following:

1. Knowledge of PHP 8.2
2. Composer installed globally
3. Redis installed and running globally
4. A Resend developer account
5. Prior Knowledge of PHP and Laravel

Introduction to task queuing in web applications

Task queuing is a method employed to manage and execute tasks asynchronously in web applications. It provides a means to decouple time-consuming or resource-intensive operations from the main application flow, allowing the application to respond quickly to user requests.

Instead of executing tasks immediately, they are placed in a queue for later processing by dedicated workers. This separation of concerns ensures that the application remains responsive, providing a smoother user experience.

The importance of queuing tasks

The importance of queuing tasks in web applications cannot be overstated. Several key reasons highlight why:

1. Improved responsiveness: Queuing tasks reduces the response time of web applications. By offloading time-consuming tasks to background workers, the application can swiftly respond to user requests, leading to a better user experience.
2. Scalability: Task queuing facilitates horizontal scalability. As the load on an application grows, you can add more workers to process queued tasks, ensuring that the application remains performant — even during peak usage.
3. Fault tolerance: Queued tasks can be retried in case of failures, ensuring the reliability of critical operations. If a task encounters an error, it can be reattempted automatically, reducing the risk of data loss or service disruptions.
4. Resource efficiency: Task queuing optimises resource utilisation. Resource-intensive tasks, such as image processing or complex database operations, can be scheduled during periods of low system load, preventing resource contention.

Key concepts

To understand task queuing in Laravel, it's essential to grasp the following key concepts:

• Jobs: A job represents a unit of work that needs to be executed asynchronously. Jobs encapsulate the task to be performed and can include data or instructions required for its execution.
• Workers: Workers are processes or threads responsible for executing queued jobs. They continuously monitor the task queue and pick up jobs for execution. Laravel provides tools to manage and scale workers efficiently.
• Queues: Queues are named channels where jobs are placed for processing. You can configure multiple queues to prioritise and organise tasks based on their importance and resource requirements.

How queuing reduces request/response time

In the realm of web application development, reducing request/response time is a constant pursuit. Responsiveness is a key metric that directly impacts user experience. To address this, developers often turn to task queuing, a technique that can significantly enhance the efficiency of web applications.

In this section, we'll explore the challenges of synchronous processing, explain how queuing mitigates these challenges, and provide examples of scenarios where queuing proves to be exceptionally beneficial.

How queuing improves application responsiveness

Task queuing addresses these challenges by introducing asynchronous processing into the application's architecture. Instead of executing tasks immediately, they are placed in a queue and processed by dedicated workers independently of the main application flow.

This decoupling of tasks from the request/response cycle offers several benefits that improve application responsiveness:

1. Faster response times: By delegating time-consuming or resource-intensive tasks to background workers, the application can respond quickly to incoming user requests. Users experience shorter wait times and a more responsive interface.
2. Parallel processing: Queuing allows multiple tasks to be processed in parallel. This parallelism can significantly accelerate the execution of tasks, especially when there are multiple cores or threads available for worker processes.
3. Resource isolation: Workers operate independently of the web server, ensuring that resource-intensive tasks do not interfere with the resources required to serve user requests. This prevents resource contention and maintains consistent performance.
4. Fault tolerance: Queued tasks can be retried in the event of failures, enhancing the robustness of the application. Failed tasks can be automatically retried, reducing the likelihood of service disruptions

How to dispatch jobs in Laravel

In Laravel, dispatching jobs is a fundamental aspect of leveraging the power of task queuing. This process allows you to define tasks as jobs and then send them off for asynchronous execution.

In this section, we will explore how to create and dispatch jobs in Laravel, and discuss the various ways you can dispatch jobs, including immediate execution and pushing them to a queue.

Step 1: Create a job class

To begin, you need to create a job class that encapsulates the task you want to execute asynchronously. Laravel provides an Artisan command (make:job) to generate a new job class:

This command would generate a new job class named ExampleJob.php in the app/Jobs directory.

Step 2: Define the job's logic

Open the newly created job class and define the logic for the job within the handle() method. This method contains the code that will be executed when the job is dispatched:

Step 3: Dispatch the job

Now that you have defined the job, you can dispatch it from any part of your application, such as a controller, route, or another job. To dispatch the job, you use the dispatch() method and pass an instance of the job class as an argument:

This code dispatches an instance of ExampleJob for asynchronous execution.

Chaining jobs in Laravel

In Laravel, job chaining is a powerful feature that allows you to create sequences of jobs executed in a synchronous order. This capability enables you to build complex workflows by breaking them down into smaller, manageable steps.

In this section, we'll delve into the concept of chaining jobs in Laravel, provide examples of scenarios where chaining is useful, and explain how to implement job chaining.

Job chaining is the process of linking multiple jobs together to form a chain of tasks. Each job in the chain is executed sequentially, meaning that the next job in the chain will only start once the previous one has completed successfully. This sequential execution ensures that complex operations can be broken down into smaller, more manageable steps.

Job chaining is particularly beneficial when you have a series of interdependent tasks that need to be executed in a specific order, such as data processing, notifications, or multi-step workflows. By chaining jobs, you can maintain a clear and structured flow of tasks within your application.

When is chaining jobs useful?

Example 1: User registration workflow

Consider a user registration workflow where you need to perform multiple tasks in a specific sequence:

1. Create a new user record in the database
2. Send a welcome email to the user
3. Log the registration activity
4. Notify an admin about the new registration

Job chaining allows you to create a chain of jobs to handle each step in this workflow, ensuring that they are executed in the correct order and with appropriate error handling.

Example 2: E-commerce order fulfilment

In an e-commerce application, when an order is placed, various tasks need to be completed before the order is considered fulfilled:

1. Reserve the ordered items in the inventory
2. Generate an invoice for the order
3. Notify the customer about the order status
4. Update the order's status in the database

By chaining these tasks, you can ensure that each step is completed before moving on to the next, maintaining consistency in the order fulfilment process.

How do you implement job chaining?

Implementing job chaining in Laravel involves creating multiple job classes and specifying the chain of jobs to be executed in a particular order. Here's a step-by-step guide.

Step 1: Create job classes

Create separate job classes for each step in your job chain using the php artisan make:job Artisan command; for example:

Step 2: Define the job's logic

In each job class, define the logic to be executed within the handle() method. Customise each job's logic according to the specific task it performs.

Step 3: Chain jobs

To chain jobs, use the withChain() method when dispatching the first job in the sequence.

Here's an example:

In this example, CreateUserJob is the first job in the chain, and it specifies the subsequent jobs in the withChain() method. Laravel will ensure that these jobs are executed in the specified order.

Step 4: Handle Errors

Implement error handling and retry mechanisms within each job to handle exceptions or failures gracefully. Laravel provides tools for handling failed jobs, such as retries, delayed retries, and custom failure strategies.

By following these steps, you can create and implement job chaining in Laravel, allowing you to build structured and reliable workflows within your application.

In the following sections, we'll explore error-handling strategies for queued jobs and dive into real-world use cases where job chaining can significantly enhance your Laravel applications.

Error handling

Error handling is a critical aspect of working with queued jobs in Laravel. When jobs are executed asynchronously, it's essential to have robust error-handling strategies in place to ensure the reliability of your application.

In this section, we will discuss five error-handling strategies for queued jobs, explain how to retry failed jobs, set up custom error-handling logic, and provide best practices for managing job failures.

Error handling strategies for queued jobs

Automatic retry on failure

Laravel allows you to specify how many times a job should be retried if it fails. This is useful for handling transient errors, such as network timeouts or temporary resource unavailability. You can configure the number of retries by adding the $tries property to your job class, such as in this example:

Delayed retries

Laravel allows you to specify a delay before retrying a failed job. This is useful for scenarios where you want to give the system time to recover or resolve the underlying issue before attempting the job again.

Custom error handling

You can implement custom error-handling logic within your job classes by defining the failed() method. This method is called when a job fails its maximum number of retry attempts. Here, you can log errors, send notifications, or take other appropriate actions.

Logging

Laravel provides comprehensive logging capabilities. You can log error messages, stack traces, and debugging information to various channels, including files, databases, or external services. Effective logging helps diagnose issues quickly, and provides insights into the root causes of job failures.

Notification

Laravel's notification system allows you to send notifications (emails, SMS, and Slack messages, etc.) when specific events occur, such as job failures. This can be useful for alerting administrators or developers when critical jobs encounter errors.

Implement an email queue system in Laravel

Project setup

First, let's set up a Laravel project usingComposer. Run the command below to create  one.

Now, move into the folder and start the application by running the commands below, sequentially.

If everything went well, you should have a live server running at http://localhost:8000 serving the Laravel template page.

Set up Redis

To be able to use Redis for the queue, we’ll need to install Predis (a Redis client for PHP), and update the config/database.php file. To install the Predis driver, in a separate terminal window or session/tab, run the command below in the project's top-level directory.

Next, we’ll add the snippet below to the list of connections (in the connections element of the returned array) in config/database.php.

Now, we can update the following settings in the .env file:

Set up the mail configuration

We’ll be using Resend for the email server, so we’ll need to add it to our application. Run the command below to do so.

Next, add the snippet below to the mailer's element in config/mail.php, to be able to use the resend in the application.

Then, add/update the mail settings below in the .env file, replacing <RESEND_API_KEY> with your  Resend API key.

Set up an email job

First, generate a new job to handle sending emails, by running the following Artisan command:

This will create a new job file in the app/Jobs directory, named SendEmail.php.

Define the email logic

Edit app/Jobs/SendEmail.php to define the logic for sending emails. Here's an example:

Dispatch the email job

In your application code (e.g., when a user registers or performs an action that triggers an email), dispatch the SendEmail job to the queue:

In this example, we're dispatching the job to a queue named "emails." You can choose a different name for your email queue if needed. You'll add that to the code, shortly.

Start the queue worker

To start processing the email queue, run the following Artisan command:

Replace emails, with the name of your email queue if it's different.

Test the application

To test the application, we’ll need to add the snippet below to the end of the routes/api.php file. It adds an API route to send the email.

Now, run this command in your terminal in a separate terminal.

You should get the following as a response.

Now, make another curl request, but this time with a valid email address, such as your own. You should see a message, similar to the example below.

Then, switch to the  first terminal, where your job is running. You should see something like the example below.

That's how to use queueing in Laravel

In this article, we went over what a task queue is, its importance in development, the different scenarios it comes in handy, and how to implement an email queue system in Laravel. We also touched on topics like task chaining and error-handling of tasks in the Laravel application. I would recommend using a task system for your next project.

Kenneth Ekandem is a full-stack developer from Nigeria currently in the blockchain space, but interested in learning everything computer science has to offer. He'd love to go to space one day and own his own vlogging channel to teach the next generation of programmers. You can reach out to him on Twitter, LinkedIn, and GitHub.

"queue" by Dako Huang is licensed under CC BY 2.0 Deed.