If you are developing large and complex projects, you have tasks that need to be run in the background. Laravel has a job mechanism for the background execution of code in a queue. Job is a class that contains code and data that will be executed in the background.
In this article I will explain how to handle jobs and queues in Laravel. I will show how to create a job class, configure a model, and a queue.
Table of Contents
Handling Jobs in Laravel
As you know, PHP does not support multi-threading. So if you want to run PHP code in parallel, you should have two or more running PHP handlers and any service that allows them to communicate between each other. First, you must run one or more queue workers. Then send a job object to the queue. Laravel will serialize this object and save it into a queue storage with job metadata. As the queue storage Laravel can use Redis or database. Next, the queue worker will extract the job object from the queue storage and deserialize it. After this it will call a handle() method from the object.
For examples from this article you need to have Laravel installed. If it is not, see this article.
1. Create a Job
For example, let’s create a TestJob class. To do this run the command below:
php artisan make:job TestJob
After this you will find the job class in the app/Jobs directory.
2. Create a Model
Usually developers use Eloquent models as data in jobs. It is pretty handy. Let’s create a model Task for this example:
php artisan make:model Task
And create migration for it:
php artisan make:migration CreateTasksTable
In this table you can store task related information, for example URL, date, any identifier and task processing status. For this example use this migration code:
Schema::create("tasks", function (Blueprint $table) {
$table->id();
$table->string("data");
$table->string("status")->default("pending");
$table->string("message")->nullable();
$table->timestamps();
});
The status field can contain a limited number of values: pending, processing, finished, failed. You may want to use an enumeration field type for this field. But it will be better when you use a simple string type. At the time of writing, Laravel does not support change enum fields in migrations. So if you want to change the field in the future, you will have to migrate it to the string or recreate it and lose all data.
Also in this example I added a message field, which you can use to store additional information about processing. For example, an exception message. Don’t forget to migrate this migration file:
php artisan migrate
3. Add Model to the Job
You can set the model as a constructor parameter and save it as a private class member. But be careful, the job doesn’t extract a model from the database, it gets the model from the queue storage, so the full model class will be stored. The eloquent model can contain its relations and their cache. It will be better when you send the model to serializing without them. Use the withoutRelations() method to achieve this. Here is a sample constructor code:
private Task $task;
public function __construct(Task $task)
{
$this->task = $task->withoutRelations();
}
2. Configure the created job
Using job class variables, you can configure job behavior. For example, you can configure, timeout of job execution, queue name, retry count, delay between retries and more. Let’s see examples. By default your job will be executed in only 30 seconds. When this time runs out the job will be killed and moved to failed jobs. You can increase timeout using the $timeout variable in the job class. To completely remove the limit, set the value to 0.
public $timeout = 0;
By default the job will be placed in a queue with the name “default”. If you want to use another queue, specify the queue name in the $queue variable:
public $queue = 'example_queue_name';
When an error occurs while executing a job, the job is immediately marked as failed. But Laravel can retry the job. You can specify the number of retries in the $retries variable:
public $retries = 3;
Furthermore, you may want to add delay between retries. Use the $backoff class variable to do this. Set time in seconds to this variable:
public $backoff = 60;
3. Configure queue
By default, queue jobs are executed synchronously. You should select a queue storage driver to activate asynchronous execution. As I said before, you can use different drivers to store queued jobs. But the most popular of them is Redis. You should have Redis configured in your system and a Redis PHP extension installed. Then check Redis variables in the .env file:
After this change the variable QUEUE_CONNECTION variable from sync to redis:
QUEUE_CONNECTION=redis
4. Overlapping
You can add to queue two equal jobs to a queue by mistake, which will duplicate each other. It may be critical when jobs manage sensitive data, for example change user balance or buy something. To avoid it you can use WithoutOverlapping middleware. It works only with Redis and can prevent duplicating jobs by a specified key. For example, you can use model ID as the key.
If someone tries to add two jobs to a queue with identical models, the second job with this middleware will be moved to failed jobs. Here is the code example of using the middleware:
public function middleware(): array
{
$middleware = new WithoutOverlapping($this->task->getKey());
return [$middleware];
}
You can specify when the queue handler should release this lock using releaseAfter() method. Or you can use the dontRelease() method if you want to add the lock permanently.
3. Add job to the queue
There are two ways to push a job into the queue. You can add еру DispatchesJobs trait to your class and use dispatch() method. Also, you can get Illuminate\Bus\Dispatcher from the container and use the dispatch() method from it. For this example you can create an Artisan command that will add a few jobs to the queue.
First, create a new model:
$task = new Task();
$task->data = "test";
$task->status = "pending";
$task->save();
After this create a new job and set the model to its constructor:
$job = new TestJob($task);
Finally, push the job into the queue. First approach looks like this:
use Illuminate\Foundation\Bus\DispatchesJobs;
class Foo
{
use DispatchesJobs;
public function handle()
{
//.....
$this->dispatch($job);
}
}
Another approach is more difficult, but perhaps better:
use Illuminate\Bus\Dispatcher;
class Foo
{
private Dispatcher $dispatcher;
public function __construct(Dispatcher $dispatcher)
{
$this->dispatcher = $dispatcher;
}
public function handle()
{
//......
$this->dispatcher->dispatch($job);
}
}
You can make sure that the job was added to the queue using RESP.app. It’s free. You can install it using a snap in Ubuntu:
snap install redis-desktop-manager
Just connect to your Redis server, open the default queue and find the job:
4. Run queue handler
You can run the queue handler using the Artisan command queue:work. You can run multiple processes of this command if you want your jobs to run in parallel:
php artisan queue:work
By default it will pick jobs from the default queue. If you want to use another queue use –queue option. For example:
php artisan queue:work --queue example_queue_name
Also you can specify the connection that the handler should use:
php artisan queue:work --queue example_queue_name redis
If you want all jobs to be handled without a timeout, you can change it to 0 using –timeout option:
php artisan queue:work --timeout 0
It is handy for development, but if you want to use queues in production, you can run queue:work processes using Supervisor. Here is an example of supervisor config for one worker:
[program:queue-worker-run]
process_name=%(program_name)s_%(process_num)02d
command= php artisan queue:work
directory = /var/www
autostart=true
autorestart=true
numprocs=1
user=www-data
5. Handle Exceptions
If any part of the code from a job throws an exception and nobody catches it the job fails. If you want to do any actions when job fails, for example change processing status to failed and log exception message, you can add the failed method:
public function failed(\Throwable $e)
{
$this->task->status = "failed";
$this->task->message = $e->getMessage();
$this->task->save();
}
It does not prevent job failure, but allows you to know about it.
Wrapping Up
In this article I have explained how to handle Laravel jobs and queues, how to create them, add them to a queue and run processing. There are several ways to do the same things, which may make job processing difficult for beginners. But jobs and queues are an important part of Laravel.
