Persistent processes: queue workers, Horizon and always-on commands

Some applications need a program that never ends: a worker that processes the queue as soon as a job arrives, Horizon, a message consumer or a bot. A cron job starts, does its work and finishes; a persistent process stays running and, if it dies, starts again on its own.

They are available on the Avanza, Max and Elástico plans:

Inicia, Webmaster and the accounts in a multi-account hosting have no persistent processes. Queues can still be processed with the scheduler or a cron job, with up to a minute's wait: we explain this in Install Laravel and in cron jobs.

Which type to choose

• Queue worker: runs php artisan queue:work with the queues you specify. For Laravel jobs in the database or in Redis, without Horizon.
• Horizon: runs php artisan horizon. For Redis queues managed by Horizon, with its dashboard and metrics.
• Persistent Redis: Valkey 8, compatible with Redis, for queues, sessions and Horizon. It is turned on from the Package manager.
• Custom command: a single shell line. For Symfony Messenger, message consumers, bots or any program that does not accept connections.

With Horizon, do not also add a queue worker for the same queues: Horizon starts and distributes its own workers. Horizon needs Redis queues, and that is what the persistent Redis is for: it takes up one of the plan's processes.

A persistent process is for background work: queues, Horizon, the persistent Redis, which connects over a socket, or your own script. It is not for servers that accept connections: we do not support programs that listen on a port, such as a Node.js server, a Python application, Laravel Reverb or another WebSockets server. For WebSockets in Laravel, use a Pusher-compatible service, as we explain in WebSockets in Laravel; if your project needs its own server, you need a VPS.

Create a process

1. In the client area, open Hosting, go into your service and go to Tools > Processes.
2. In the Persistent processes block, click Add process and choose the type.
3. Enter the application folder, relative to your home and without /home/usuario. If you deploy with Deploy+, it is deploy/miapp/current, with your repository's name instead of miapp; if you installed the application by hand, the project folder, for example mi-proyecto. For a custom command it is optional: without a folder, it starts in your home.
4. For the queue worker and Horizon, choose the PHP version: 8.1 or later. The process uses that version's binary, for example /opt/alt/php84/usr/bin/php for PHP 8.4, and not the php you find in the terminal, which may be another one. Choose the same one your website uses and, in that version, turn on the extensions the application needs from the PHP selector.
5. Save. The process appears as Starting and changes to Running in less than a minute.

Queue worker

Enter the queues separated by commas, in order of priority: with high,default, the worker empties high first. Unless you say otherwise, it processes default with the QUEUE_CONNECTION connection from your .env. These are the default options:

php artisan queue:work --queue=default --sleep=3 --tries=3 --max-time=3600 --memory=256

• --sleep=3: when the queue is empty, it waits 3 seconds before checking again.
• --tries=3: three attempts per job before marking it as failed. Failed jobs stay in failed_jobs; you can see them with php artisan queue:failed and retry them with php artisan queue:retry.
• --max-time=3600: after an hour, the worker exits and starts again within a second. This frees memory and picks up new code even if nobody restarts it.
• --memory=256: if it goes over 256 MB, it exits after finishing the current job and starts again.

For high-volume queues you can create several workers on the same queues, one per plan process. Start with one and add another only if jobs pile up.

Custom command

Write a single line, as you would type it in the terminal, with absolute paths for PHP and for anything that is not in the chosen folder. For example, a Symfony Messenger consumer:

/opt/alt/php84/usr/bin/php bin/console messenger:consume async --time-limit=3600 --memory-limit=256M

A custom command can launch a Python script, or one in another language available in the account, for a background job, but we only officially support PHP applications: anything that is not PHP runs at your own risk.

The program has to stay in the foreground. Do not add &, nohup or options that send it to the background: the watchdog would see it exit straight away, launch it again and it would end up Failed with several stray copies. As with --max-time in Laravel, it is a good idea for the program to exit on its own every so often to free memory: it starts again by itself.

How it stays running

• A watchdog checks the account's processes every minute and starts any that are down. You will see it in your cron jobs as a line ending in # hostsuar-procesos: do not edit or delete it, it is managed from Processes.
• If the program exits, because of --max-time, memory or after a php artisan queue:restart, it starts again within a second. It enters the folder again on every start, so after a restart it picks up the version published at that moment.
• If it dies 5 times in a row right after starting, before reaching 30 seconds, it stays Failed and is not tried again until you restart it from the panel. Between one attempt and the next it waits a little longer each time, so as not to spend the account's CPU on a program that cannot start.
• It runs as your account's user, within its resources: the plan's CPU, memory and number of processes are the same ones your websites use. If the account reaches its memory limit, the system may kill the process and the watchdog starts it again. Check usage in CPU and resource usage.

Statuses and actions

• Running: the program has been alive for a few seconds without dying.
• Starting: it has been requested and is not yet confirmed alive, or it has just started; the watchdog starts it in less than a minute.
• Retrying: it is dying right after starting and the watchdog is launching it again. If it dies 5 times in a row, it stays Failed. The reason is in its log.
• Stopped: you have stopped it and it does not start on its own.
• Failed: it has died 5 times in a row on start-up and is waiting for you to restart it.

For each process you can:

• Restart: stops it and starts it immediately, even if it was Failed.
• Stop and Start: once stopped it is not launched again, but it stays in the list and keeps its configuration.
• View log: the last lines the program has written, with an arranca line (starting) on every start and a terminó con el código N tras S s line (exited with code N after S s) every time it exits. When the log goes over 1 MB, it is trimmed and keeps the most recent part.
• Edit: when you save, the process restarts with the changes.
• Delete: stops it and removes it from the account.

In Laravel, job errors are also written to storage/logs/laravel.log, and failed jobs to failed_jobs.

If a process stays "Failed"

Open its log: almost always the last line before each terminó con el código explains the cause. The most common are:

• «no existe la carpeta de trabajo» (the working folder does not exist): the folder is wrong. Remember it is relative to your home, without /home/usuario.
• A PHP extension is missing, such as redis, or pcntl and posix for Horizon: turn it on in the process's PHP version, which may not be your website's.
• The application cannot connect to Redis: the .env points to a Redis that is not active or to another socket.
• A custom command that goes into the background or that exits straight away because it has nothing to do.
• An error when starting the application, for example after a deployment with a missing dependency.

Fix the cause and click Restart. While it stays Failed, it does not use resources.

Deployments and persistent processes

A worker loads the code when it starts and keeps it in memory, so it has to restart to use a new version.

• With Deploy+ you do not have to do anything: as soon as each deployment finishes, and also when rolling back to a previous version, it restarts the persistent processes whose folder is that application's current or a folder inside it. Each worker finishes the current job and starts again within a second with the new code, and the deployment log shows how each one ended up. Do not put queue:restart or horizon:terminate in the script: it runs before the version is activated and the worker would start again on the previous one. From the repository page in Deploy+ you can add that application's queue worker with the folder already filled in.
• If you publish some other way (Git by hand, FTP), click Restart on the process when you finish, or run php artisan queue:restart, php artisan horizon:terminate or php bin/console messenger:stop-workers once the new code is in place. Laravel notifies the workers through its cache: the command and the workers must use the same CACHE_STORE.

SSH while there are processes

While the account has persistent processes, even stopped ones, SSH access stays enabled and cannot be disabled: turning it off would close all the account's processes. If you want to disable SSH, delete the persistent processes first. The processes do not need you to open any session.

Change plan

• To a plan with fewer processes: the most recent ones that do not fit are stopped, and the persistent Redis memory is adjusted to the new plan's maximum.
• To a plan without persistent processes: they are all stopped and stay in the list, so you can see what you had. If you go back to Avanza, Max or Elástico, you can start them again.

Before you change plan, prepare the application to process its queues without them.