Lập trình

Reliable Laravel Queues in Production: Retries, Idempotency, and Monitoring

Moving work to a queue is only the beginning. In production, workers restart during deployments, third-party APIs fail intermittently, jobs may run more than once, and incoming work may outpace processing capacity. A reliable queue must fail predictably and recover safely.

Laravel Queue trên production: Retry, idempotency và giám sát

Moving work to a queue is only the beginning. In production, workers restart during deployments, third-party APIs fail intermittently, jobs may run more than once, and incoming work may outpace processing capacity. A reliable queue must fail predictably and recover safely.

This guide focuses on operational Laravel Queue design: small payloads, idempotent jobs, controlled retries, sensible timeouts, and metrics that expose congestion before users report it.

Separate connections from queues

A connection is a backend such as Redis, Amazon SQS, or a database. One connection can hold several named queues for priority and isolation:

GenerateInvoice::dispatch($orderId)->onQueue('high');
SendCampaignEmail::dispatch($campaignId)->onQueue('bulk');

php artisan queue:work --queue=high,default,bulk

Order indicates priority, but sustained high-priority traffic can starve bulk work. Larger systems should allocate separate worker pools and resource limits.

Keep job payloads small

Pass stable identifiers and load current state in handle():

final class GenerateInvoice implements ShouldQueue
{
    use Queueable;

    public function __construct(public int $orderId) {}

    public function handle(InvoiceService $service): void
    {
        $order = Order::query()->findOrFail($this->orderId);
        $service->generateFor($order);
    }
}

Laravel serializes model identifiers, but loaded relationships may enlarge payloads and be reloaded without their original constraints. Use withoutRelations() or scalar IDs when practical.

Make every job idempotent

A job can run again after a timeout, network failure, or worker interruption. Its handler must tolerate repetition without generating duplicate documents, charges, or notifications.

Define a stable business key.
Check whether the intended result already exists.
Use a database unique constraint as the final guard.
Send an idempotency key to external services that support one.
Group related database changes in an appropriate transaction.

ShouldBeUnique reduces duplicate queued work, but it does not replace idempotent side effects.

Configure retries, backoff, and timeouts

public int $tries = 5;
public int $timeout = 120;
public bool $failOnTimeout = true;

public function backoff(): array
{
    return [10, 30, 120, 300];
}

Set the worker timeout lower than the connection's retry_after. Otherwise, a job may be delivered again while its first execution is still running. HTTP calls also need explicit connection and request timeouts.

Retry only transient failures

Network timeouts, HTTP 429, and 503 responses are often temporary. Invalid input and missing configuration usually require an immediate failure and alert. Laravel queue middleware can centralize rate limits, overlap prevention, and exception throttling.

Dispatch after database commit

A fast worker can receive a job before the transaction that created its data commits. Enable after_commit or dispatch explicitly after commit:

ProcessOrder::dispatch($order->id)->afterCommit();

Treat workers as long-running processes

queue:work keeps the application in memory and does not automatically load deployed code. Restart workers gracefully:

php artisan queue:restart

Use Supervisor or another process manager to recover crashed workers and start them after reboot. For Horizon, place php artisan horizon:terminate in the deployment flow so the process manager restarts it with the new code.

Monitor the queue, not only the process

Queue depth: pending jobs per queue.
Wait time: dispatch-to-start latency.
Throughput: completed jobs over time.
Failure rate: failures grouped by job and cause.
Runtime: p50, p95, and p99 execution time.

Laravel Horizon provides a dashboard and worker configuration for Redis queues. Alerts should focus on wait time and failure rate, not merely whether a worker process exists.

Use a disciplined failed-job workflow

Inspect the exception, stable input IDs, and correlation ID without logging secrets.
Classify the failure as transient or deterministic.
Fix the cause before a bulk retry.
Retry one representative job and verify that side effects are not duplicated.
Retry only the affected group.
Prune failed jobs according to a retention policy.