yii2 queue

Posted May 26, 20206 min read

Worker starting control

\ []([ https://github.com/yiisoft/yi ...]( https://github.com/yiisoft/yii2-queue/blob/master/docs/guide-zh-CN/worker . md # supervisor) Supervisor)


Supervisor is a Linux process monitor.
It will automatically start your console process.
To install on Ubuntu, you need to run the command:

sudo apt-get install supervisor

Supervisor configuration files are usually available in /etc/supervisor/conf.d. You can create any number of configuration files.

Configuration example:

[program:yii-queue-worker]
process_name =%(program_name) s _%(process_num) 02d
command =/usr/bin/php/var/www/my_project/yii queue/listen --verbose = 1 --color = 0
autostart = true
autorestart = true
user = www-data
numprocs = 4
redirect_stderr = true
stdout_logfile =/var/www/my_project/log/yii-queue-worker.log

In this case, Supervisor will start 4 queue/listen workers. The output will be written to the corresponding log file.

For more information on the configuration and use of Supervisor, please refer to Documentation .

Workers started in daemon mode are supported using the queue/listen command File , Db , [Redis]( https://github.com /yiisoft/yii2-queue/blob/master/docs/guide-zh-CN/driver-redis.md), RabbitMQ , Beanstalk , Gearman driver. For other parameters, see the driver guide.

\ []([ https://github.com/yiisoft/yi ...]( https://github.com/yiisoft/yii2-queue/blob/master/docs/guide-zh-CN/worker . md # systemd) Systemd)


Systemd is an init system used in Linux to bootstrap the user space. To configure workers startup using systemd, create a config file named yii-queue @ .service in/etc/systemd/system with the following contents:

[Unit]
Description = Yii Queue Worker%I
After = network.target
# the following two lines only apply if your queue backend is mysql
# replace this with the service that powers your backend
After = mysql.service
Requires = mysql.service

[Service]
User = www-data
Group = www-data
ExecStart =/usr/bin/php/var/www/my_project/yii queue/listen --verbose
Restart = on-failure

[Install]
WantedBy = multi-user.target

You need to reload systemd in order for it to re-read configuration:

systemctl daemon-reload

Set of commands to control workers:

To start two workers

systemctl start yii-queue @ 1 yii-queue @ 2

To get status of running workers

systemctl status "yii-queue @ \ *"

To stop the worker

systemctl stop yii-queue @ 2

To stop all running workers

systemctl stop "yii-queue @ \ *"

To start two workers at system boot

systemctl enable yii-queue @ 1 yii-queue @ 2

To learn all features of systemd, check its documentation .

\ []([ https://github.com/yiisoft/yi ...]( https://github.com/yiisoft/yii2-queue/blob/master/docs/guide-zh-CN/worker . md # cron) Cron)


You can start the worker with cron. The queue/run command is required. As long as the queue contains jobs, it can be executed.

Configuration example:

\ * \ * \ * \ * \ */usr/bin/php/var/www/my \ _project/yii queue/run

In this case, cron will start the command every minute.

queue/run command support File , Db , [Redis]( https://github.com/yiisoft/yii2-queue/blob /master/docs/guide-zh-CN/driver-redis.md), [Beanstalk]( https://github.com/yiisoft/yii2-queue/blob/master/docs/guide-zh-CN/driver- beanstalk.md), Gearman driver. Refer to the driver guide for other options.

Errors and repeated execution

An exception can be thrown during job processing. When the requested service and external resources are unavailable, internal errors are caused by poorly written code.
In the second case, the job can be retried after a period of time.

There are several ways to do this.

\ []([ https://github.com/yiisoft/yi ...]( https://github.com/yiisoft/yii2-queue/blob/master/docs/guide-zh-CN/retryable . md #%E9%87%8D%E8%AF%95%E9%80%89%E9%A1%B9)%E9%87%8D%E8%AF%95%E9%80%89%E9%A1%B9)


The first method is implemented by component options:

'components' => \ [

'queue' => \ [
    'class' => \\ yii \\ queue \\ <driver> \\ Queue ::class,
    'ttr' => 5 \ * 60, //Max time for anything job handling
    'attempts' => 3, //Max number of attempts
\],

],

The ttr option sets the time to keep work in the queue. If a job is not executed during this time, it will return to the queue and try again.
The attempts option sets the maximum number of attempts. If the attempt has ended and the job has not been completed, it will be removed from the queue.

This will set all the jobs in the queue globally. If you need to make different settings for multiple jobs, you can use the second method.

\ []([ https://github.com/yiisoft/yi ...]( https://github.com/yiisoft/yii2-queue/blob/master/docs/guide-zh-CN/retryable . md #%E9%87%8D%E8%AF%95%E4%BD%9C%E4%B8%9A%E6%8E%A5%E5%8F%A3)%E9%87%8D%E8%AF%95%E4%BD%9C%E4%B8%9A%E6%8E%A5%E5%8F%A3)


Separate control of retry is implemented by RetryableJobInterface interface. Example:

class SomeJob extends BaseObject implements RetryableJobInterface
{

public function execute($queue)
{
    //...
}

public function getTtr()
{
    return 15 \ * 60;
}

public function canRetry($attempt, $error)
{
    return($attempt <5) &&($error instanceof TemporaryException);
}

}

The getTtr() and canRetry() methods have higher priority than component configuration.

\ []([ https://github.com/yiisoft/yi ...]( https://github.com/yiisoft/yii2-queue/blob/master/docs/guide-zh-CN/retryable . md #%E4%BA%8B%E4%BB%B6%E5%A4%84%E7%90%86)%E4%BA%8B%E4%BB%B6%E5%A4%84%E7%90%86)


The third method to set TTR and jobs that need to retry failure includes using Queue ::EVENT_BEFORE_PUSH and Queue ::EVENT_AFTER_ERROR events.

The Queue ::EVENT_BEFORE_PUSH event can be used to set the TTR:

Yii ::$app \-> queue \-> on(Queue ::EVENT \\ _ BEFORE \\ _ PUSH, function(PushEvent $event) {

if($event \-> job instanceof SomeJob) {
    $event \-> ttr = 300;
}

});

And the Queue ::EVENT_AFTER_ERROR event can be used to set up new attempts:

Yii ::$app \-> queue \-> on(Queue ::EVENT \\ _ AFTER \\ _ ERROR, function(ExecEvent $event) {

if($event \-> job instanceof SomeJob) {
    $event \-> retry =($event \-> attempt <5) &&($event \-> error instanceof TemporaryException);
}

});

The event handler is executed after the RetryableJobInterface method, so it has the highest priority.

\ []([ https://github.com/yiisoft/yi ...]( https://github.com/yiisoft/yii2-queue/blob/master/docs/guide-zh-CN/retryable . md #%E9%99%90%E5%88%B6)%E9%99%90%E5%88%B6)


Fully support Beanstalk , [DB]( https://github.com /yiisoft/yii2-queue/blob/master/docs/guide-zh-CN/driver-db.md), File and Redis driver Program retry tool. Sync The driver will not retry the failed job Gearman Retry is not supported. RabbitMQ Basic version support, but retry number cannot be obtained.