The article takes you to understand the operating mechanism of Laravel schedule.

This article takes you to talk about the operation mechanism of schedule scheduling in Laravel, hoping to help you!

Laravel's console command line greatly facilitates the setting and running of PHP timing tasks. In the past, the process of configuring timing tasks through crontab was relatively complicated, and it was difficult to prevent the overlapping operation of tasks by setting timing tasks through crontab.

The so-called overlapping operation of tasks means that due to the long running time of scheduled tasks and the unreasonable running cycle set by crontab, the started tasks have not finished running, and the system has started new tasks to perform the same operation. If the problem of data consistency is not handled properly in the program, then two tasks operate the same data at the same time, which is likely to lead to serious consequences. 1. Runinbackground and withoutOverlapping In order to prevent tasks from overlapping, Laravel provides the withoutOverlapping () method; Laravel provides the runInBackground () method to enable multi-tasks to be executed in parallel in the background.

(1) Runinbackground () method Every command in the console command line represents an Event, and the function of the schedule () method in AppConsoleKernel is only to register the events represented by these command lines in the attribute $events of Illuminate console scheduling schedule.

// namespace IlluminateConsoleSchedulingSchedule public function command($command, array $parameters = []) { if (class_exists($command)) { $command = Container::getInstance()->make($command)->getName(); } return $this->exec( Application::formatCommandString($command), $parameters ); } public function exec($command, array $parameters = []) { if (count($parameters)) { $command .= ' '.$this->compileParameters($parameters); } $this->events[] = $event = new Event($this->eventMutex, $command, $this->timezone); return $event; }

Event runs in two ways: Foreground and Background. The difference between the two is whether multiple Event can be executed in parallel. Events run in the Foreground mode by default. In this mode, multiple Events are executed in sequence, and the subsequent events cannot be executed until the previous event is completed.

But in practical application, we often hope that multiple Events can be executed in parallel. At this time, we need to call the runInBackground () method of an event to set its running mode to Background.

The Laravel framework's handling of these two running modes differs in the way the command line is assembled and the way the callback method is called.

// namespace IlluminateConsoleSchedulingEvent protected function runCommandInForeground(Container $container) { $this->callBeforeCallbacks($container); $this->exitCode = Process::fromShellCommandline($this->buildCommand(), base_path(), null, null, null)->run(); $this->callAfterCallbacks($container); } protected function runCommandInBackground(Container $container) { $this->callBeforeCallbacks($container); Process::fromShellCommandline($this->buildCommand(), base_path(), null, null, null)->run(); } public function buildCommand() { return (new CommandBuilder)->buildCommand($this); } // namespace IlluminateConsoleSchedulingCommandBuilder public function buildCommand(Event $event) { if ($event->runInBackground) { return $this->buildBackgroundCommand($event); } return $this->buildForegroundCommand($event); } protected function buildForegroundCommand(Event $event) { $output = ProcessUtils::escapeArgument($event->output); return $this->ensureCorrectUser( $event, $event->command.($event->shouldAppendOutput ? ' >> ' : ' > ').$output.' 2>&1' ); } protected function buildBackgroundCommand(Event $event) { $output = ProcessUtils::escapeArgument($event->output); $redirect = $event->shouldAppendOutput ? ' >> ' : ' > '; $finished = Application::formatCommandString('schedule:finish').' "'.$event->mutexName().'"'; if (windows_os()) { return 'start /b cmd /c "('.$event->command.' & '.$finished.' "%errorlevel%")'.$redirect.$output.' 2>&1"'; } return $this->ensureCorrectUser($event, '('.$event->command.$redirect.$output.' 2>&1 ; ' .$finished.' "$?" ) > ' .ProcessUtils::escapeArgument($event->getDefaultOutput()).' 2>&1 &' ); }

As can be seen from the code, an ampersand will be added at the end of the command line of the Event running in the Background mode, which is used to make the command line program run in the background; In addition, the callback method of an Event running in the Foreground mode is called synchronously, while the subsequent callback of an Event running in the Background mode is executed through the schedule:finish command line.

⑵ When setting the running cycle of an Event with the method without overlapping (), it is difficult to avoid that a specific Event needs to run for a long time to complete in a certain period of time, even when the next running cycle starts, because the application scenarios are constantly changing. If this situation is not handled, it will lead to multiple identical events running at the same time, and if these events involve the operation of data and the idempotent problem is not handled well in the program, it is likely to cause serious consequences.

In order to avoid the above problems, the Event provides the withoutOverlapping () method. By setting the withoutOverlapping property of the Event to TRUE, this method will check whether there is the same Event currently being executed every time the Event is to be executed, and if there is, it will not execute a new Event task.

// namespace IlluminateConsoleSchedulingEvent public function withoutOverlapping($expiresAt = 1440) { $this->withoutOverlapping = true; $this->expiresAt = $expiresAt; return $this->then(function () { $this->mutex->forget($this); })->skip(function () { return $this->mutex->exists($this); }); } public function run(Container $container) { if ($this->withoutOverlapping && ! $this->mutex->create($this)) { return; } $this->runInBackground ? $this->runCommandInBackground($container) : $this->runCommandInForeground($container); }

When mutex mutex calls withoutOverlapping (), this method also realizes two other functions: one is to set the timeout, which is 24 hours by default; The other is to set the callback of the Event.

(1) Timeout First, time out. This time out is not the time out of the Event, but the time out of the attribute mutex of the Event. When registering an Event in the attribute $Events of Illuminate Console Scheduling Schedule, the exec () method in the schedule will be called, and a new Event object will be created in this method. At this time, an EventMutex will be passed into the constructor of the event, which is the attribute mutex in the event object, and the timeout is set for this mutex. EventMutex in Schedule is created by instantiating CacheEventMutex.

// namespace IlluminateConsoleSchedulingSchedule $this->eventMutex = $container->bound(EventMutex::class) ? $container->make(EventMutex::class) : $container->make(CacheEventMutex::class);

An Event with withoutOverlapping set will first try to obtain mutex mutex lock before execution. If the lock cannot be obtained successfully, then the Event will not be executed. The operation of obtaining the mutex is completed by calling the create () method of mutex.

When the CacheEventMutex is instantiated, it needs to pass in an instance of type Illuminate Contracts Cache Factory, and it finally passes in an instance of IlluminateCacheCacheManager. When calling the create () method to obtain the mutex, you also need to set the storage engine by calling the store () method.

// namespace IlluminateFoundationConsoleKernel protected function defineConsoleSchedule() { $this->app->singleton(Schedule::class, function ($app) { return tap(new Schedule($this->scheduleTimezone()), function ($schedule) { $this->schedule($schedule->useCache($this->scheduleCache())); }); }); } protected function scheduleCache() { return Env::get('SCHEDULE_CACHE_DRIVER'); } // namespace IlluminateConsoleSchedulingSchedule public function useCache($store) { if ($this->eventMutex instanceof CacheEventMutex) { $this->eventMutex->useStore($store); } /* ... ... */ return $this; } // namespace IlluminateConsoleSchedulingCacheEventMutex public function create(Event $event) { return $this->cache->store($this->store)->add( $event->mutexName(), true, $event->expiresAt * 60 ); } // namespace IlluminateCacheCacheManager public function store($name = null) { $name = $name ? : $this->getDefaultDriver(); return $this->stores[$name] = $this->get($name); } public function getDefaultDriver() { return $this->app['config']['cache.default']; } protected function get($name) { return $this->stores[$name] ? ? $this->resolve($name); } protected function resolve($name) { $config = $this->getConfig($name); if (is_null($config)) { throw new InvalidArgumentException("Cache store [{$name}] is not defined."); } if (isset($this->customCreators[$config['driver']])) { return $this->callCustomCreator($config); } else { $driverMethod = 'create'.ucfirst($config['driver']).'Driver'; if (method_exists($this, $driverMethod)) { return $this->{$driverMethod}($config); } else { throw new InvalidArgumentException("Driver [{$config['driver']}] is not supported."); } } } protected function getConfig($name) { return $this->app['config']["cache.stores.{$name}"]; } protected function createFileDriver(array $config) { return $this->repository(new FileStore($this->app['files'], $config['path'], $config['permission'] ? ? null)); }

When initializing the Schedule, the storage engine of eventMutex will be specified, which defaults to the value of the configuration item SCHEDULE_CACHE_DRIVER in the environment variable. But usually this configuration does not exist in the environment variable, so the parameter value of useCache () is empty, and the store property value of eventMutex is also empty. In this way, when the store () method is called in the create () method of eventMutex to set the storage engine for it, the parameter value of the store () method is also empty.

When the pass parameter of the store () method is empty, the default storage engine of the application will be used (if no modification is made, the storage engine of the default cache is file). After that, the configuration information of the default storage engine (engine, storage path, connection information, etc.) will be obtained, and then the storage engine will be instantiated. Finally, the file storage engine instantiates IlluminateCacheFileStore.

After the storage engine is set up, the add () method will be called to obtain the mutex lock. Because the store () method returns an instance of type Illuminate Contracts Cache Repository, the add () method in IlluminateCacheRepository is finally called.

// namespace IlluminateCacheRepository public function add($key, $value, $ttl = null) { if ($ttl ! == null) { if ($this->getSeconds($ttl) store, 'add')) { $seconds = $this->getSeconds($ttl); return $this->store->add( $this->itemKey($key), $value, $seconds ); } } if (is_null($this->get($key))) { return $this->put($key, $value, $ttl); } return false; } public function get($key, $default = null) { if (is_array($key)) { return $this->many($key); } $value = $this->store->get($this->itemKey($key)); if (is_null($value)) { $this->event(new CacheMissed($key)); $value = value($default); } else { $this->event(new CacheHit($key, $value)); } return $value; } // namespace IlluminateCacheFileStore public function get($key) { return $this->getPayload($key)['data'] ? ? null; } protected function getPayload($key) { $path = $this->path($key); try { $expire = substr( $contents = $this->files->get($path, true), 0, 10 ); } catch (Exception $e) { return $this->emptyPayload(); } if ($this->currentTime() >= $expire) { $this->forget($key); return $this->emptyPayload(); } try { $data = unserialize(substr($contents, 10)); } catch (Exception $e) { $this->forget($key); return $this->emptyPayload(); } $time = $expire - $this->currentTime(); return compact('data', 'time'); }

It needs to be explained here that the essence of the so-called mutex is to write files. If the file does not exist, or the file content is empty, or the expiration time stored in the file is less than the current time, the mutex can be obtained smoothly; Otherwise, the mutex cannot be obtained. The file content is in a fixed format: timestampb:1.

The so-called timeout is closely related to the value of timestamp here. The timestamp when the mutex was obtained, plus the number of seconds of timeout, is the value of timestamp here. Because the add () method does not exist in the FileStore, the program will directly try to call the get () method to get the contents of the file. If the result returned by get () is NULL, it means that the mutex is successfully obtained, and then the put () method of FileStore will be called to write the file; Otherwise, it means that the same Event is currently running, and no new Event will be run.

When calling the put () method to write a file, first you need to calculate the seconds of the timeout of the eventMutex according to the passed parameters, and then call the put () method in the FileStore to write the data into the file.

// namespace IlluminateCacheRepository public function put($key, $value, $ttl = null) { /* ... ... */ $seconds = $this->getSeconds($ttl); if ($seconds forget($key); } $result = $this->store->put($this->itemKey($key), $value, $seconds); if ($result) { $this->event(new KeyWritten($key, $value, $seconds)); } return $result; } // namespace IlluminateCacheFileStore public function put($key, $value, $seconds) { $this->ensureCacheDirectoryExists($path = $this->path($key)); $result = $this->files->put( $path, $this->expiration($seconds).serialize($value), true ); if ($result ! == false && $result > 0) { $this->ensureFileHasCorrectPermissions($path); return true; } return false; } protected function path($key) { $parts = array_slice(str_split($hash = sha1($key), 2), 0, 2); return $this->directory.'/'.implode('/', $parts).'/'.$hash; } // namespace IlluminateConsoleSchedulingSchedule public function mutexName() { return 'framework'.DIRECTORY_SEPARATOR.'schedule-'.sha1($this->expression.$this->command); }

What needs to be emphasized here is the generation method of $key and the generation method of file path. The $key is generated by calling the mutexName () method of the Event, in which the $expression and $command properties of the Event are needed. Among them, $command is the command line defined by us, which is passed in when calling the $schedule->comand () method and then formatted, and $expression is the running cycle of the Event.

Take the command line schedule:test as an example. After formatting, the command behavior is `/usr/local/PHP/bin/PHP```artificial' schedule: test. If the running cycle set by the command line is once every minute, that is, * * * *, the final calculated value of $key is framework/schedule-768a42da74f005b3ac29. The file path is to divide the value of $key into arrays by two characters after sha1 calculation again, and then take the first two items of the array to form a secondary directory, and the default storage path of the file engine in the configuration file is storage/framework/cache/data. So the final file path is storage/framework/cache/data/EB/60/EB608Bf555895F742E5B D57E186CBD 97F9A6F432. The content stored in the file is 1642122685b:1.

(2) Callback method Let's talk about the set Event callback. Calling the withoutOverlapping () method will set two callbacks for the Event: one is the callback after the Event runs, which is used to release the mutex, that is, to clean up the cache file; The other is to judge whether the mutex is occupied before running the Event, that is, whether the cache file already exists.

No matter whether the Event runs in the Foreground or the Background, the callAfterCallbacks () method will be called to execute the callback in afterCallbacks after the event is finished, and one callback is used to release the mutex and delete the cache file $this->mutex->forget($this). The difference is that an Event running in the Foreground mode explicitly calls these callback methods after running, while an Event running in the Background mode needs to call these callback methods with the help of schedule:finish.

All Event registered in AppConsoleKernel are scheduled through the command line schedule:run. Before scheduling, it will first judge whether the current time point meets the requirements of the running cycle configured by each Event. If so, the next step is to judge some filter conditions, including judging whether the mutex is occupied. Event can only run if the mutex is not occupied.

// namespace IlluminateConsoleSchedulingScheduleRunCommand public function handle(Schedule $schedule, Dispatcher $dispatcher) { $this->schedule = $schedule; $this->dispatcher = $dispatcher; foreach ($this->schedule->dueEvents($this->laravel) as $event) { if (! $event->filtersPass($this->laravel)) { $this->dispatcher->dispatch(new ScheduledTaskSkipped($event)); continue; } if ($event->onOneServer) { $this->runSingleServerEvent($event); } else { $this->runEvent($event); } $this->eventsRan = true; } if (! $this->eventsRan) { $this->info('No scheduled commands are ready to run.'); } } // namespace IlluminateConsoleSchedulingSchedule public function dueEvents($app) { return collect($this->events)->filter->isDue($app); } // namespace IlluminateConsoleSchedulingEvent public function isDue($app) { /* ... ... */ return $this->expressionPasses() && $this->runsInEnvironment($app->environment()); } protected function expressionPasses() { $date = Carbon::now(); /* ... ... */ return CronExpression::factory($this->expression)->isDue($date->toDateTimeString()); } // namespace CronCronExpression public function isDue($currentTime = 'now', $timeZone = null) { /* ... ... */ try { return $this->getNextRunDate($currentTime, 0, true)->getTimestamp() === $currentTime->getTimestamp(); } catch (Exception $e) { return false; } } public function getNextRunDate($currentTime = 'now', $nth = 0, $allowCurrentDate = false, $timeZone = null) { return $this->getRunDate($currentTime, $nth, false, $allowCurrentDate, $timeZone); }

Sometimes, we may need to kill some command lines running in the background, but then we will find that these command lines that have been killed cannot be automatically scheduled according to the set running cycle for a period of time. The reason is that the command lines that have been manually killed have not called schedule:finish to clean up the cache file and release the mutex lock. This will cause the mutex to be occupied until the set expiration time, and the new Event will not run again. [Related recommendation: laravel video tutorial]