Finished Scheduler Disciplines feature implementation, test fixes, do…

…cumentation improvements

README.md

@@ -144,10 +144,20 @@ Scheduler is responsible for:
 
 - running _Server Services_ in parallel using __preemptive multitasking__
 - supporting custom _Multi-Processing Modules_
-- managing the number of processes based on a Task-Pool strategy
+- managing the number of processes based on a Task-Pool strategy and Scheduler Disciplines
 - handling Process lifecycle
 - keeping track of and reporting Process state
 
+#### Scheduler Disciplines
+
+In ZEUS, scheduling disciplines are algorithms used for distributing hardware resources (such as CPU time or memory) among Scheduler processes. 
+
+The main purpose of scheduling algorithms is to minimize resource starvation by creating or termination of processes to keep number of active processes within the boundaries specified by Scheduler configuration. 
+Some algorithms may focus on termination of processes that were idle for too long, while other may terminate processes based on their actual memory footprint or a number of requests that they already processed.
+
+> Since version 1.3.4, Schedulers can be configured to use custom `\Zeus\Kernel\ProcessManager\Scheduler\Discipline\DisciplineInterface` implementations. 
+> **If no such implementation is specified, ZEUS defaults to a built-in _LRU (Least Recently Used) Discipline_.**
+
 #### Multi-Processing Modules
 
 Certain multitasking architectures are incompatible or not efficient enough on different operating systems. To remedy this issue, ZEUS provides a Multi-Processing Module interface between an application and the underlying operating system that is designed to hide these differences by providing a consistent platform on which the application is run. 
@@ -243,11 +253,15 @@ Multiple _Process Schedulers_ can be configured in a regular Zend Framework 3 co
 ```php
 // contents of "zf3-application-directory/config/some-config.config.php" file:
 
+use Zeus\Kernel\ProcessManager\MultiProcessingModule\PosixProcess;
+use Zeus\Kernel\ProcessManager\Scheduler\Discipline\LruDiscipline;
+
 return [
     'zeus_process_manager' => [
         'schedulers' => [
             'scheduler_1' => [
                 'scheduler_name' => 'sample_scheduler',
+                'scheduler_discipline' => LruDiscipline::class, // choice available since version 1.3.4
                 'multiprocessing_module' => PosixProcess::class,
                 'max_processes' => 32,
                 'max_process_tasks' => 100,
@@ -266,6 +280,7 @@ The table below describes the configuration parameters:
 | Parameter              | Required | Description                                                                           |
 |------------------------|:--------:|---------------------------------------------------------------------------------------|
 | scheduler_name         | yes      | Unique name of the scheduler configuration                                            |
+| scheduler_discipline   | no       | Zend Framework service providing Scheduler's process management strategy              |
 | multiprocessing_module | yes      | Specifies a `MultiProcessingModuleInterface` implementation to be used in a Scheduler |
 | start_processes        | yes      | Specifies the number of processes that will initially launch with each Server Service |
 | max_processes          | yes      | Maximum number of running/waiting processes of each Server Service                    |
@@ -283,6 +298,8 @@ Multiple _Server Services_ can be configured in a regular Zend Framework 3 confi
 ```php
 // contents of "zf3-application-directory/config/some-config.config.php" file:
 
+use Zeus\ServerService\Shared\Logger\LoggerInterface;
+
 return [
     'zeus_process_manager' => [
         'services' => [
@@ -358,6 +375,9 @@ Each service must be listed in the `services` section of `zeus_process_manager`
 ```php
 // contents of "zf3-application-directory/config/some-config.config.php" file:
 
+use Zeus\ServerService\Shared\Logger\LoggerInterface;
+use Zeus\Kernel\ProcessManager\MultiProcessingModule\PosixProcess;
+
 return [
     'zeus_process_manager' => [
         'services' => [
@@ -441,6 +461,13 @@ The above configuration parameters have been described in the __Process Schedule
 
 # Changelog
 
+## Version 1.3.4
+- [Feature] Implemented Scheduler Disciplines functionality
+- [Feature] Extracted LRU Discipline from Scheduler core
+- [Fix] Scheduler was too aggressive in its calculations of number of spare processes to create
+- [Fix] Documentation fixes (added missing namespaces in configuration examples)
+- [Tests improvements] Improved code coverage
+
 ## Version 1.3.3
 - [Tests improvements] Improved code coverage
 - [Improvement] Documentation improvements and enhancements

config/module.config.php

@@ -12,6 +12,8 @@
 use Zeus\Kernel\ProcessManager\Factory\ManagerFactory;
 use Zeus\Kernel\ProcessManager\Factory\SchedulerFactory;
 use Zeus\Kernel\ProcessManager\Factory\ProcessFactory;
+use Zeus\Kernel\ProcessManager\Scheduler\Discipline\Factory\LruDisciplineFactory;
+use Zeus\Kernel\ProcessManager\Scheduler\Discipline\LruDiscipline;
 use Zeus\ServerService\Manager;
 use Zeus\Kernel\ProcessManager\Scheduler;
 use Zeus\Kernel\ProcessManager\Process;
@@ -44,6 +46,7 @@
             Process::class => ProcessFactory::class,
             Manager::class => ManagerFactory::class,
             PosixProcess::class => PosixProcessFactory::class,
+            LruDiscipline::class => LruDisciplineFactory::class,
             //Service::class => ServiceFactory::class,
         ],
         'abstract_factories' => [
@@ -56,6 +59,7 @@
             'zeus_web_scheduler_1' => [
                 'scheduler_name' => 'zeus_web_scheduler',
                 'multiprocessing_module' => PosixProcess::class,
+                'scheduler_discipline' => LruDiscipline::class,
                 'max_processes' => 32,
                 'max_process_tasks' => 100,
                 'min_spare_processes' => 3,

src/Zeus/Kernel/ProcessManager/Factory/SchedulerFactory.php

@@ -9,6 +9,7 @@
 use Zend\ServiceManager\Factory\FactoryInterface;
 use Zeus\Kernel\ProcessManager\Scheduler;
 use Zeus\Kernel\ProcessManager\Process;
+use Zeus\Kernel\ProcessManager\Scheduler\Discipline\LruDiscipline;
 use Zeus\Kernel\ProcessManager\SchedulerEvent;
 use Zeus\ServerService\Shared\Logger\LoggerInterface;
 
@@ -35,10 +36,12 @@ public function __invoke(ContainerInterface $container, $requestedName, array $o
 
         $serviceLoggerAdapter = $options['service_logger_adapter'];
         $mainLoggerAdapter = $options['main_logger_adapter'];
+        $schedulerDiscipline =
+            isset($schedulerConfig['scheduler_discipline']) ? $container->get($schedulerConfig['scheduler_discipline']) : $container->get(LruDiscipline::class);
 
         $processService = $container->build(Process::class, ['logger_adapter' => $serviceLoggerAdapter, 'process_event' => $schedulerEvent]);
 
-        $scheduler = new Scheduler($schedulerConfig, $processService, $mainLoggerAdapter, $options['ipc_adapter'], $schedulerEvent);
+        $scheduler = new Scheduler($schedulerConfig, $processService, $mainLoggerAdapter, $options['ipc_adapter'], $schedulerEvent, $schedulerDiscipline);
         $container->build($schedulerConfig['multiprocessing_module'], ['scheduler' => $scheduler, 'scheduler_event' => $schedulerEvent]);
 
         return $scheduler;

src/Zeus/Kernel/ProcessManager/MultiProcessingModule/PosixProcess.php

@@ -24,9 +24,6 @@ final class PosixProcess implements MultiProcessingModuleInterface
     /** @var SchedulerEvent */
     protected $processEvent;
 
-    /** @var bool|null */
-    private static $isPcntlExtensionLoaded = null;
-
     /** @var PosixProcessBridgeInterface */
     protected static $pcntlBridge;
 

src/Zeus/Kernel/ProcessManager/Scheduler.php

@@ -17,6 +17,11 @@
 use Zeus\Kernel\ProcessManager\Status\ProcessTitle;
 use Zeus\Kernel\ProcessManager\Helper\EventManager;
 
+/**
+ * Class Scheduler
+ * @package Zeus\Kernel\ProcessManager
+ * @internal
+ */
 final class Scheduler
 {
     use Logger;
@@ -126,25 +131,24 @@ public function setContinueMainLoop($continueMainLoop)
     }
 
     /**
-     * Handles input arguments.
-     *
+     * Scheduler constructor.
      * @param mixed[] $config
      * @param Process $processService
      * @param LoggerInterface $logger
      * @param IpcAdapterInterface $ipcAdapter
-     * @param $schedulerEvent
+     * @param SchedulerEvent $schedulerEvent
+     * @param DisciplineInterface $discipline
      */
-    public function __construct($config, Process $processService, LoggerInterface $logger, IpcAdapterInterface $ipcAdapter, $schedulerEvent)
+    public function __construct($config, Process $processService, LoggerInterface $logger, IpcAdapterInterface $ipcAdapter, SchedulerEvent $schedulerEvent, DisciplineInterface $discipline)
     {
         $this->discipline = new LruDiscipline();
         $this->config = new Config($config);
         $this->ipcAdapter = $ipcAdapter;
         $this->processService = $processService;
         $this->schedulerStatus = new ProcessState($this->config->getServiceName());
-        $this->processStatusTemplate = new ProcessState($this->config->getServiceName());
-
-        $this->processes = new ProcessCollection($this->config->getMaxProcesses() + 1);
         $this->setLogger($logger);
+
+        $this->processes = new ProcessCollection($this->config->getMaxProcesses());
         $this->setLoggerExtraDetails(['service' => $this->config->getServiceName()]);
 
         if (!Console::isConsole()) {
@@ -203,9 +207,9 @@ protected function onProcessMessage(EventInterface $event)
     }
 
     /**
-     * @param EventInterface $event
+     * @param SchedulerEvent $event
      */
-    protected function onProcessExit(EventInterface $event)
+    protected function onProcessExit(SchedulerEvent $event)
     {
         $pid = $event->getParam('uid');
         if ($pid === $this->getId()) {
@@ -214,7 +218,7 @@ protected function onProcessExit(EventInterface $event)
             $event->setName(SchedulerEvent::EVENT_SCHEDULER_STOP);
             $event->setParams($this->getEventExtraData());
             $this->getEventManager()->triggerEvent($event);
-            return $this;
+            return;
         }
 
         $this->log(\Zend\Log\Logger::DEBUG, "Process $pid exited");
@@ -228,8 +232,6 @@ protected function onProcessExit(EventInterface $event)
 
             unset($this->processes[$pid]);
         }
-
-        return $this;
     }
 
     /**
@@ -245,11 +247,10 @@ public function stop()
             $pid = (int)$pid;
 
             if ($pid) {
-                $this->events->trigger(SchedulerEvent::EVENT_PROCESS_TERMINATE, $this,
-                    $this->getEventExtraData([
-                        'uid' => $pid, 'soft' => true,
-                    ]
-                    ));
+                $event = $this->event;
+                $event->setName(SchedulerEvent::EVENT_PROCESS_TERMINATE);
+                $event->setParams($this->getEventExtraData(['uid' => $pid, 'soft' => true]));
+                $this->events->triggerEvent($event);
                 $this->log(\Zend\Log\Logger::INFO, "Server stopped");
                 unlink($fileName);
 
@@ -434,7 +435,10 @@ protected function onShutdown(EventInterface $event)
 
         foreach (array_keys($this->processes->toArray()) as $pid) {
             $this->log(\Zend\Log\Logger::DEBUG, "Terminating process $pid");
-            $this->events->trigger(SchedulerEvent::EVENT_PROCESS_TERMINATE, $this, $this->getEventExtraData(['uid' => $pid]));
+            $event = $this->event;
+            $event->setName(SchedulerEvent::EVENT_PROCESS_TERMINATE);
+            $event->setParams($this->getEventExtraData(['uid' => $pid]));
+            $this->events->triggerEvent($event);
         }
 
         $this->handleMessages();

src/Zeus/Kernel/ProcessManager/Scheduler/Discipline/Factory/LruDisciplineFactory.php

@@ -0,0 +1,31 @@
+<?php
+
+namespace Zeus\Kernel\ProcessManager\Scheduler\Discipline\Factory;
+
+use Interop\Container\ContainerInterface;
+use Interop\Container\Exception\ContainerException;
+use Zend\ServiceManager\Exception\ServiceNotCreatedException;
+use Zend\ServiceManager\Exception\ServiceNotFoundException;
+use Zend\ServiceManager\Factory\FactoryInterface;
+use Zeus\Kernel\ProcessManager\Scheduler\Discipline\LruDiscipline;
+
+class LruDisciplineFactory implements FactoryInterface
+{
+
+    /**
+     * Create an object
+     *
+     * @param  ContainerInterface $container
+     * @param  string $requestedName
+     * @param  null|array $options
+     * @return object
+     * @throws ServiceNotFoundException if unable to resolve the service.
+     * @throws ServiceNotCreatedException if an exception is raised when
+     *     creating a service.
+     * @throws ContainerException if any other error occurs
+     */
+    public function __invoke(ContainerInterface $container, $requestedName, array $options = null)
+    {
+        return new LruDiscipline();
+    }
+}

src/Zeus/Kernel/ProcessManager/Scheduler/Discipline/LruDiscipline.php

@@ -38,7 +38,7 @@ public function manage(Config $config, ProcessCollection $processes)
 
         $statusSummary = $processes->getStatusSummary();
         $idleProcesses = $statusSummary[ProcessState::WAITING];
-        //$busyProcesses = $statusSummary[ProcessState::RUNNING];
+        $busyProcesses = $statusSummary[ProcessState::RUNNING];
         //$terminatedProcesses = $statusSummary[ProcessStatus::STATUS_EXITING] + $statusSummary[ProcessStatus::STATUS_KILL];
         $allProcesses = $processes->count();
 
@@ -70,7 +70,7 @@ public function manage(Config $config, ProcessCollection $processes)
         if ($idleProcesses < $config->getMinSpareProcesses()) {
             $idleProcessSlots = $processes->getSize() - $processes->count();
 
-            $processesToCreate = min($idleProcessSlots, $config->getMinSpareProcesses());
+            $processesToCreate = min($idleProcessSlots, $config->getMinSpareProcesses() - $idleProcesses);
         }
 
         if ($allProcesses === 0 && $config->getMinSpareProcesses() === 0 && $config->getMaxSpareProcesses() > 0) {

src/Zeus/Module.php

@@ -22,7 +22,7 @@ class Module implements
     ConsoleUsageProviderInterface,
     ConsoleBannerProviderInterface
 {
-    const MODULE_VERSION = "1.3.3";
+    const MODULE_VERSION = "1.3.4";
 
     protected static $overrideConfig = [];
 

src/Zeus/ServerService/Http/Service.php

@@ -61,7 +61,6 @@ protected function createReactLoop($messageComponent)
         $loop = LoopFactory::create();
         $reactServer = new ReactServer($loop);
         $reactServer->listen($httpConfig->getListenPort(), $httpConfig->getListenAddress());
-        //$socket->listenByUri('unix:///tmp/test.sock');
         $loop->removeStream($reactServer->master);
 
         $dispatcherConfig = $this->getConfig();

test/Helpers/ZeusFactories.php

@@ -22,6 +22,8 @@
 use Zeus\Kernel\ProcessManager\Factory\SchedulerFactory;
 use Zeus\Kernel\ProcessManager\Process;
 use Zeus\Kernel\ProcessManager\Scheduler;
+use Zeus\Kernel\ProcessManager\Scheduler\Discipline\Factory\LruDisciplineFactory;
+use Zeus\Kernel\ProcessManager\Scheduler\Discipline\LruDiscipline;
 use Zeus\Kernel\ProcessManager\SchedulerEvent;
 use Zeus\ServerService\Manager;
 use Zeus\ServerService\Shared\Factory\AbstractServerServiceFactory;
@@ -48,6 +50,7 @@ public function getServiceManager()
         $sm->setFactory(DummyServiceFactory::class, DummyServiceFactory::class);
         $sm->setFactory(ZeusController::class, ZeusControllerFactory::class);
         $sm->setFactory(Manager::class, ManagerFactory::class);
+        $sm->setFactory(LruDiscipline::class, LruDisciplineFactory::class);
         $sm->setFactory('ServiceListener', ServiceListenerFactory::class);
         $sm->setFactory('EventManager', EventManagerFactory::class);
         $sm->setFactory('ModuleManager', ModuleManagerFactory::class);