Skip to content

Commit

Permalink
Allow configuration of job launcher with Symfony configuration (#121)
Browse files Browse the repository at this point in the history
* Allow configuration of job launcher with Symfony configuration

* Fixed code style

* Fixed static analysis and extract job launcher definition factory

* Add symfony/messenger as suggested dependency of yokai/batch-symfony-framework

* Added documentation about job launcher DSN
  • Loading branch information
yann-eugone committed May 2, 2024
1 parent b333150 commit 00ea597
Show file tree
Hide file tree
Showing 13 changed files with 326 additions and 90 deletions.
1 change: 1 addition & 0 deletions src/batch-symfony-framework/composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
"suggest": {
"sonata-project/admin-bundle": "If you want a SonataAdmin like rendering in the user interface",
"symfony/form": "If you want the JobExecution form filter in the user interface",
"symfony/messenger": "If you to launch jobs via messenger",
"symfony/security-bundle": "If you want to secure the access to JobExecution in the user interface",
"symfony/translation": "Required if you want to enable the user interface",
"symfony/twig-bundle": "Required if you want to enable the user interface"
Expand Down
62 changes: 56 additions & 6 deletions src/batch-symfony-framework/docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,57 @@ return [
];
```

There is few things that can be configured in the bundle at the moment.
But the most important one will be the `JobExecution` storage:
### Job launcher

You can use many different job launcher in your application, you will be able to register these using configuration:

```yaml
# config/packages/yokai_batch.yaml
yokai_batch:
launcher:
default: simple
launchers:
simple: ...
async: ...
```

> **note**: if you do not configure anything here, you will be using the [`SimpleJobLauncher`](https://github.com/yokai-php/batch/blob/0.x/src/Launcher/SimpleJobLauncher.php).
The `default` job launcher, must reference a launcher name, defined in the `launchers` list.
The `default` job launcher will be the autowired instance of job launcher when you ask for one.
All `launchers` will be registered as a service, and an autowire named alias will be configured for it.
For instance, in the example below, you will be able to register all these launchers like this:

```php
<?php

use Yokai\Batch\Launcher\JobLauncherInterface;

final class YourAppCode
{
public function __construct(
private JobLauncherInterface $jobLauncher, // will inject the default job launcher
private JobLauncherInterface $simpleJobLauncher, // will inject the "simple" job launcher
private JobLauncherInterface $messengerJobLauncher, // will inject the "messenger" job launcher
) {
}
}
```

All `launchers` are configured using a DSN, every scheme has it's own associated factory.
- `simple://simple`: a [`SimpleJobLauncher`](https://github.com/yokai-php/batch/blob/0.x/src/Launcher/SimpleJobLauncher.php), no configuration allowed
- `messenger://messenger`: a [`DispatchMessageJobLauncher`](https://github.com/yokai-php/batch-symfony-messenger/blob/0.x/src/DispatchMessageJobLauncher.php), no configuration allowed
- `console://console`: a [`RunCommandJobLauncher`](https://github.com/yokai-php/batch-symfony-console/blob/0.x/src/RunCommandJobLauncher.php), configurable options:
- `log`: the filename where command output will be redirected (defaults to `batch_execute.log`)
- `service://service`: pointing to a service of your choice, configurable options:
- `service`: the id of the service to use (required, an exception will be thrown otherwise)

### JobExecution storage

You can have only one storage for your `JobExecution`, and you have several options:
- `filesystem` will create a file for each `JobExecution` in `%kernel.project_dir%/var/batch/{execution.jobName}/{execution.id}.json`
- `dbal` will create a row in a table for each `JobExecution`
- `service` will use a service you have defined in your application

```yaml
# config/packages/yokai_batch.yaml
Expand All @@ -22,12 +69,16 @@ yokai_batch:
filesystem: ~
# Or with yokai/batch-doctrine-dbal (& doctrine/dbal)
# dbal: ~
# Or with a service of yours
# service: ~
```

> **note**: the default storage is `filesystem`, because it only requires a writeable filesystem.
> But if you already have `doctrine/dbal` in your project, it is highly recommended to use it instead.
> Because querying `JobExecution` in a filesystem might be slow, specially if you are planing to add UIs on top.
### Job as a service

As Symfony supports registering all classes in `src/` as a service,
we can leverage this behaviour to register all jobs in `src/`.
We will add a tag to every found class in `src/` that implements `Yokai\Batch\Job\JobInterface`:
Expand Down Expand Up @@ -88,13 +139,13 @@ use Yokai\Batch\Storage\JobExecutionStorageInterface;
final class MyClass
{
public function __construct(
private JobLauncherInterface $executionStorage,
private JobLauncherInterface $jobLauncher,
) {
}

public function method(): void
{
$this->launcher->launch('job.import_users');
$this->jobLauncher->launch('job.name');
}
}
```
Expand Down Expand Up @@ -122,14 +173,13 @@ final readonly class YourService
private LoggerInterface $yokaiBatchLogger,
) {
}

public function method()
{
$this->yokaiBatchLogger->error(...);
}
}
```
```

## On the same subject

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
*
* @phpstan-type Config array{
* storage: StorageConfig,
* launcher: LauncherConfig,
* ui: UserInterfaceConfig,
* }
* @phpstan-type StorageConfig array{
Expand All @@ -26,6 +27,10 @@
* dir: string,
* },
* }
* @phpstan-type LauncherConfig array{
* default: string|null,
* launchers: array<string, string>,
* }
* @phpstan-type UserInterfaceConfig array{
* enabled: bool,
* security: array{
Expand Down Expand Up @@ -53,6 +58,7 @@ public function getConfigTreeBuilder(): TreeBuilder
$root
->children()
->append($this->storage())
->append($this->launcher())
->append($this->ui())
->end()
;
Expand Down Expand Up @@ -97,6 +103,34 @@ private function storage(): ArrayNodeDefinition
return $node;
}

private function launcher(): ArrayNodeDefinition
{
/** @var ArrayNodeDefinition $node */
$node = (new TreeBuilder('launcher'))->getRootNode();

$isInvalidDsn = fn(string $value) => \parse_url($value) === false
|| (\parse_url($value)['scheme'] ?? null) === null;

$node
->addDefaultsIfNotSet()
->children()
->scalarNode('default')
->defaultValue('simple')
->end()
->arrayNode('launchers')
->defaultValue(['simple' => 'simple://simple'])
->useAttributeAsKey('name')
->scalarPrototype()
->validate()
->ifTrue($isInvalidDsn)->thenInvalid('Invalid job launcher DSN.')
->end()
->end()
->end()
;

return $node;
}

private function ui(): ArrayNodeDefinition
{
/** @var ArrayNodeDefinition $node */
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
<?php

declare(strict_types=1);

namespace Yokai\Batch\Bridge\Symfony\Framework\DependencyInjection;

use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Definition;
use Symfony\Component\DependencyInjection\Exception\LogicException;
use Symfony\Component\DependencyInjection\Reference;
use Symfony\Component\Messenger\MessageBusInterface;
use Yokai\Batch\Bridge\Symfony\Console\CommandRunner;
use Yokai\Batch\Bridge\Symfony\Console\RunCommandJobLauncher;
use Yokai\Batch\Bridge\Symfony\Messenger\DispatchMessageJobLauncher;
use Yokai\Batch\Launcher\JobLauncherInterface;
use Yokai\Batch\Launcher\SimpleJobLauncher;
use Yokai\Batch\Storage\JobExecutionStorageInterface;

/**
* This is a helper for building services definitions of {@see JobLauncherInterface}.
*/
final class JobLauncherDefinitionFactory
{
/**
* Build a service definition from DSN string.
*/
public static function fromDsn(ContainerBuilder $container, string $dsn): Definition
{
$dsnParts = \parse_url($dsn);
$launcherType = $dsnParts['scheme'] ?? null;
\parse_str($dsnParts['query'] ?? '', $launcherConfig);
/** @var array<string, string> $launcherConfig */

return match ($launcherType) {
'simple' => self::simple(),
'console' => self::console($launcherConfig),
'messenger' => self::messenger(),
'service' => self::service($container, $launcherConfig),
default => throw new LogicException('Unsupported job launcher type "' . $launcherType . '".'),
};
}

private static function simple(): Definition
{
return new Definition(SimpleJobLauncher::class, [
'$jobExecutionAccessor' => new Reference('yokai_batch.job_execution_accessor'),
'$jobExecutor' => new Reference('yokai_batch.job_executor'),
]);
}

/**
* @param array<string, string> $config
*/
private static function console(array $config): Definition
{
$log = $config['log'] ?? 'batch_execute.log';

return new Definition(RunCommandJobLauncher::class, [
'$jobExecutionFactory' => new Reference('yokai_batch.job_execution_factory'),
'$commandRunner' => new Definition(CommandRunner::class, [
'$binDir' => '%kernel.project_dir%/bin',
'$logDir' => '%kernel.logs_dir%',
]),
'$jobExecutionStorage' => new Reference(JobExecutionStorageInterface::class),
'$logFilename' => $log,
]);
}

private static function messenger(): Definition
{
return new Definition(DispatchMessageJobLauncher::class, [
'$jobExecutionFactory' => new Reference('yokai_batch.job_execution_factory'),
'$jobExecutionStorage' => new Reference(JobExecutionStorageInterface::class),
'$messageBus' => new Reference(MessageBusInterface::class),
]);
}

/**
* @param array<string, string> $config
*/
private static function service(ContainerBuilder $container, array $config): Definition
{
$service = $config['service'] ?? throw new LogicException(
'Missing "service" parameter to configure the job launcher.',
);

return $container->getDefinition($service);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
*
* @phpstan-import-type Config from Configuration
* @phpstan-import-type StorageConfig from Configuration
* @phpstan-import-type LauncherConfig from Configuration
* @phpstan-import-type UserInterfaceConfig from Configuration
*/
final class YokaiBatchExtension extends Extension
Expand Down Expand Up @@ -62,16 +63,9 @@ public function load(array $configs, ContainerBuilder $container): void
}

$this->configureStorage($container, $config['storage']);
$this->configureLauncher($container, $config['launcher']);
$this->configureUserInterface($container, $loader, $config['ui']);

$launchers = [
'yokai_batch.job_launcher.dispatch_message' => $this->installed('symfony-messenger'),
'yokai_batch.job_launcher.run_command' => $this->installed('symfony-console'),
];
$container->setAlias(
JobLauncherInterface::class,
\array_keys(\array_filter($launchers))[0] ?? 'yokai_batch.job_launcher.simple'
);
$container->registerAliasForArgument('yokai_batch.logger', LoggerInterface::class, 'yokaiBatchLogger');
}

Expand Down Expand Up @@ -168,6 +162,33 @@ private function configureStorage(ContainerBuilder $container, array $config): v
}
}

/**
* @param LauncherConfig $config
*/
private function configureLauncher(ContainerBuilder $container, array $config): void
{
if (!isset($config['launchers'][$config['default']])) {
throw new LogicException(\sprintf(
"Default job launcher \"%s\" was not registered in launchers config. Available launchers are %s.",
$config['default'],
\json_encode(\array_keys($config['launchers']), flags: \JSON_THROW_ON_ERROR),
));
}

foreach ($config['launchers'] as $name => $dsn) {
$definition = JobLauncherDefinitionFactory::fromDsn($container, $dsn);
$launcherId = 'yokai_batch.job_launcher.' . $name;
$container->setDefinition($launcherId, $definition);
$parameterName = $name . 'JobLauncher';
$container->registerAliasForArgument($launcherId, LoggerInterface::class, $parameterName);
}

$container->setAlias(
JobLauncherInterface::class,
'yokai_batch.job_launcher.' . $config['default'],
);
}

/**
* @param UserInterfaceConfig $config
*/
Expand Down

This file was deleted.

This file was deleted.

This file was deleted.

0 comments on commit 00ea597

Please sign in to comment.