RFC: Built in CLI tool to run jobs

[mariano]

One of the things I wanted to add for 2.0 is a built-in CLI tool that would make it easy for people to run jobs without them having to build an actual CLI job runner. I've just pushed to the cli_tool branch my proposal for this.

What's your opinion @Revisor?

TO-DO

• Add command line option to run a single job (wait for one, run, then exit) for atomic runs
• Add tests
• Add a section in the CLI tool documentation mentioning supervisord and memmon

[Revisor]

Hi Mariano, I think it's a great idea.

In fact, we have already been working on a CLI app running on top of disque-php the last few weeks and I'd like to ask you for your opinion and thoughts, even contributions if you have the time and feel like it, as soon as we publish the first version. Right now we are laying down the foundation.

Planned features include:

• Job metadata (track anything)
• Repeated jobs
• Language agnostic workers
• Failure handling and logging
• Configurable asynchronous worker calls
• Synchronous mode for debugging (process new jobs immediately)
• Extendable via event subscribers
• Configurable autoscaling (adding more workers when there are suddenly many jobs)

[mariano]

@Revisor that sounds pretty cool. Sure I'd love to check that out.

Any takes on this implementation, or you think we should focus on yours?

[Revisor]

You did a great job. The Worker even listens to PCNTL signals and the documentation is well written as is usual for disque-php.

However I cannot escape the feeling that the code screams "I want to be an independent application". I'll explain:

• Disque-php is a library. It doesn't have any other communication channels than calling it via PHP.
• This branch adds a console command and YAML configuration (btw. we have gone in the same direction and our configuration proposal looks similar)
• You have not included the two new dependencies, symfony/console and symfony/yaml, in the library composer.json, but guide the user to install them manually. I guess you feel that these two are not really disque-php's core dependencies.
• The Worker does too much. At the very least I would suggest moving the part that reads configuration and the part that outputs to the console to their own classes (btw. we were just talking about it today with my colleague and came to the same conclusion as you - we should use Symfony verbosity levels and compare the current one with the verbosity level of the message)
• When refactoring the settings() method, wouldn't it be cleaner and easier to use the dedicated symfony/configuration? That might be another possible dependency.
• The Worker is missing any logging necessary for unattended runs in the background. Right now it only prints messages to the console. But what if there is no one to read them? :)
• Talking about logging, why not use Monolog or any other battle tested solution? That's another possible dependency that has nothing to do with the library itself.
• What if the Handler needs its own environment? What if it needs the service container and configuration, DB/Redis/... connections, API keys from a different app - from disque-php's caller - to do its job? This opens a whole another can of worms.

And if you add just a few more sensible requirements, say repeated jobs ("I want this job to run every day at 5 AM") and language-agnostic workers (which is actually nothing else than calling other console commands), I think the scales have tipped and it really makes sense to build an independent app on top of the library.

Which is what we've been doing and what I'd like to invite you to soon.

What do you think? Does it make sense to split it into an independent application running on top of disque-php?

---

Oh, a small detail, if you want to continue working on the branch, in the docs you write that the configuration accepts a parameter marshaler for the source, but in the settings() you expect marshaller.

[Revisor]

I just wanted to let you know that I'm still working on it. It takes longer than I expected.

[mariano]

FYI I forgot to tell you I totally agree with the separate library approach for the CLI. So closing this for now :)

[amiri27]

@Revisor @mariano any chance to publish first version? Then community can help to improve the library?

Regards,

[mariano]

Are you talking about the first version of the cli tool?
On Sat, Jun 11, 2016 at 02:04 Mostafa Amiri notifications@github.com
wrote:

@Revisor https://github.com/Revisor @mariano
https://github.com/mariano any chance to publish first version? Then
community can help to improve the library?

Regards,

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#22 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AABIpuF8yNDniLBJ1PBx37Pw4HWCaEmCks5qKkG_gaJpZM4GwbpY
.

[amiri27]

Exactly..

[Revisor]

I hope to publish an alpha version soon.