Skip to content

Add Selinon - a task flow orchestrator on top of Celery project#1429

Closed
fridex wants to merge 1 commit intovinta:masterfrom
fridex:selinon
Closed

Add Selinon - a task flow orchestrator on top of Celery project#1429
fridex wants to merge 1 commit intovinta:masterfrom
fridex:selinon

Conversation

@fridex
Copy link
Copy Markdown

@fridex fridex commented Nov 7, 2019

What is this Python project?

An advanced task flow management on top of famous Celery project (a distributed task queue). It allows one to create advanced task flows by creating YAML configuration files which state dependencies between tasks, fallback tasks (run on task failures) or run whole flows (transitively or even recursively). Dependencies between tasks or flows are graphs (without any limitation to DAG). It also enables one to use different storage or database adapters or running only certain tasks in a flow definition - system will automatically compute dependencies between tasks (which tasks needs to be run in order to run desired tasks). Selinon comes with a set of prepared storage/database adapters and support for tracing system actions (easy to integrate with Sentry or other monitoring systems) or logging ELK (Elasticsearch, logstash, Kibana) stack to provide ready-to-be-used system in clusters (using Kubernetes, OpenShift or others). A built in simutor can run your flows by executing CLI.

What's the difference between this Python project and similar ones?

Celery is limited when it comes to modelling more advanced task flows (dependencies between tasks). Selinon offers fine-granted control on how your task flow will work, how it will perform and how it will be traced in distributed environment.

--

Anyone who agrees with this pull request could vote for it by adding a 👍 to it, and usually, the maintainer will merge it when votes reach 20.

@fridex
Copy link
Copy Markdown
Author

fridex commented Nov 7, 2019

The project already aggregated 20+ thumbs up in #943 but the PR was closed by the bot. PTAL.

@stale stale Bot added the stale label Dec 7, 2019
@stale stale Bot closed this Dec 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant