Are you looking for help in developing Orchestra?

[JTF4]

Me and a friend of mine would love to help you out if you would like help developing it. We would love to see this project be a success.

[jhuckaby]

Hey @JTF4,

Thank you so much for the offer! I'm currently developing Orchestra in stealth mode (offline), as I did the original Cronicle back in 2015. What I like to do is open source it when it's ready for launch. It's just how I prefer to design and code apps.

That being said, I would love to hear which features you'd like to see in Orchestra. Here is what I have completed or have planned at a high level, beyond the features in Cronicle:

• Orchestra will have a full server monitoring and alerting system built-in
  • Similar to my own Performa app, but much improved and built right in.
• Redesigned server communication system
  • Servers no longer have to be "added" to the UI manually -- they add themselves on startup.
  • This also allows for ephemeral servers like AWS autoscale.
• Redesigned web socket system for the UI
  • The user only requires a connection to the main server. The UI will no longer make direct connections to worker servers.
  • This should solve many problems reported by people with complex network architectures.
• Ship entire app as a compiled Linux binary
  • i.e. no install-time dependencies, not even Node.js
  • You will still be able to install from source if you want.
• Completely redesigned user interface
  • Dark mode / light mode
  • Responsive design for tablet/mobile
• Much improved real-time monitoring systems for running jobs
  • Show live CPU/memory as a timeseries chart
  • Show complete process tree with full parent/child relationships
  • Option to kill all processes on job abort.
• Proper international date/time support
• Much better "event chaining" feature
  • Allow users to create a "decision tree" of events that fire each other off
  • Improved UI to display this event relationship
• Much better event history, with filters such as "show only errors" or "show only successes".
  • Allow users to "tag" their jobs with keywords, which are also searchable.
• Remove a bunch of dependencies
  • Instead, use native or in-house solutions.
  • Examples include removal of socket.io, replaced with native WebSockets in the client, and ws on the server.
  • Also, removal of all graph/chart libraries, replaced with native SVG / Canvas code.
  • Also, removal of the legacy moment.js, replaced with native browser/Node APIs.

Anything else you'd like to see? I'm all ears!

[JTF4]

One big feature that I would love to see if a group schedule. It would be fantastic to create a group that runs at X time and all of the actions have time offsets in them that are set from the trigger time on the master group. I use cronicle for a lot of one off events and it's a lot of work to go through 30 different actions and change the time on them all, and this feature would make it so that I just have to change one time for the entire group of actions.

[jhuckaby]

@JTF4 Awesome! I love it! This fits right in with my redesign of the event chaining feature, where you can now define entire "event groups" and run jobs within that group. I will come up with a way to do this with time offsets like you said, in addition to chaining. Thank you! 🙏🏻

[justinmdudley]

This all sounds amazing. I would love to add one suggestion around users. It would be fantastic if we could utilize other identity providers using OAuth. I absolutely love Cronicle, but it is the only application we run which requires a username/password.

[jhuckaby]

@justinmdudley Ah, thank you! I forgot to list that I am adding Active Directory login support, and I will definitely work on OAuth as well.

[josephdadams]

What changes/improvements were you thinking regarding the use of plugins?

[jhuckaby]

@josephdadams Ah, I have some fun changes in store for Plugins. They will still work as they do now (launch child process and communicate with NDJSON over STDIO), but you will now be able to send back real-time telemetry about running jobs. For example, you can send back a full HTML snippet to display in real-time on the job status screen while the job is running. This is great for long-running jobs that need more detailed real-time status, more than simply "progress" and a "log".

There's more. I'm planning on a way for Plugins to submit entire files to the Cronicle master host, as a form of "output" which can be sent during or at the end of a job. These files will be downloadable from the UI, links included in the notification e-mail, can be sent as web hooks to URLs of your choice, and can even be passed between jobs by way of chaining.

Cool, right?

[JTF4]

Are you keeping with the python plugin form?

[jhuckaby]

@JTF4 I'm sorry, I am having trouble understanding. What is a "python plugin form"? Cronicle and Orchestra are both language-agnostic. They don't care what language your Plugins are written in, as long as they can read JSON via STDIN and send it via STDOUT. Python should work fine.

[josephdadams]

@JTF4 we wrote most of our plugins in Python because it was fast and easy

[ericma15]

@josephdadams啊，我为插件准备了一些有趣的变化。它们仍将像现在一样工作（启动子进程并通过 STDIO与NDJSON通信），但您现在可以发回有关正在运行的作业的实时遥测数据。例如，您可以发回完整的 HTML 片段，以便在作业运行时实时显示在作业状态屏幕上。这对于需要更详细的实时状态而不仅仅是“进度”和“日志”的长时间运行的作业来说非常有用。

还有更多。我正计划让插件将整个文件提交给 Cronicle 主控主机，作为一种“输出”形式，可以在作业期间或结束时发送。这些文件可以从 UI 下载，通知电子邮件中包含的链接，可以作为 Web 挂钩发送到您选择的 URL，甚至可以通过链接的方式在作业之间传递。

酷，对吧？

Wow, that sounds so cool!

[arivictor]

Hi @jhuckaby did you have any thoughts on secrets or credential management? If you're familiar with Jenkins, for example, it has a global credential store where users can enter usernames/passwords, secrets, keys, etc, that can then be referenced in pipelines.

We bashed out a proof of concept into Cronicle to allow users to add secrets within events which can then be passed into the plugin at job runtime, its not very robust however, but does the trick. Its not shown in the screenshot, but after entering the secret and saving the event it then encrypts it. When the job runs it gets decrypted. We've also opted to follow your syntax eg. [/secret/credential]

[mikeTWC1984]

@arivictor I agree, some sort of secret management would be extremely useful. @jhuckaby if you don't have much time to implement your own solution you can create integration with some existing tool, like hashicorp vault.

[jhuckaby]

@arivictor @mikeTWC1984 Yes this is an excellent suggestion -- thank you! I'll look into the best way to implement this, including Hashicorp Vault, of which I'm currently unfamiliar.

[care2DavidDPD]

@jhuckaby - I see that you remove the disclaimer about maintenance-only mode and Orchestra - c2bfdc1 - did plans change ?

We are going to deploy a dcron system shortly, Cronicle really fits the bill (most others are too big and complex) and can't wait for Orchestra. Just don't want to deploy something new that is DOA.

[jhuckaby]

@care2DavidDPD I apologize for the confusion -- the disclaimer is still active, it was just moved to the pull request template (actually, it was always there). I removed it from the README as it was a redundant copy, and I wanted to emphasize the latest upgrade notice about v0.8 to v0.9. Nothing has changed -- Cronicle is still being maintained, and I am working on Orchestra every free moment I have 😊

[falldeaf]

Hi @jhuckaby I just found Cronicle and started digging in, I really love the functionality. I just had a few questions if I could about Orchestra if I could.

I love the feature upgrades that you've mentioned so it sounds like I'll be happy to migrate when it's out (especially if there's automatic data migration. I was thinking it'd be neat to write a little node library you could call from your plugin code that handles communication with cronicle/orchestra. Not that the stdin and json is overly difficult but it'd be neat to have really simple promise-based commands to capture input and send info.

Since you'd said you might be interested in hearing feature ideas for Orchestra, one feature I would really love is the ability to write and test plugins right in the web interface! I'd love to be able to select 'create new plugin', in addition to the 'add new plugin', that would go to an in-browser code editor. with a run button that tests the plugin and captures the output as the system really would. Then be able to save it, and edit your plugins without logging into the server.

Lastly, this is probably not something you'd want to think about for an initial launch, but a public plugin library would be really great, too. If developers could publish plugins somewhere, and users could look though and add them, it could save devs time and make Orchestra a low/no code solution for automation as well!

Thanks! (And sorry for the wall of text, I'm excited about finding your software and hearing about a big upgrade on the horizon)

edit:(removed stupid question already answered elsewhere)

[jhuckaby]

Hi there, @falldeaf! Sorry for the late reply. Been super busy over here.

I was thinking it'd be neat to write a little node library you could call from your plugin code that handles communication with cronicle/orchestra.

Feel free! Although, I should tell you, I actually already wrote one years ago: pixl-json-stream. This is what the Shell Plugin uses to communicate with Cronicle, but I use it in a bunch of other places as well. It's not promise-based, but rather event based. It has served me very well over the years, but I'm sure it could be improved upon.

one feature I would really love is the ability to write and test plugins right in the web interface! I'd love to be able to select 'create new plugin', in addition to the 'add new plugin', that would go to an in-browser code editor.

This is a cool idea, but one problem I see with it is that Plugins often require dependencies to be installed. They also aren't always one single file, but rather many files across multiple directories. I feel like, to implement this kind of thing properly, you'd need a complete file and directory management system with a remote file editor, and the ability to install dependencies, that would need to support multiple programming languages. It kinda feels like that's a bit out of scope for Orchestra.

with a run button that tests the plugin and captures the output as the system really would. Then be able to save it, and edit your plugins without logging into the server.

I do like the idea of "testing" a Plugin, without having to create an event in the schedule. I'll definitely think about implementing this.

a public plugin library would be really great, too. If developers could publish plugins somewhere, and users could look though and add them, it could save devs time and make Orchestra a low/no code solution for automation as well!

Interesting idea. What sort of Plugins would be useful in a public library? I mean, like, what kinds of things would they do? In my own experience my Plugins have been very specialized, and wouldn't be useful to anyone else.

Semi-related to this: I was actually already thinking about opening up the "hook" system to the idea of Plugins. For example, right now Cronicle (and Orchestra) can send out web hooks for events like job completions, but people probably would love some common 3rd party actions like: post message to slack or discord, send SMS via Twilio, send push notification via Pushover, etc. Those kinds of things could be made into "Hook Plugins" or "Action Plugins" and distributed publicly, probably via NPM. I could see that kind of thing becoming a public library.

Anyway, thank you very much for the suggestions!