Remove the requirement to restart the sceduler after adding a new account.

[scriptsrc]

After adding a new AWS account in the web UI, users must restart the scheduler with supervisorctl.

It would be better if this was not required.

[JaguarSecurity]

Thanks!

[ollytheninja]

Starting work on this today

[cstewart87]

Any updates on the status of this issue?

[ollytheninja]

Just waiting on the go ahead to publish it 👍

[cstewart87]

Awesome! Does that mean you have an existing branch that I can take a look at and you're waiting to submit a PR? Or, you're waiting for approval to push to the public?

[sP3n1]

Any updates on this issue?

[AlanMars]

Any updates on this issue via Web UI?
What is the root reason for this issue? Is it the web UI issue or Security Monkey API issue?
And do we have the same issue above when we call API to add new account? Thank you.

[ollytheninja]

Hi Alan,

The issue is that the watcher collects all the jobs to be run at startup, rather than dynamically checking what needs to be run.
This results in two issues:
1 the watcher won't pick up new accounts.
2 you cannot run the watcher in a distributed setup i.e. accross multiple servers in an autoscaling group.

The underlying cause of all of that is the APScheduler implementation, specifically limitations of the old version of APScheduler.
The right way to fix this is to rewrite the APScheduler implementation to use APScheduler 3.
I did this work however due to a number issues, by the time I had published my work Security Monkey had diverged far enough that it wasn't mergable.
It is still sitting in my fork if you want to take a look, but at this stage it basically needs to be fixed again from scratch.

The UI calls the API, so yes API and UI are in parity on this.

[AlanMars]

@ollytheninja Thank you so much for your quick reply with details.

[mikegrima]

This might actually not be relevant anymore due to the changes in #911 .

The main thing to confirm is if it will automatically updated on scheduler timing changes.

This will still be relevant. We'll need to consider using something that is able to hot-modify the scheduler in the message broker.

[tiwari85aman]

will it be right if we write a script to automatically restart the supervisorctl whenever a new account is being added to the database or we encounter such thing in the logs?
a naive solution for this

[bharathsat]

I have configured security monkey and started supervisor for all UI, scheduler, and worker, Scheduler and UI is started running but worker went to FATAL state and its not starting if i try to start.

security_monkey]# supervisorctl
securitymonkeyscheduler          RUNNING   pid 19966, uptime 0:52:24
securitymonkeyui                 RUNNING   pid 14588, uptime 2:34:14
securitymonkeyworkers            FATAL     Exited too quickly (process log may have details)
supervisor>

Can you please help me to fix this.

[rohammosalli]

@ollytheninja any update about the fix, is this issue fixed? I have Redis and celery also security monkey scheduler is in a running state. I use the 1.1.3 version

[mikegrima]

Hello @rohammosalli:

In the full interest of transparency, we are actively working on Security Monkey's replacement, and are only supporting Security Monkey in a very limited fashion at this time.