Feature - refactor cron formatting

[azeemshaikh38]

Is your feature request related to a problem? Please describe.
The cron job appends the output of scorecard commandline to an output. This is a problem since, the cron job deals with string output from the scorecard command and tries to get the formatting right.

Describe the solution you'd like
Instead, the cron job should be dealing with in-memory data structure of check results which it can use in any way it likes, including formatting it using pre-existing formatting libraries (like JSON). To do this, we need to refactor such that we have a common library of functions/types which will be used by both the commandline tool as well as the cron job.

@inferno-chromium - this is a fair amount of refactor work, but IMO worth the time/effort. Let me know if this LG to you and I can get started on it.

[naveensrinivasan]

Instead, the cron job should be dealing with in-memory data structure of check results which it can use in any way it likes, including formatting it using pre-existing formatting libraries (like JSON). To do this, we need to refactor such that we have a common library of functions/types which will be used by both the commandline tool as well as the cron job.

A good idea IMO. But with the in-memory you could run into memory bottlenecks as we scale to 10,000 -100,000 repository.

Will the solution be an issue for such scales?

[azeemshaikh38]

Depends on how we handle the results. We could keep writing to file/DB every time a result comes back or we can batch it to reduce IO. But yes, we mostly will not hold all results in memory at the same time.

[inferno-chromium]

SGTM.

[oliverchang]

Big +1 for this :) This would clean up a lot of how the current outputting is done.