Logs from "files" processor are capped at 256000 bytes

[jimmyw]

Describe the bug
Logs from "files" processor are capped at 256000 bytes

To Reproduce
Steps to reproduce the behavior:

1. Create a job, use the files processor

Expected behavior
There is plenty of disk to use for logs. There should be no cap.

** Specifications:**

• OS: Linux
• Version 3.2.1

Additional context
Add any other context about the problem here.

[vcastellm]

This is a limitation of the executor https://github.com/distribworks/dkron/blob/master/builtin/bins/dkron-executor-shell/shell.go#L27 not the processor, there are several ways of fixing this, I've got this from the top of my head:

• Change the architecture to execute processors on streaming output to continuously execute processors

[cobolbaby]

Change the architecture to execute processors on streaming output to continuously execute processors

The design should make this easier

[jimmyw]

Streaming out the logs is the correct way, buffering in ram is wrong in so many ways...