Stable pipeline with rovers and borges

[ajnavarro]

Parent objective: #14
Progress: 0/3

• [P0] Retry on non-fatal failures with some delay (e.g. 1h for temporary, 1d for permanent)
• [P0] Improve debugging capabilities (e.g. recover from panics, develop the right tooling to retrieve logs for failed jobs)
• [P1] add different priorities to jobs
• [P2] update already fetched repositories depending on last commit date and fetched at date
• [P2] regression tests
• [P3] handle correctly repository deletion

[ajnavarro]

@smola should you take a look?

[ajnavarro]

@smola I added regression tests tasks. I think we will need this to avoid unexpected bugs on different parts of our stack.

[marnovo]

[OKRs review] This is too detailed for the OKRs, should belong to an epic.