Documenting the usage of database backend options

[mhausenblas]

Currently there exists no documentation for database backend options, esp. around the HTTP option. Create user-friendly docs incl. example in a docs/ dir.

---

Original issue for clarity reasons:

Currently, the aggregation and reporting back-end is hardwired to BigQuery. It would be very beneficial, for example for on-prem deployments or individuals who don't want to or can not use BQ to make Spartakus support different back-ends in a pluggable manner. For example, one might want to use Minio+PrestoDB or simply a RDBMS such as Postgres.

[mhausenblas]

@thockin WDYT? Also, I seem not be able to self-assign this one?

[squat]

@mhausenblas currently the way to do this is to use the provided http database option. Both volunteers and collectors can report directly to an http endpoint, which means the record is simply POSTed as JSON to the provided URL. Someone wanting to report to a custom database could have volunteers report to an http service that accepts records and sends them off to the desired backing database. This is all the collector does in reality.

This repo is very slim and I think that unless any plugin system is likewise simple and slim it would be better to encourage use of the http backend as the plugin system. For example, users could deploy a Postgres Spartakus “plugin” as a sidecar to the volunteer that listens on some port, e.g. 8443 and configure the volunteer to post to localhost:8443.

[mhausenblas]

Thanks a lot for the speedy reply @squat much appreciated. I must have overlooked that HTTP database option, can you please point me to the docs for it?

[squat]

@mhausenblas there are no user-friendly docs covering database options at the moment :/ this would be a good addition to the project. To enumerate the options, the best choice is to spartakus volunteer —print-databases

[mhausenblas]

Thanks for taking the time to sync on this topic @squat and I've changed the title and description. Can you please review and either LGTM it or add as you see fit?

[mhausenblas]

A few things that should be documented in addition to above:

• how to interact with Spartakus
• RBAC setup
• currently only works with rest.InClusterConfig

[thockin]

docs are good

I am OK to add more backend modules, but we should refactor away from a monolith package into an interface and a bunch of plugin implementations. but yeah, it's basically an HTTP-to-whatever adaptor.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.