dynamic updaters

[hdonnay]

This PR provides for having all updaters dynamically constructed upon use.

This allows databases to be discovered at runtime (e.g. RHEL's use of a Pulp repository) and for graceful degradation when distributions fall out of support (e.g. Ubuntu's non-LTS releases).

[ldelossa]

PR looks pretty good.

I have a concern around concurrency control.

Currently concurrency control is happening in Libvuln via a weighted sem.
Once the weighted sem hands work to each executor, that executor runs all update operations in parallel.

Ideally the configuration would reliably say "this is how many update operations will occur in parallel".
I'd be less concerned about this, but fetching, reading, parsing, and batch insertion into the database are our heaviest tasks on the matcher side, and being deterministic about how many of those tasks are running at once will be a key factor in scaling/tuning/troubleshooting.

My concerns also exist in slowing down VulnerabilityReportGeneration when too many background update operations are occurring. IMO, we should design to maintain predictable vuln report generation times, as Quay will be querying this endpoint often.

I'd like to see some control structure which deterministically says "this is the max number of update operations running" in terms of concurrency control.

[ldelossa]

The flow of control for form Pulp -> RHEL Updaters seems a bit silly.

I have to construct a pulp object in another package, feed it a function in the rhel package, to have it construct updaters in the rhel package.

an init function is used to hardcore the construction of the pulp object anyway.

Can we please collapse this into just the rhel package and not make Pulp a stateful object that just couples itself to the rhel package immediately?

[hdonnay]

The way it's implemented, the pulp object isn't coupled to rhel at all. It's the rhel object that's coupled to the pulp object, because that's how the databases are delivered. I like the separate pulp object because it's a nice, self-contained bit of code.

The init function is only used to preserve the "default" behavior. I'll rework that to construct the factories in the libvuln constructor.

[ldelossa]

But what's the purpose of self containing it, is what I'm failing to see. All you do is feed it a function implemented somewhere else to construct what you need. It being collapsed would make a lot more sense and pulp being placed inside rhel/ it will never be used outside RHEL.

[ldelossa]

LGTM, cool PR.