feat!: Support table_concurrency and resource_concurrency

[yevgenypats]

Summary

Closes cloudquery/cloudquery#2313 cloudquery/cloudquery#159 #264

This renames concurrency to table_concurrency (this has the same behaviour as previously but the name was too generic).

Also this introduce a new option resource_concurrency - this limits the number of go routines that resolve a specific resource (useful when a lot of column resolvers that have api calls and PreResourceResolver).

Importe Note: both options are only for top-level concurrency control i.e we only spawn go-routines for parent tables and for resources of parent table. The reason for that is that there is no concurrency model that can work with one variable for recursive calls otherwise it will get deadlock.

So right now the SDK will support only for one level - in the future if we want to support additional level we can do table_concurrency_1 and resource_concurrency_1 but I think for now we can skip this.

[hermanschaaf]

This is great 👍 I left a few comments / thoughts.

Not for this PR, but I suppose we could add another option later to control concurrency for child tables at well. It wouldn't be a global pool (could get deadlock), but we could say, for example, that every table is only allowed to spin up 10 goroutines for its children. That way you'd have some control and children can be processed concurrently, but you can't run into a deadlock situation. Goroutines and memory usage can theoretically grow unbounded though.

There are other more complicated options too. Having parent tables relinquish control over their goroutine before waiting on child tables to complete should also help avoid deadlock. But it gets real complicated fast; I like the approach in this PR.

[yevgenypats]

This is great 👍 I left a few comments / thoughts.

Not for this PR, but I suppose we could add another option later to control concurrency for child tables at well. It wouldn't be a global pool (could get deadlock), but we could say, for example, that every table is only allowed to spin up 10 goroutines for its children. That way you'd have some control and children can be processed concurrently, but you can't run into a deadlock situation. Goroutines and memory usage can theoretically grow unbounded though.

There are other more complicated options too. Having parent tables relinquish control over their goroutine before waiting on child tables to complete should also help avoid deadlock. But it gets real complicated fast; I like the approach in this PR.

Thanks! I don't think the 10 per child will work as then we can't bound goroutines (this is what we had with the list/detail resolvers where we just spawn up as many goroutines as we wanted) which is the whole point otherwise we can just have unlimited go routines and that's it but that we know can cause trouble.