[10.x] Make DynamoDB failed job provider countable

[timacdonald]

See: #48177

Dynamo DB documentation: https://docs.aws.amazon.com/aws-sdk-php/v2/api/class-Aws.DynamoDb.DynamoDbClient.html#_describeTable

I've left this one as a draft as I would absolutely love if someone from the community could test this out for me.

Should note that this is the "performant" way of counting records.

The number of items in the specified table. DynamoDB updates this value approximately every six hours. Recent changes might not be reflected in this value.

Maybe we need to not use this? I'm not sure what best practice is with Dynamo and if this kind of "eventually consistent" approach is just part of how it works.

[timacdonald]

Uh! I've just realised this will require filtering by queue and connection, so this implementation likely isn't suitable.

[driesvints]

@timacdonald should we close this one?

[timacdonald]

Nah, I'd like to get it moving still.

[GrahamCampbell]

Counting in dynamodb can be very slow. If there are millions of items, this will take minutes with many requests, if done sequentially via Query configured to only return item count.

[GrahamCampbell]

It actually looks like a (parallel) scan is needed here, since queue and connection are not part of the pk/sk pair, or a gsi.

[GrahamCampbell]

I d'no if anyone is actually using this driver at any scale anyway, since the primary key design here is not suitable for doing more than a ~100 req/s.

[GrahamCampbell]

Given all this, and given that we already have an all method, which also won;t work if there are lots of jobs... why not just implement count by doing count($this->all()), or something close to this?

[timacdonald]

hmmm. I'm not sure about all this. Maybe I do close this and someone can implement a driver if they want the feature.

I think most applications are using MySQL / Postgres or something similar that the other drivers already support.