[CON-118 + CON-126] (Discovery Node) Paginate /v1/full/users/content_node route

[theoilie]

Description

• Add pagination to Discovery Node for /v1/full/users/content_node route and SQL query
• Maintain backwards compatibility by ignoring the 2 new params (prev_user_id and max_users) and just returning all users
• Cache results of the route for 5 minutes

Example: https://discoveryprovider2.staging.audius.co/v1/full/users/content_node/all?creator_node_endpoint=https://creatornode11.staging.audius.co&prev_user_id=0&max_users=100 will only return 100 users, but if the Discovery Node isn't updated to this PR, it will just return all users.

Pagination works by accepting prev_user_id and max_users params. It will return all user IDs (in ascending order of user_id) that are greater than prev_user_id, up to max_users number of users. The consumer of this pagination should then pass the highest user_id as the prev_user_id into the subsequent pagination request.

Tests

Verified query by execing into Postgres container and running it. Also hit the endpoint manually with different params locally -- e.g. http://dn1_web-server_1:5000/v1/full/users/content_node/all?creator_node_endpoint=http://cn1_creator-node_1:4000&prev_user_id=3&max_users=2

How will this change be monitored? Are there sufficient logs?

Search for v1/full/users/content_node/all in Discover Node logs.

[vicky-g]

[SidSethi]

looks great!! couple small comments - can merge after

before that though, can you update pr test description to include the back-compat case i.e. that it returns everything when params not provided etc. logs would be great if possible, but not necessary

also if you could reference the new pagination format in PR desc would be great for future reference, and also mention that this handled back-compat and how it handles it

[SidSethi]

also can you re-run maddog after merging latest master to see if it works with the change that draj merged #3072

[theoilie]

also can you re-run maddog after merging latest master to see if it works with the change that draj merged #3072

I had to re-run a couple times because it failed for errors seemed unrelated and intermittent (logged them in Linear) but it passed now so I'm merging!