[PAY-127] Tips API Endpoint #3065

rickyrombo · 2022-05-10T00:44:51Z

Description

Add endpoint(s) to get tips. Used for the feed tile.

Also updates seed.py to be a bit more useable

Tests

Tested using a seeded db with 2mil follows and 2mil tips with 200k users. I think the performance looked decentish? < 5s according to prometheus

TODO: Write unittests/integration tests for this gargantuan query. Will come in a fast-follow along with whatever metrics we want to record

How will this change be monitored? Are there sufficient logs?

…ping

discovery-provider/src/queries/get_tips.py

piazzatron

Curious to hear @isaacsolo's ideas for query optimization here. Logic looks good to me

discovery-provider/src/api/v1/tips.py

discovery-provider/src/queries/get_tips.py

discovery-provider/src/api/v1/helpers.py

discovery-provider/src/api/v1/models/tips.py

isaacsolo · 2022-05-13T22:01:18Z

discovery-provider/src/queries/get_tips.py

+#             followees AS followees_for_sender
+#             ON user_tips.sender_user_id = followees_for_sender.followee_user_id
+#     WHERE
+#         followees_for_receiver.followee_user_id IS NOT NULL


not that it impacts perf much but i don't think this null check is necessary since these columns should not be nullable.

They're null if there's no followee/follower (since it's an outer join)

probably doesn't impact perf but it might be more readable to filter out null on user_tips instead of these two checks.

Not sure what you mean by that?

oops nvm i thought they were both using the same user_tips column

discovery-provider/src/queries/get_tips.py

isaacsolo · 2022-05-13T22:13:09Z

discovery-provider/src/queries/get_tips.py

+#         followees_for_receiver.followee_user_id IS NOT NULL
+#         OR followees_for_sender.followee_user_id IS NOT NULL
+#     ORDER BY
+#         user_tips.slot DESC LIMIT 100 OFFSET 0


is it necessary to sort and limit again later on?

very nit - how did we decide slot vs created at? from my understanding, created_at is a bit more accurate but just curious.

We definitely don't - actually this might break right due to the offset? Good catch.

Also the reason we don't use created_at here is I'm timestamping this per DN, but good callout maybe I can use the date set on the transaction

ooh yeah i think it could break because of the group by.

i think generally we used created_at for transaction timestamp and updated_at for inserting into the DB.

I should follow suit!! thanks, will make a follow up

isaacsolo · 2022-05-13T22:33:12Z

discovery-provider/src/queries/get_tips.py

+#         aggregate_user_tips.receiver_user_id AS receiver_user_id,
+#         followees_for_aggregate.followee_user_id AS followee_user_id
+#     FROM
+#         followees AS followees_for_aggregate


is it possible to use tips since followees isn't limited?

You mean swap the FROM and JOIN tables?

oh i meant the tips CTE. it looks like all you need is the followee_user_id and that was limited in the tips CTE.

ooh hmmm that's a good point, we could filter to only show tippers you folllow that are in the same tips query? I think that changes the output but might be a worthwhile change?

oh yeah i guess it would be a subset. maybe worth it if there's a big impact on perf.

…3065)

[e804aa1] [C-2248, C-2373] Use playlistUpdates, remove legacyNotifications (#3094) Dylan Jeffers [824933e] [C-2366] Improve web notification selection performance (#3103) Dylan Jeffers [4b8edef] [PLAT-696] Add trending-playlists/underground notifications (#3089) Dylan Jeffers [1f9cf3e] [C-2275] Fix android drawer offsets (#3095) Dylan Jeffers [fc14c82] [PAY-1063][PAY-1085][PAY-1086] Update UI for inaccessible gated tracks from favorites and history pages (#3100) Saliou Diallo [b0441f5] [C-2365] Update play buttons on web and mobile to show resume when track is current (#3101) Kyle Shanks [453910f] [C-2378] Add upload v2 feature flag (#3099) Sebastian Klingler [962a6df] [C-2337] Remove reachability mobile web (#3090) Raymond Jacobson [4ad5cd2] Fix visible collectibles for upload popup (#3093) Saliou Diallo [c143078] Fix feature flag bug (#3092) Saliou Diallo [44435b5] Fix upload prompt modal learn more url (#3091) Saliou Diallo [c9024ad] Use chat.messagesStatus instead of selector (#3087) Reed [38d43c4] [C-2369] Fix issue where notification poll can break app on signout (#3088) Dylan Jeffers [90122d9] [PAY-923] DMs: Add desktop entrypoints (#3083) Marcus Pasell [00f27e8] [PAY-907] Mobile chat reactions (#3020) Reed [4678b89] DMs: Fix broken typecheck on main (#3086) Marcus Pasell [756ade4] [PAY-1000][PAY-1084][PAY-1096][PAY-1097][PAY-1098] - More gated content fixes (#3085) Saliou Diallo [820aa9d] Fix upload and repost probers tests and lint (#3076) Sebastian Klingler [345607e] [C-2320] Fix profile socials alignment (#3079) Dylan Jeffers [569199c] Fix prod build timeout (#3084) Sebastian Klingler [12f6c22] Remove ports for local dev (#3082) Theo Ilie [1940618] Fix broken Main build due to typeerror (#3080) Marcus Pasell [eb8d47e] [PAY-1082] DMs: Dedupe sent messages (#3066) Marcus Pasell [50a11c3] Update SDK to 2.0.3-beta.0 (#3078) Marcus Pasell [c420fbb] Clean up NPM package lock (#3077) Marcus Pasell [35d1124] [C-2327] Add playlist updates slice (#3063) Dylan Jeffers [59862ad] [C-2344] Update the web playbar scrubber to respect the playback speed of podcasts (#3075) Kyle Shanks [ffeb0d3] [C-2349] Default download on wifi only to false (#3074) Andrew Mendelsohn [cafae41] [C-2325] Fix playlist table date-added column (#3073) Dylan Jeffers [384a510] [PAY-927] DMs: Empty messages state (#3068) Marcus Pasell [1132f83] Update @jup-ag/core to 2.0.0-beta.9 (#3072) Marcus Pasell [49c0ebf] [PAY-1072] Change "Download App" icon on Settings Page (#3067) Marcus Pasell [928dcaf] [PAY-1056] - More gated content updates and fixes (#3070) Saliou Diallo [1e1f769] [C-2345] Move PlaybackRate drawer to common drawers map (#3071) Kyle Shanks [f5d1251] Fix web-dist CI steps (#3069) Sebastian Klingler [5f89800] Fix heavy rotation playlist on client (#3056) sabrina-kiam [c0191e2] [C-2316] Add remote config for all oauth verification (#3052) Raymond Jacobson [40f5627] [PAY-1074][PAY-1075][PAY-1076][PAY-1080] - Update availability settings states + more QA fixes (#3059) Saliou Diallo [5be60ac] [C-2339] Update podcast control updates to also work for audiobooks (#3065) Kyle Shanks [163ebf5] [C-2297] Add fallback flag to podcast feature (#3064) Sebastian Klingler [f206391] [PAY-904] - Add gated content upload prompt (#3057) Saliou Diallo [1afc4e5] [C-1344] Move probers to monorepo and make tests pass (#3061) Sebastian Klingler [e198279] Remove random line (#3062) Saliou Diallo [24a001b] Add playback position logic for mobile (#3051) Kyle Shanks [d210124] [PAY-1070] Update TabSlider/SegmentedControl slider size on resize (#3044) Marcus Pasell

pull-request-size bot added the size/XL label May 10, 2022

rickyrombo added 6 commits May 11, 2022 02:46

Add /tips endpoint

6da9274

Fix aggregate_user

020f480

Update seed script

1f028ee

Use CTE to optimize query

70a6cb3

Add support for total_supporter_count and followee_supporters, add ty…

07f85d4

…ping

Bad merge

fb682f4

rickyrombo force-pushed the mjp-tips-endpoint branch from 1f4ca13 to fb682f4 Compare May 11, 2022 02:49

rickyrombo added 10 commits May 11, 2022 07:19

Update query for speed

bb4f562

Add logs

5756b0b

Add transaction_signature to tips

078eb97

Remove total_supporter_count

a70d008

Move models, add transaction_signatures to parser

9b29612

Fix error on no transaction signatures

cc364b6

Missed indentation

91d744a

Use alias to fix issue with DISTINCT wrapper

ca871ea

Better arg handling

c2764b8

Remove unused import

39ea078

rickyrombo assigned piazzatron May 11, 2022

rickyrombo changed the title ~~[WIP] Tips API Endpoint~~ [PAY-127] Tips API Endpoint May 11, 2022

rickyrombo marked this pull request as ready for review May 11, 2022 17:42

Update seed to not run after clearing

65052d6

rickyrombo requested a review from isaacsolo May 11, 2022 19:17

rickyrombo commented May 11, 2022

View reviewed changes

discovery-provider/src/queries/get_tips.py Show resolved Hide resolved

rickyrombo assigned dylanjeffers and sddioulde May 11, 2022

piazzatron approved these changes May 11, 2022

View reviewed changes

rickyrombo added 2 commits May 11, 2022 21:57

Use inner joins where possible

6b3dc53

Rename to tx_signature, use args.get()

0c06f16

rickyrombo merged commit 4dc66a2 into master May 12, 2022

rickyrombo deleted the mjp-tips-endpoint branch May 12, 2022 22:58

isaacsolo reviewed May 13, 2022

View reviewed changes

sliptype pushed a commit that referenced this pull request Sep 10, 2023

[C-2339] Update podcast control updates to also work for audiobooks (#…

449a8e5

…3065)

AudiusProject deleted a comment from linear bot Sep 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PAY-127] Tips API Endpoint #3065

[PAY-127] Tips API Endpoint #3065

rickyrombo commented May 10, 2022 •

edited

Loading

piazzatron left a comment

isaacsolo May 13, 2022

rickyrombo May 13, 2022

isaacsolo May 13, 2022 •

edited

Loading

rickyrombo May 13, 2022

isaacsolo May 13, 2022

isaacsolo May 13, 2022

rickyrombo May 13, 2022

rickyrombo May 13, 2022

isaacsolo May 13, 2022

rickyrombo May 13, 2022

isaacsolo May 13, 2022

rickyrombo May 13, 2022

isaacsolo May 13, 2022 •

edited

Loading

rickyrombo May 13, 2022

isaacsolo May 14, 2022

[PAY-127] Tips API Endpoint #3065

[PAY-127] Tips API Endpoint #3065

Conversation

rickyrombo commented May 10, 2022 • edited Loading

Description

Tests

How will this change be monitored? Are there sufficient logs?

piazzatron left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

isaacsolo May 13, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

isaacsolo May 13, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rickyrombo commented May 10, 2022 •

edited

Loading

isaacsolo May 13, 2022 •

edited

Loading

isaacsolo May 13, 2022 •

edited

Loading