-
-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add multi-user licenses (again) #802
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ezekg
force-pushed
the
feature/add-multi-user-licenses-2
branch
3 times, most recently
from
March 9, 2024 08:40
0c02fda
to
bdbf11d
Compare
ezekg
commented
Mar 11, 2024
ezekg
commented
Mar 12, 2024
ezekg
force-pushed
the
feature/add-multi-user-licenses-2
branch
3 times, most recently
from
March 20, 2024 01:25
b5bedcc
to
9c440e4
Compare
Here are some queries that seem to perform well, and are able to be indexed efficiently: -- licenses for user
select licenses.* from licenses left outer join license_users on license_users.license_id = licenses.id and license_users.user_id = '598840ca-c529-40fd-9d9b-fe650619726a' and license_users.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' where licenses.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' and (licenses.user_id = '598840ca-c529-40fd-9d9b-fe650619726a' or license_users.user_id = '598840ca-c529-40fd-9d9b-fe650619726a') order by licenses.created_at desc limit 10;
-- users for product
select distinct users.* from users left outer join license_users on license_users.user_id = users.id and license_users.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' inner join licenses on (licenses.user_id = users.id or licenses.id = license_users.license_id) and licenses.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' inner join policies on policies.id = licenses.policy_id and policies.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' where users.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' and policies.product_id = '00dda7a7-deb7-4a76-8a84-2be19adde374' order by users.created_at desc limit 10;
-- users for product (denormalized)
select distinct users.* from users left outer join license_users on license_users.user_id = users.id and license_users.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' inner join licenses on (licenses.user_id = users.id or licenses.id = license_users.license_id) and licenses.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' where users.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' and licenses.product_id = '00dda7a7-deb7-4a76-8a84-2be19adde374' order by users.created_at desc limit 10;
-- users for license
select distinct users.* from users left outer join license_users on license_users.user_id = users.id and license_users.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' inner join licenses on (licenses.user_id = users.id or licenses.id = license_users.license_id) and licenses.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' where users.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' and licenses.id = 'aa8ac994-a3ed-4c06-89fd-4c11736b6c55' order by users.created_at desc limit 10;
-- users for machine
select distinct users.* from users left outer join license_users on license_users.user_id = users.id and license_users.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' inner join licenses on (licenses.user_id = users.id or licenses.id = license_users.license_id) and licenses.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' inner join machines on machines.license_id = licenses.id and machines.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' where users.account_id = 'f98c3ce2-9373-498a-813a-d74b1506f6e7' and machines.id = 'db10ee10-a1ac-41c4-a7c9-332a534fe398' order by users.created_at desc limit 10; |
ezekg
force-pushed
the
feature/add-multi-user-licenses-2
branch
9 times, most recently
from
March 26, 2024 21:41
cf8ebce
to
f024ece
Compare
ezekg
commented
Mar 26, 2024
end | ||
} | ||
|
||
describe '.denormalizes' do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add more tests. Would be really nice to introduce temp tables for this and union_of
.
ezekg
commented
Mar 27, 2024
ezekg
force-pushed
the
feature/add-multi-user-licenses-2
branch
3 times, most recently
from
March 27, 2024 21:22
271eaa0
to
af3e1a2
Compare
ezekg
commented
Mar 28, 2024
ezekg
commented
Mar 28, 2024
ezekg
commented
Mar 28, 2024
ezekg
force-pushed
the
feature/add-multi-user-licenses-2
branch
from
April 2, 2024 17:30
6f148f6
to
db611b3
Compare
ezekg
commented
Apr 2, 2024
- some of these have already been run in production
- unions are already distinct so we can push the distinct clause up
ezekg
force-pushed
the
feature/add-multi-user-licenses-2
branch
from
April 25, 2024 01:55
849c33f
to
40da878
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #534. Reverted #774 due to performance issues with
union_of
.The issues are mainly i.r.t.
INNER JOIN ... ON id IN (SELECT id FROM ...)
joins. For large accounts, performance is just not where I want it. E.g. an account with 600k licenses takes 2s to load a user's machines, because we have to join across 600k licenses to see if the user has any associated licenses with other users, so that we can list the user's teammates' machines as well as the user's machines. The same issue is present when listing a user's licenses, and a user's teammates.I tried optimizing a few queries by hand, but there's just too many joins on the users union association. We're going to need to optimize
union_of
itself so that we don't do anJOIN ... ON id IN (SELECT id FROM ...)
, since that could potentially select hundreds of thousands or even millions of IDs, which is not memory efficient (not to mention slow).This is what I get for skipping performance testing.
Prerequisites
Pre-deploy
license.users.attached
,license.users.detached
,license.owner.updated
,machine.owner.updated
event types.license.users.attach
,license.users.detach
,license.owner.update
,machine.owner.update
permissions.users
table).users
andlicenses
tables.Enable query logging for denormalization migration.Post-deploy
VACUUM ANALYZE licenses
.VACUUM ANALYZE users
.