Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pagination support to ReverseQueryRelationships #1280

Merged
merged 1 commit into from
May 3, 2023

Conversation

josephschorr
Copy link
Member

@josephschorr josephschorr commented Apr 21, 2023

Ran the additional benchmarks at a relationship count of 100K (vs the 1K in the test code here) and found that the changes to the postgres relationship filter are a tiny (within margin of error it seemed) performance loss vs the current sorted reads; there are likely additional optimization opportunities here though

@github-actions github-actions bot added the area/datastore Affects the storage system label Apr 21, 2023
@josephschorr josephschorr marked this pull request as ready for review April 27, 2023 19:09
@josephschorr josephschorr requested review from vroldanbet and a team as code owners April 27, 2023 19:09
@github-actions github-actions bot added the area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools) label Apr 27, 2023
Copy link
Contributor

@vroldanbet vroldanbet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me, I left 2 suggestions

cursor.Subject.Relation,
)
// For performance reasons, remove any column names that have static values in the query.
columnNames := make([]string, 0, len(columnsAndValues))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both TupleComparison and ExpandedLogicComparison have become tricky enough that I think it would be useful to have a couple of unit tests. Once they are refactored, and these being in the critical path, I think it would also be useful to run some testing.B benchmarks to see if there are opportunities to reduce allocations and reduce CPU. E.g. there is a bunch of places in the codebase where we actively moved away from fmt.Sprintf. If I'm reading this correctly, the squirrel operands seem to have a backing slice, so preallocating their capacity may come a long way at scale.

vroldanbet
vroldanbet previously approved these changes May 3, 2023
Copy link
Contributor

@vroldanbet vroldanbet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left an optimization suggestion

}

if len(comparisonSlots) > 0 {
comparisonTuple := "(" + strings.Join(columnNames, ",") + " > " + strings.Join(comparisonSlots, ",") + ")"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is micro-optimizations, but believe at large-scale all those allocations do matter. We can spare one slice allocation using strings.Repeat. It's also just a bit faster.

func BenchmarkRepeat(b *testing.B) {
	columnNames := make([]string, 6, 6)
	for i := 0; i < b.N; i++ {
		_ = strings.Repeat(",?", len(columnNames))[1:]
	}
}

func BenchmarkJoin(b *testing.B) {
	columnNames := []string{"?", "?", "?", "?", "?", "?"}
	for i := 0; i < b.N; i++ {
		strings.Join(columnNames, ",")
	}
}
cpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
BenchmarkRepeat
BenchmarkRepeat-16    	23180281	        50.46 ns/op
PASS

cpu: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
BenchmarkJoin
BenchmarkJoin-16    	18482060	        61.02 ns/op
PASS

@josephschorr
Copy link
Member Author

Updated

vroldanbet
vroldanbet previously approved these changes May 3, 2023
@josephschorr josephschorr merged commit 436e957 into authzed:main May 3, 2023
24 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators May 3, 2023
@josephschorr josephschorr deleted the reverse-query-pagination branch May 3, 2023 20:06
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/datastore Affects the storage system area/tooling Affects the dev or user toolchain (e.g. tests, ci, build tools)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants