Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MB-54131: Geoshape query decode optimization #14

Merged
merged 9 commits into from Aug 30, 2023

Conversation

Likith101
Copy link

@Likith101 Likith101 commented Aug 8, 2023

- Added buffer pool pointer to polygon
- Added parent polygon pointer to loop
- Added decoder function to select, populate and return a pointer to the buffer from a pool of buffers
- Updated loop decode function to handle chunks of data being read
- Updated polygon decode function to pass on the buffer pool to the loops
- Updated extractShapesFromBytes and FilterGeoShapesOnRelation to include the buffer pool in its parameters

Related PRs
- blevesearch/bleve#1864

	- Added buffer pool pointer to polygon
	- Added parent polygon pointer to loop
	- Added decoder function to select, populate and return a pointer to the buffer from a pool of buffers
	- Updated loop decode function to handle chunks of data being read
	- Updated polygon decode function to pass on the buffer pool to the loops
	- Updated extractShapesFromBytes and FilterGeoShapesOnRelation to include the buffer pool in its parameters
geojson/geojson_shapes_util.go Outdated Show resolved Hide resolved
geojson/geojson_shapes_util.go Outdated Show resolved Hide resolved
s2/loop.go Outdated Show resolved Hide resolved
s2/loop.go Outdated Show resolved Hide resolved
s2/loop.go Outdated Show resolved Hide resolved
s2/loop.go Outdated Show resolved Hide resolved
s2/polygon.go Outdated Show resolved Hide resolved
s2/polygon.go Outdated Show resolved Hide resolved
 - Changed buffer pool from pointer to slice
 - Removed randomly formatted comments
@Likith101 Likith101 force-pushed the master branch 2 times, most recently from 167dcc2 to d279cac Compare August 22, 2023 09:56
s2/encode.go Outdated Show resolved Hide resolved
s2/encode.go Outdated Show resolved Hide resolved
s2/loop.go Outdated Show resolved Hide resolved
 - Added a buffer pool struct
 - Increased size of the maximum buffer to 24KB
 - Added better comments
s2/buffer_pool.go Outdated Show resolved Hide resolved
s2/buffer_pool.go Outdated Show resolved Hide resolved
s2/buffer_pool.go Outdated Show resolved Hide resolved
 - Minor formatting fixes
 - Reusing buffers instead of creating everytime
s2/loop.go Outdated Show resolved Hide resolved
s2/loop.go Outdated Show resolved Hide resolved
s2/loop.go Outdated Show resolved Hide resolved
s2/loop.go Outdated Show resolved Hide resolved
s2/buffer_pool.go Outdated Show resolved Hide resolved
 - Better comments
 - Removed some magic numbers
s2/loop.go Outdated Show resolved Hide resolved
 - Changed buffer pool implementation to give the smallest buffer that fits the data completely
 - Changed decoder implementation to allow buffer sizes bigger than number of bytes needed
if size >= len(buf) {
_, d.err = io.ReadFull(d.r, buf)
return len(buf)
} else {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we are using return statement inside "if" block, thus we don't need an else block.

if size >= len(buf) {
  _, d.err = io.ReadFull(d.r, buf)
  return len(buf)
}

_, d.err = io.ReadFull(d.r, buf[0:size])
return size

 - Added benchmark for loop decode
@@ -24,6 +25,9 @@ import (
"github.com/golang/geo/s1"
)

const SizeOfFloat = 8
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should perhaps use reflect to detect size of float64. Will add separately.

@abhinavdangeti abhinavdangeti merged commit 045f1ed into blevesearch:master Aug 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants