Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ydb filer store #2996

Merged
merged 20 commits into from May 5, 2022
Merged

ydb filer store #2996

merged 20 commits into from May 5, 2022

Conversation

kmlebedev
Copy link
Contributor

@kmlebedev kmlebedev marked this pull request as ready for review May 2, 2022 19:21
@kmlebedev
Copy link
Contributor Author

@chrislusf Checked filer starts here

@kmlebedev
Copy link
Contributor Author

@chrislusf I'm done, all tests passed except #3011

@chrislusf
Copy link
Collaborator

you may need to add one empty file to the ydb folder, to ensure compilation when "ydb" tag is not defined.

@kmlebedev
Copy link
Contributor Author

you may need to add one empty file to the ydb folder, to ensure compilation when "ydb" tag is not defined.

done

@chrislusf chrislusf merged commit a69bf53 into seaweedfs:master May 5, 2022
weed/filer/ydb/ydb_store.go Show resolved Hide resolved
weed/filer/ydb/ydb_store.go Show resolved Hide resolved
var name string
var data []byte
for res.NextResultSet(ctx) {
for res.NextRow() {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YDB has limit for return rows (default - 1000).
The client code must check Truncated() flag to detect if response truncated (and select next page/return error).

res.CurrentResultSet().Truncated()

Copy link
Contributor Author

@kmlebedev kmlebedev May 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rekby How to get this page if the request already has a limit and a start name ?
Perhaps there are regular means without changing the select query ?

		SELECT name, meta
		FROM filemeta
		WHERE dir_hash = $dir_hash AND directory = $directory and name >= $start_name and name LIKE $prefix
		ORDER BY name ASC LIMIT $limit;`

weed/filer/ydb/ydb_queries.go Show resolved Hide resolved
weed/filer/ydb/ydb_store_kv.go Show resolved Hide resolved
@chrislusf
Copy link
Collaborator

Please add some tests similar to weed/filer/etcd/etcd_store_test.go in d5fe5d1

kmlebedev added a commit to kmlebedev/seaweedfs that referenced this pull request May 11, 2022
@kmlebedev kmlebedev mentioned this pull request May 11, 2022
kmlebedev added a commit to kmlebedev/seaweedfs that referenced this pull request May 12, 2022
add some tests

seaweedfs#2996
@kmlebedev kmlebedev mentioned this pull request May 12, 2022
@kmlebedev
Copy link
Contributor Author

add some tests

done #3043

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants