Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to do full text search on vector of references #1616

Closed
yssrku opened this issue Sep 21, 2023 · 2 comments
Closed

Unable to do full text search on vector of references #1616

yssrku opened this issue Sep 21, 2023 · 2 comments

Comments

@yssrku
Copy link

yssrku commented Sep 21, 2023

What happend?

This is more likely just lack of error / docs.

Some document suggest using vector of reference to do tags.
I also see full text search is supported on vector columns

I want to do both exact match and full text search on tags, so I tried combing both.
The result is it just not work without any error.

Currently the only way to do this seems to be using vector of string, do full text search and use vector_find for exact match.
I'm wondering if there's a better way.
Thanks!

How to reproduce it

table_create Reports TABLE_HASH_KEY ShortText
table_create ReportTags TABLE_HASH_KEY ShortText
table_create ReportsIndex TABLE_PAT_KEY ShortText --default_tokenizer TokenBigram --normalizer NormalizerAuto

column_create Reports tags COLUMN_VECTOR ReportTags

column_create ReportsIndex index_tags COLUMN_INDEX|WITH_POSITION|WITH_SECTION --type Reports --source tags

load --table Reports
[
{"_key":"1","tags":["All News","sanctions","State-Sponsored","Hacktivist","South Korea","Maritime"]}
]

select Reports --query tags:@sanctions

Expected behavior

Perform full text search or report error.

Environment

  • OS: MacOS
  • OS Version: 11.6.1
  • Groonga version: 13.0.7

Additional context

No response

@yssrku yssrku added the Bug label Sep 21, 2023
@kou
Copy link
Member

kou commented Sep 21, 2023

Could you try this?

table_create ReportTags TABLE_HASH_KEY ShortText
# For full text search.
table_create ReportTagsIndex TABLE_PAT_KEY ShortText --default_tokenizer TokenBigram --normalizer NormalizerAuto
# Text data exists in ReportTags._key. So --source of
# ReportTagsIndex.index_tags is "_key". We don't need WITH_SECTION
# because ReportTags._key is a scalar value.
column_create ReportTagsIndex index_tags COLUMN_INDEX|WITH_POSITION --type ReportTags --source _key

table_create Reports TABLE_HASH_KEY ShortText
column_create Reports tags COLUMN_VECTOR ReportTags

# For exact match. We add an index column to ReportTags.
# We don't need to create a new table for exact match
# against reference column.
# (Reports.tags's type is a table (ReportTags).)
# We don't need WITH_POSITION because this is
# not for a full text search. We don't need WITH_SECTION too because
# we don't need to care about offset in "Reports.tags" for exact match.
column_create ReportTags index_report COLUMN_INDEX Reports tags

load --table Reports
[
{"_key":"1","tags":["All News","sanctions","State-Sponsored","Hacktivist","South Korea","Maritime"]}
]

# You can see which index column is used by "select" in groonga.log.
# log_level debug

# Full text search. "._key" is important. ReportTagsIndex.index_tags
# is used.
select Reports --query tags._key:@korea

# Exact match. ReportTags.index_report is used.
select Reports --query tags:@sanctions

@kou kou removed the Bug label Sep 21, 2023
@yssrku
Copy link
Author

yssrku commented Sep 22, 2023

It works! I didn't know groonga can do tags._key, awesome!!

@yssrku yssrku closed this as completed Sep 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants