Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix/internal issue #566 #9497

Merged
merged 16 commits into from Jul 22, 2019
Merged

Bug fix/internal issue #566 #9497

merged 16 commits into from Jul 22, 2019

Conversation

Dronplane
Copy link
Contributor

New behavior for TOKENS function.
Added ability to tokenize numbers, nulls and booleans with corresponding analyzers (terms returned as Base64Encoded)
Analyzer name (second parameter) is now optional (identity used by default)
Added ability to process arrays (possible nested)
Added tests for new functionality (js and GTEST)

https://jenkins.arangodb.biz/view/PR/job/arangodb-matrix-pr/5262/

@Dronplane Dronplane requested review from gnusi and goedderz July 17, 2019 15:56
@goedderz
Copy link
Member

Is there documentation for the new behaviours / functionality? Anything that needs to be mentioned in UpgradingChanges?

@Dronplane
Copy link
Contributor Author

Implementation is backward compatible, e.g. all existing queries utilizing TOKENS functions will behave the same way as before. So, I assume there is nothing to mention in UpgradeChanges? Documentation will be updated with separate PR.

arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
} else if (current.isBool()) {
arangodb::iresearch::addStringRef(
builder, arangodb::basics::StringUtils::encodeBase64(irs::ref_cast<char>(
irs::boolean_token_stream::value(current.getBoolean()))));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea what this should do. At the very least I've got to see the documentation first.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will give us internal analyzer representation for given value, just like with strings

arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
arangod/IResearch/IResearchAnalyzerFeature.cpp Outdated Show resolved Hide resolved
@Dronplane
Copy link
Contributor Author

@Dronplane Dronplane requested a review from gnusi July 19, 2019 12:22
@gnusi gnusi merged commit 11ef4bf into devel Jul 22, 2019
ObiWahn added a commit that referenced this pull request Jul 23, 2019
…-fix/optimization-of-nested-collect

* 'devel' of https://github.com/arangodb/arangodb:
  fix asan issue (#9540)
  Docs: Add missing Cookbook placeholder folder
  Remove documentation, now here: github.com/arangodb/docs (#8918)
  Revert "Fix name of example graph in k Shortest Paths documentation (#8972)"
  Docs: Revert more accidental changes
  fixed an issue with smart joins if both collections were restricted to a (#9535)
  Remove some minReplicationFactor doc changes, fix indent
  Bug fix/internal issue #566 (#9497)
ObiWahn added a commit that referenced this pull request Jul 24, 2019
…ture/one-shard-db

* 'devel' of https://github.com/arangodb/arangodb:
  fix asan issue (#9540)
  Docs: Add missing Cookbook placeholder folder
  Remove documentation, now here: github.com/arangodb/docs (#8918)
  Revert "Fix name of example graph in k Shortest Paths documentation (#8972)"
  Docs: Revert more accidental changes
  fixed an issue with smart joins if both collections were restricted to a (#9535)
  Remove some minReplicationFactor doc changes, fix indent
  Bug fix/internal issue #566 (#9497)
@fceller fceller deleted the bug-fix/internal-issue-#566 branch July 30, 2019 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants