Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also compare across forks.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also compare across forks.
...
  • 2 commits
  • 2 files changed
  • 0 commit comments
  • 2 contributors
Commits on Jul 21, 2014
@wolfgangmm wolfgangmm [performance] ngram index: method for computing string offsets by uni…
…code code points turned out to be a major bottleneck. Replacing the code to avoid String.offsetByCodePoints improves indexing performance by at least factor 2.
340364c
@ljo ljo Merge pull request #275 from wolfgangmm/develop
[performance] ngram index: method for computing string offsets
20292fc
View
14 extensions/indexes/ngram/src/org/exist/indexing/ngram/NGramIndexWorker.java
@@ -578,18 +578,18 @@ public StoredNode getReindexRoot(StoredNode node, NodePath path, boolean insert,
}
private void indexText(NodeId nodeId, QName qname, String text) {
- String[] ngram = tokenize(text);
- int len = ngram.length;
- for (int i = 0; i < len; i++) {
- int offset = text.offsetByCodePoints(0, i);
- QNameTerm key = new QNameTerm(qname, ngram[i]);
+ final String[] ngram = tokenize(text);
+ final int len = text.length();
+ for (int i = 0, j = 0, cp; i < len; i += Character.charCount(cp), j++) {
+ cp = text.codePointAt(i);
+ final QNameTerm key = new QNameTerm(qname, ngram[j]);
OccurrenceList list = ngrams.get(key);
if (list == null) {
list = new OccurrenceList();
- list.add(nodeId, offset);
+ list.add(nodeId, i);
ngrams.put(key, list);
} else {
- list.add(nodeId, offset);
+ list.add(nodeId, i);
}
}
}
View
1  extensions/indexes/ngram/src/org/exist/xquery/modules/ngram/NGramSearch.java
@@ -136,6 +136,7 @@ public NGramSearch(XQueryContext context, FunctionSignature signature) {
@Override
public void setArguments(List<Expression> arguments) throws XPathException {
+ steps.clear();
Expression path = arguments.get(0);
steps.add(path);

No commit comments for this range

Something went wrong with that request. Please try again.