LuceneSearcher redesign and fix memory leak #8112

skabashnyuk · 2018-01-02T06:55:06Z

What does this PR do?

LuceneSearcher redesign and fix memory leak

org.eclipse.che.api.search.server.Searcher reworked exceptions. Instead of transport exceptions, it will throw search specific exceptions.
OffsetData moved to separate class, outside from LuceneSearcher.
QueryExpression and OffsetData now on the same level as Searcher interface
Used dedicated one-time used demon thread for initial indexing instead of thread pool
Added CountDownLatch to test if initial indexing complete.
Removed dependency on org.eclipse.jface.text in offset calculation
Used Files.walkFileTree to simplify add directory method
added printStatistic method to put some statistics in log.debug output
optimized delete method to the fact that is called after folder or file removal
Called indexWriter.commit after adding, update, removal of file or folder
Added implementation agnostic test for org.eclipse.che.api.search.server.Searcher
Added specific to LuceneSearche test

What issues does this PR fix or reference?

Fixes #8111

Release Notes

LuceneSearcher redesign and fix memory leak

Docs PR

n/a

garagatyi · 2018-01-04T10:58:24Z

wsagent/che-core-api-project/src/test/java/org/eclipse/che/api/search/SearcherTest.java

+
+  @BeforeMethod
+  public void setUp() throws Exception {
+    indexDirectory = Files.createTempDir(); // Paths.get("/tmp/indexdir").toFile();


Is this comment needed?

garagatyi · 2018-01-04T11:24:34Z

...ore-api-project/src/main/java/org/eclipse/che/api/project/server/impl/ProjectServiceApi.java

@@ -680,7 +692,7 @@ public ProjectSearchResponseDto search(ProjectSearchRequestDto request) {

    try {
      return search(path, name, text, maxItems, skipCount);
-    } catch (ServerException | ConflictException | NotFoundException | ForbiddenException e) {
+    } catch (ServerException | NotFoundException | BadRequestException e) {


Not sure it makes sense to throw these exceptions in other code to lose the information about exception type here.

…E_MAVEN__SERVER__JAVA__OPTIONS looks like for end user

skabashnyuk · 2018-01-05T21:19:21Z

ci-test

codenvy-ci · 2018-01-06T06:18:39Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: riuvshin/che-server:8112
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

skabashnyuk · 2018-01-06T12:10:27Z

ci-test

codenvy-ci · 2018-01-08T06:49:04Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: riuvshin/che-server:8112
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

skabashnyuk · 2018-01-11T09:27:04Z

ci-test

codenvy-ci · 2018-01-11T14:55:38Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: eclipseche/che-server:8112
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

skabashnyuk · 2018-01-11T15:12:08Z

ci-test

codenvy-ci · 2018-01-11T21:14:59Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: eclipseche/che-server:8112
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

skabashnyuk · 2018-01-12T06:41:03Z

ci-test

codenvy-ci · 2018-01-12T12:46:35Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: eclipseche/che-server:8112
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

mkuznyetsov · 2018-02-02T15:14:58Z

ci-test

codenvy-ci · 2018-02-02T21:41:34Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: eclipseche/che-server:8112
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

skabashnyuk added 10 commits December 30, 2017 23:12

Initial refactor

0b47181

More tests

f2fa2f3

Added more test

166a007

More test + simplified assert

4f9f53b

Added more test by name

7697b1e

Added searchesByTextAndPath

fff9240

fixup! Added searchesByTextAndPath

36cb797

fixup! Added searchesByTextAndPath

362dd0a

Replace transport exception with search specific

1653d53

Removed usage of org.eclipse.text

74f268d

skabashnyuk requested a review from vparfonov as a code owner January 2, 2018 06:55

skabashnyuk requested review from dkuleshov and garagatyi January 2, 2018 06:55

benoitf added status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. kind/bug Outline of a bug - must adhere to the bug report template. labels Jan 2, 2018

skabashnyuk changed the title ~~[WP] LuceneSearcher redesign and fix memory leak~~ LuceneSearcher redesign and fix memory leak Jan 2, 2018

skabashnyuk added 2 commits January 3, 2018 08:58

Merge remote-tracking branch 'origin/master' into reworkindexsearcher

4e7b91c

Added skabashnyuk as codeowner for server side search classes

64a8561

dkuleshov approved these changes Jan 3, 2018

View reviewed changes

Merge remote-tracking branch 'origin/master' into reworkindexsearcher

447ec79

garagatyi approved these changes Jan 4, 2018

View reviewed changes

skabashnyuk added 6 commits January 5, 2018 09:10

Merge remote-tracking branch 'origin/master' into reworkindexsearcher

a4ed48d

Added score to debug output

f584252

Explicitly specified sorting order by score + path

c3f6375

Merge remote-tracking branch 'origin/master' into reworkindexsearcher

c7623d5

fixup! Sync how CHE_WORKSPACE_WSAGENT__JAVA__OPTIONS and CHE_WORKSPAC…

cc0684b

…E_MAVEN__SERVER__JAVA__OPTIONS looks like for end user

Remove docid from interfaces

78c721a

Merge remote-tracking branch 'origin/master' into reworkindexsearcher

b3322e0

eclipse-che deleted a comment from codenvy-ci Jan 11, 2018

skabashnyuk added 2 commits January 11, 2018 17:05

Merge remote-tracking branch 'origin/master' into reworkindexsearcher

7a2dd14

Update license

5b254f5

vparfonov approved these changes Jan 15, 2018

View reviewed changes

mkuznyetsov added 3 commits January 22, 2018 10:44

Merge branch 'master' of github.com:eclipse/che into reworkindexsearcher

8a37cba

Merge branch 'master' of github.com:eclipse/che into reworkindexsearcher

2d016de

Display correct search result line numbers

342af7d

davidwindell mentioned this pull request Jan 31, 2018

Search fails on certain workspaces #8552

Closed

mkuznyetsov added 3 commits February 2, 2018 16:35

Perform lucene index commit as scheduled task

4d51a92

Merge branch 'master' of github.com:eclipse/che into reworkindexsearcher

c2487b4

fix formatting

fffbe16

Remove unused code

1986888

mkuznyetsov merged commit 70b87d7 into master Feb 5, 2018

mkuznyetsov deleted the reworkindexsearcher branch February 5, 2018 12:46

benoitf removed the status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. label Feb 5, 2018

benoitf added this to the 6.1.0 milestone Feb 5, 2018

skabashnyuk mentioned this pull request Apr 25, 2018

Che IDE locks up #9477

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LuceneSearcher redesign and fix memory leak #8112

LuceneSearcher redesign and fix memory leak #8112

skabashnyuk commented Jan 2, 2018 •

edited

Loading

garagatyi Jan 4, 2018

garagatyi Jan 4, 2018

skabashnyuk commented Jan 5, 2018

codenvy-ci commented Jan 6, 2018 •

edited by riuvshin

Loading

skabashnyuk commented Jan 6, 2018

codenvy-ci commented Jan 8, 2018 •

edited by riuvshin

Loading

skabashnyuk commented Jan 11, 2018

codenvy-ci commented Jan 11, 2018

skabashnyuk commented Jan 11, 2018

codenvy-ci commented Jan 11, 2018

skabashnyuk commented Jan 12, 2018

codenvy-ci commented Jan 12, 2018

mkuznyetsov commented Feb 2, 2018

codenvy-ci commented Feb 2, 2018

LuceneSearcher redesign and fix memory leak #8112

LuceneSearcher redesign and fix memory leak #8112

Conversation

skabashnyuk commented Jan 2, 2018 • edited Loading

What does this PR do?

What issues does this PR fix or reference?

Release Notes

Docs PR

garagatyi Jan 4, 2018

Choose a reason for hiding this comment

garagatyi Jan 4, 2018

Choose a reason for hiding this comment

skabashnyuk commented Jan 5, 2018

codenvy-ci commented Jan 6, 2018 • edited by riuvshin Loading

skabashnyuk commented Jan 6, 2018

codenvy-ci commented Jan 8, 2018 • edited by riuvshin Loading

skabashnyuk commented Jan 11, 2018

codenvy-ci commented Jan 11, 2018

skabashnyuk commented Jan 11, 2018

codenvy-ci commented Jan 11, 2018

skabashnyuk commented Jan 12, 2018

codenvy-ci commented Jan 12, 2018

mkuznyetsov commented Feb 2, 2018

codenvy-ci commented Feb 2, 2018

skabashnyuk commented Jan 2, 2018 •

edited

Loading

codenvy-ci commented Jan 6, 2018 •

edited by riuvshin

Loading

codenvy-ci commented Jan 8, 2018 •

edited by riuvshin

Loading