Skip to content

Commit

Permalink
Resolved Lucene security vulnerability patch
Browse files Browse the repository at this point in the history
Resolved spring core security vulnerability patch

Bumping lucene forced us to implicitly support long return values.  However the Lucene search predicated don't all support long per-page values and some of our pagination implementations need to be able to handle all results in one page.

Since we don't anticipate larger than Integer.MAX_VALUE documents from lucene, we're simply going to validate that the returning hit
count is less than that and throw an exception if it isn't. (+1 squashed commit)
  • Loading branch information
michael-mclawhorn committed Nov 1, 2018
1 parent 2ef1b40 commit a0bf7c4
Show file tree
Hide file tree
Showing 3 changed files with 25 additions and 3 deletions.
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,14 @@ Current
* Fixed bad format in error message
* Moved tests off of serialization of `SimplifiedIntervalList`. That's turning out to be hard to solve.

- [Bump lucene version to patch vulnerability](https://github.com/yahoo/fili/issues/819)
* Bumped dependency version to 7.5.0
* Added error in case of greater than maxint hits from Lucene

- [Bump spring code to patch vulnerability](https://github.com/yahoo/fili/issues/820)
* Bumped dependency version to [5.1.2,)
* Throw validation error if excessive documents are returned from Lucene now that it supports up to long hitcounts

### Known Issues:

### Added:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ public class LuceneSearchProvider implements SearchProvider {
SYSTEM_CONFIG.getPackageVariableName("lucene_search_timeout_ms"), 600000
);

public static String TOO_MANY_DOCUMENTS = "Unexpectedly large response from search provider. Found %l hits.";

/**
* The maximum number of results per page.
*/
Expand Down Expand Up @@ -700,7 +702,17 @@ private Pagination<DimensionRow> getResultsPage(Query query, PaginationParameter
perPage
);
hits = hitDocs.scoreDocs;
documentCount = hitDocs.totalHits;
// The change to supprt long document sizes is incompletely supported in Lucene
// Since we can't request up to long documents we'll only expect to receive up to Integer.MAX_VALUE
// responses, and throw an error if we exceed that.
if (hitDocs.totalHits > Integer.MAX_VALUE) {
String message = String.format(TOO_MANY_DOCUMENTS, hitDocs.totalHits);
RowLimitReachedException exception = new RowLimitReachedException(message);
LOG.error(exception.getMessage(), exception);
throw exception;
}
documentCount = (int) hitDocs.totalHits;

int requestedPageNumber = paginationParameters.getPage(documentCount);
if (hits.length == 0) {
if (requestedPageNumber == 1) {
Expand Down
6 changes: 4 additions & 2 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,9 @@
<version.hk2>2.5.0-b36</version.hk2>
<version.metrics>3.2.2</version.metrics>
<version.logback>1.2.3</version.logback>
<version.lucene>6.6.0</version.lucene>
<version.lucene>7.5.0</version.lucene>

<version.spring>[5.1.2,)</version.spring>
<version.groovy>2.4.15</version.groovy>
<version.jackson>2.9.5</version.jackson>

Expand Down Expand Up @@ -195,7 +197,7 @@
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>4.3.4.RELEASE</version>
<version>${version.spring}</version>
</dependency>

<!-- Test -->
Expand Down

0 comments on commit a0bf7c4

Please sign in to comment.