REVIEW: NXCM-4867 - Index update concurrency problem #848

Merged
merged 9 commits into from May 3, 2013

Conversation

Projects
None yet
4 participants
@scarlucci
Contributor

scarlucci commented Apr 30, 2013

https://issues.sonatype.org/browse/NXCM-4867

These changes fix a synchronization problem caused by attempting an incremental index update when an incremental update is not possible, and a full index update must be performed. In this case, since the maven indexer did not support a way to request an incremental-only update, it was not possible to know the appropriate locking mechanism to use for the update.

Previously, we were grabbing a read-only lock to perform the incremental update. But in cases where an incremental was not possible, and a full index update was performed, there was a window towards the end of the full index update where another thread attempting to read the index could experience an NPE.

The maven indexer has been updated to allow clients to specify an incremental-only index update. These changes take advantage of this new support. If an incremental update is not possible, then a full index update is performed with the appropriate locking

CI build on the way

scarlucci added some commits Apr 11, 2013

Switch to new pseudo-release of maven indexer
Switch to new pseudo-release of maven indexer, which adds support for
running an incremental-only index update
Merge remote-tracking branch 'origin/master' into NXCM-4867-indexer-s…
…ync-simple

Conflicts:
	plugins/indexer/nexus-indexer-lucene-plugin/pom.xml
	plugins/indexer/nexus-indexer-lucene-plugin/src/main/java/org/sonatype/nexus/index/DefaultIndexerManager.java
@@ -2800,4 +2858,14 @@ public IndexingContext getRepositoryIndexContext( String repositoryId )
{
return mavenIndexer.getIndexingContexts().get( getContextId( repositoryId ) );
}
+

This comment has been minimized.

@ifedorenko

ifedorenko Apr 30, 2013

Contributor

I recommend changing exception class name to IndexUpdateException. Please also mark the class as static as it does not and should not need access any member of the outer class.

@ifedorenko

ifedorenko Apr 30, 2013

Contributor

I recommend changing exception class name to IndexUpdateException. Please also mark the class as static as it does not and should not need access any member of the outer class.

This comment has been minimized.

@scarlucci

scarlucci Apr 30, 2013

Contributor

Changed the class to be static. Left the name to be as explicit as possible about the conditions under which this exception is used, but can be convinced to change if someone has a strong opinion about this name

@scarlucci

scarlucci Apr 30, 2013

Contributor

Changed the class to be static. Left the name to be as explicit as possible about the conditions under which this exception is used, but can be convinced to change if someone has a strong opinion about this name

@ifedorenko

This comment has been minimized.

Show comment
Hide comment
@ifedorenko

ifedorenko Apr 30, 2013

Contributor

+1

Couple of cosmetic comments, but I don't feel too strong about them so feel free to ignore.

Contributor

ifedorenko commented Apr 30, 2013

+1

Couple of cosmetic comments, but I don't feel too strong about them so feel free to ignore.

@@ -30,8 +30,7 @@
<properties>
<pluginName>Nexus Indexer Lucene Plugin</pluginName>
<pluginDescription>Adds search capabilities for repository content.</pluginDescription>
-
- <mavenIndexer.version>5.1.2-c65990c</mavenIndexer.version>
+ <mavenIndexer.version>5.1.2-bf3d48b</mavenIndexer.version>

This comment has been minimized.

@kellyrob99

kellyrob99 Apr 30, 2013

Member

Just out of curiousity, why the strange version number?

@kellyrob99

kellyrob99 Apr 30, 2013

Member

Just out of curiousity, why the strange version number?

This comment has been minimized.

@cstamas

cstamas Apr 30, 2013

Contributor

These are "pseudo" releases of ASF Maven Indexer. ASF MI 5.1.1 was released last, codebase is currently on 5.1.2-SNAPSHOT. The "bf3d48b" suffix is basically the Git sha1 commit hash out of which the pseudo release of MI was done.

@cstamas

cstamas Apr 30, 2013

Contributor

These are "pseudo" releases of ASF Maven Indexer. ASF MI 5.1.1 was released last, codebase is currently on 5.1.2-SNAPSHOT. The "bf3d48b" suffix is basically the Git sha1 commit hash out of which the pseudo release of MI was done.

Change inner class to static
Inner class does not need access to outer class
@kellyrob99

This comment has been minimized.

Show comment
Hide comment
@kellyrob99

kellyrob99 May 1, 2013

Member

+1

Member

kellyrob99 commented May 1, 2013

+1

Update downloadRepositoryIndex to conform to new indexer semantics
The maven indexer semantics were changed to fail if an incremental only
update was requested, but not possible. Previously, it would move on to
a full index update if an incremental failed. downloadRepositoryIndex
needed to be updated to account for this. To keep the behavior the same,
downloadRepositoryIndex will handle a failed incremental only update,
and try a full index update.
@ifedorenko

This comment has been minimized.

Show comment
Hide comment
@ifedorenko

ifedorenko May 3, 2013

Contributor

+1

Please merge this if the feature build is successful

Contributor

ifedorenko commented May 3, 2013

+1

Please merge this if the feature build is successful

scarlucci added a commit that referenced this pull request May 3, 2013

Merge pull request #848 from sonatype/NXCM-4867-indexer-sync-simple
REVIEW: NXCM-4867 - Index update concurrency problem

@scarlucci scarlucci merged commit ec06047 into master May 3, 2013

@scarlucci scarlucci deleted the NXCM-4867-indexer-sync-simple branch May 3, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment