Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Optimistic tx semantics #52

Merged
merged 22 commits into from

3 participants

@snoopdave

Work complete for optimistic transaction semantics
https://issues.apache.org/jira/browse/USERGRID-52

snoopdave added some commits
@snoopdave snoopdave Minor formating. 3d6461a
@snoopdave snoopdave Merge branch 'two-dot-o' into optimistic-tx-semantics 6178ec0
@snoopdave snoopdave First stab at optimistic verification 5bfacf0
@snoopdave snoopdave Merge branch 'two-dot-o' into optimistic-tx-semantics 13120f5
@snoopdave snoopdave Progress on optimistic verify and test. ce9e895
@snoopdave snoopdave Merge branch 'two-dot-o' into optimistic-tx-semantics
Conflicts:
	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/guice/CollectionModule.java
	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
7d3a9c7
@snoopdave snoopdave shell of a WriteOptimisticVerifyTest now in place, but it does not re…
…ally test anything.
c04f49d
@snoopdave snoopdave Merge branch 'two-dot-o' into optimistic-tx-semantics
Conflicts:
	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/guice/CollectionModule.java
	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
1d06007
@snoopdave snoopdave Separate WriteOptimisticVerify test into to parts, one Theories part …
…and one Jukito part.
6365279
@snoopdave snoopdave Use mocks instead of real live serialization strategy. 4d1b7c0
@zznate

@snoopdave take a look at:

http://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-and-the-perils-of-network-partitions

https://github.com/aphyr/jepsen

A lot there, but does describe in detail an excellent way to break test transactional stuff.

snoopdave added some commits
@snoopdave snoopdave Merge branch 'two-dot-o' into optimistic-tx-semantics
Conflicts:
	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
	stack/corepersistence/collection/src/test/java/org/apache/usergrid/persistence/collection/guice/TestCollectionModule.java
10f757c
@snoopdave snoopdave Fix broken test. 717200b
@snoopdave snoopdave Operation specific exception types. e072f0b
@snoopdave snoopdave We now cleanup and unique values on version conflict; also, added bet…
…ter tests for optimistic and unique verification.
29a4966
@snoopdave snoopdave Operation now removes self from log instead of writing a rollback entry. f7a5072
@tnine

What about the deletes of the unique properties as well? I think we'll need to do them too, otherwise you'll get incorrect duplicate property exceptions if you try to write again <= timeout on the columns of unique properties

That's there now. Take a look at the previous commits.

snoopdave added some commits
@snoopdave snoopdave New RollbackAction and exception handling changes. 8c4d8dd
@snoopdave snoopdave Merge branch 'two-dot-o' into optimistic-tx-semantics
Conflicts:
	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/guice/CollectionModule.java
	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/mvcc/stage/write/WriteOptimisticVerify.java
	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/mvcc/stage/write/WriteUniqueVerify.java
	stack/corepersistence/collection/src/test/java/org/apache/usergrid/persistence/collection/util/EntityBuilder.java
	stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/legacy/EntityManagerFacade.java
	stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/utils/EntityBuilder.java
	stack/corepersistence/queryindex/src/test/java/org/apache/usergrid/persistence/index/impl/EntityCollectionIndexTest.java
	stack/corepersistence/queryindex/src/test/java/org/apache/usergrid/persistence/index/legacy/EntityBuilder.java
b19e45e
@snoopdave snoopdave Rollback of MVCC log and UniqueValues now working and verified by test. 5d99e0a
@snoopdave

@tnine the RollbackAction is now hooked in via RxJava and I think this is ready to merge with two-dot-o. Please take another look when you get a chance.

...tence/collection/mvcc/stage/write/RollbackAction.java
((68 lines not shown))
+ final Entity entity = cre.getEntity();
+ final CollectionScope scope = cre.getCollectionScope();
+
+ // Delete all unique values of entity, and do it concurrently
+ List<Observable<FieldDeleteResult>> results
+ = new ArrayList<Observable<FieldDeleteResult>>();
+
+ int uniqueFieldCount = 0;
+ for (final Field field : entity.getFields()) {
+
+ // if it's unique, create a function to delete it
+ if (field.isUnique()) {
+
+ uniqueFieldCount++;
+
+ Observable<FieldDeleteResult> result = Observable.from(field)
@tnine Owner
tnine added a note

Rather than do this in parallel for each rollback, it would actually probably be more performant to roll them all into 1 single mutation batch for a single network call. Then you wouldn't need to mess with the observables here in this state since it would be a single network call. Thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...tence/collection/mvcc/stage/write/RollbackAction.java
((117 lines not shown))
+ + "{} version: {} name: {}",
+ new String[]{
+ entity.getId().toString(),
+ entity.getVersion().toString(),
+ result.getName()
+ });
+ }
+ return true;
+ }
+ };
+
+ // "zip" up the concurrent results
+ Observable.zip(results, zipFunction).toBlockingObservable().last();
+ }
+
+ logEntryStrat.delete( scope, entity.getId(), entity.getVersion() );
@tnine Owner
tnine added a note

The mutation needs executed here. Again, this could be rolled in the batch with all the unique fields.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@tnine tnine commented on the diff
.../collection/mvcc/stage/write/WriteUniqueVerifyIT.java
((76 lines not shown))
+ entityManager.write( entity ).toBlockingObservable().last();
+
+ // another enity that tries to use two unique values already taken by first
+ final Entity entity2 = TestEntityGenerator.generateEntity();
+ entity2.setField(new StringField("name", "Aston Martin Vanquish", true));
+ entity2.setField(new StringField("identifier", "v12", true));
+ entity2.setField(new IntegerField("top_speed_mph", 120));
+
+ try {
+ entityManager.write( entity2 ).toBlockingObservable().last();
+ fail("Write should have thrown an exception");
+
+ } catch ( WriteUniqueVerifyException e ) {
+ // verify two unique value violations
+ assertEquals( 2, e.getVioliations().size() );
+ }
@tnine Owner
tnine added a note

Might want to attempt to update the original entity here. Just to ensure all the unique values were removed from the CF's and that nothing is blocking an update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
snoopdave added some commits
@snoopdave snoopdave Merge branch 'two-dot-o' into optimistic-tx-semantics
Conflicts:
	stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/legacy/EntityManagerFacade.java
b73cb58
@snoopdave snoopdave Merge branch 'two-dot-o' into optimistic-tx-semantics
Conflicts:
	stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/legacy/EntityManagerFacade.java
82ce860
@snoopdave snoopdave Use one mutation batch instead of concurrent ops in Rollback action, …
…and add update to end of test.
3719d4f
@snoopdave snoopdave Use one mutation batch instead of concurrent ops in Rollback action, …
…and add update to end of test.
13ff924
@snoopdave

Thanks @tnine. I made those suggested changes.

@tnine tnine merged commit d23254c into two-dot-o
@tnine tnine deleted the optimistic-tx-semantics branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jan 21, 2014
  1. @snoopdave

    Minor formating.

    snoopdave authored
  2. @snoopdave
Commits on Jan 27, 2014
  1. @snoopdave
  2. @snoopdave
Commits on Jan 29, 2014
  1. @snoopdave
Commits on Feb 12, 2014
  1. @snoopdave

    Merge branch 'two-dot-o' into optimistic-tx-semantics

    snoopdave authored
    Conflicts:
    	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/guice/CollectionModule.java
    	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
  2. @snoopdave
Commits on Feb 13, 2014
  1. @snoopdave

    Merge branch 'two-dot-o' into optimistic-tx-semantics

    snoopdave authored
    Conflicts:
    	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/guice/CollectionModule.java
    	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
  2. @snoopdave
  3. @snoopdave
Commits on Mar 11, 2014
  1. @snoopdave

    Merge branch 'two-dot-o' into optimistic-tx-semantics

    snoopdave authored
    Conflicts:
    	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
    	stack/corepersistence/collection/src/test/java/org/apache/usergrid/persistence/collection/guice/TestCollectionModule.java
  2. @snoopdave

    Fix broken test.

    snoopdave authored
  3. @snoopdave
Commits on Mar 12, 2014
  1. @snoopdave

    We now cleanup and unique values on version conflict; also, added bet…

    snoopdave authored
    …ter tests for optimistic and unique verification.
  2. @snoopdave
Commits on Mar 13, 2014
  1. @snoopdave
Commits on Apr 15, 2014
  1. @snoopdave

    Merge branch 'two-dot-o' into optimistic-tx-semantics

    snoopdave authored
    Conflicts:
    	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/guice/CollectionModule.java
    	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
    	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/mvcc/stage/write/WriteOptimisticVerify.java
    	stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/mvcc/stage/write/WriteUniqueVerify.java
    	stack/corepersistence/collection/src/test/java/org/apache/usergrid/persistence/collection/util/EntityBuilder.java
    	stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/legacy/EntityManagerFacade.java
    	stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/utils/EntityBuilder.java
    	stack/corepersistence/queryindex/src/test/java/org/apache/usergrid/persistence/index/impl/EntityCollectionIndexTest.java
    	stack/corepersistence/queryindex/src/test/java/org/apache/usergrid/persistence/index/legacy/EntityBuilder.java
Commits on Apr 16, 2014
  1. @snoopdave
Commits on Apr 21, 2014
  1. @snoopdave

    Merge branch 'two-dot-o' into optimistic-tx-semantics

    snoopdave authored
    Conflicts:
    	stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/legacy/EntityManagerFacade.java
  2. @snoopdave

    Merge branch 'two-dot-o' into optimistic-tx-semantics

    snoopdave authored
    Conflicts:
    	stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/legacy/EntityManagerFacade.java
  3. @snoopdave

    Use one mutation batch instead of concurrent ops in Rollback action, …

    snoopdave authored
    …and add update to end of test.
  4. @snoopdave

    Use one mutation batch instead of concurrent ops in Rollback action, …

    snoopdave authored
    …and add update to end of test.
Something went wrong with that request. Please try again.