New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated ExportQ and CFM to use new ObserverProvider API #130

Merged
merged 1 commit into from May 3, 2017

Conversation

Projects
None yet
3 participants
@keith-turner
Contributor

keith-turner commented May 2, 2017

Create a new CombineQueue API that replaces the CollisionFreeMap. The new
combineQ uses the new Fluo Observer APIs exclusively. The CollisionFreeMap
only uses the old Fluo Observer APIs and was deprecated.

The ExportQueue was modified to support a Fluent configuration mechanism. This
can only be used when using the new Observer API.

@mikewalch

Looks good. Nice change.

Show outdated Hide outdated docs/combine-queue.md
// Set application properties for the combine queue. These properties are read later by
// the observers running on each worker.
CombineQueue.configure(WcObserverProvider.ID)
.keyType(String.class).valueType(Long.class).buckets(119).finish(fluoConfig);

This comment has been minimized.

@mikewalch

mikewalch May 3, 2017

Member

finish() could be called save()

@mikewalch

mikewalch May 3, 2017

Member

finish() could be called save()

This comment has been minimized.

@keith-turner

keith-turner May 3, 2017

Contributor

I thought about save(), to me has a feeling of I/O. What the method is doing is setting properties, I couldn't think of something short that would express this. One good name is configure(), but then I don't know what to name the first method that starts the fluent chain.

@keith-turner

keith-turner May 3, 2017

Contributor

I thought about save(), to me has a feeling of I/O. What the method is doing is setting properties, I couldn't think of something short that would express this. One good name is configure(), but then I don't know what to name the first method that starts the fluent chain.

This comment has been minimized.

@mikewalch

mikewalch May 3, 2017

Member

Another option is store() but its up to you. I am fine with finish().

@mikewalch

mikewalch May 3, 2017

Member

Another option is store() but its up to you. I am fine with finish().

This comment has been minimized.

@keith-turner

keith-turner May 3, 2017

Contributor

I like store() also thinking of set()

CombineQueue.configure(WcObserverProvider.ID)
        .keyType(String.class).valueType(Long.class).buckets(119).store(fluoConfig);
CombineQueue.configure(WcObserverProvider.ID)
        .keyType(String.class).valueType(Long.class).buckets(119).set(fluoConfig);
@keith-turner

keith-turner May 3, 2017

Contributor

I like store() also thinking of set()

CombineQueue.configure(WcObserverProvider.ID)
        .keyType(String.class).valueType(Long.class).buckets(119).store(fluoConfig);
CombineQueue.configure(WcObserverProvider.ID)
        .keyType(String.class).valueType(Long.class).buckets(119).set(fluoConfig);

This comment has been minimized.

@mikewalch

mikewalch May 3, 2017

Member

Either work for me. They are both better than finish()

@mikewalch

mikewalch May 3, 2017

Member

Either work for me. They are both better than finish()

This comment has been minimized.

@keith-turner

keith-turner May 3, 2017

Contributor

I decided to go with save()

@keith-turner

keith-turner May 3, 2017

Contributor

I decided to go with save()

wccq.add(tx, changes);
// update the current content and delete the new content
tx.set(row, CUR_COL, newContent);

This comment has been minimized.

@mikewalch

mikewalch May 3, 2017

Member

shouldn't this be NEW_COL and the delete be CUR_COL

@mikewalch

mikewalch May 3, 2017

Member

shouldn't this be NEW_COL and the delete be CUR_COL

This comment has been minimized.

@keith-turner

keith-turner May 3, 2017

Contributor

I think this is correct. The new becomes current once its been processed.

@keith-turner

keith-turner May 3, 2017

Contributor

I think this is correct. The new becomes current once its been processed.

This comment has been minimized.

@mikewalch

mikewalch May 3, 2017

Member

ok i see. you are right.

@mikewalch

mikewalch May 3, 2017

Member

ok i see. you are right.

Show outdated Hide outdated docs/combine-queue.md
}
};
// Register observer that handles updates to the CollisionFreeMap. This observer will use the

This comment has been minimized.

@mikewalch

mikewalch May 3, 2017

Member

replace CFM with CQ

@mikewalch

mikewalch May 3, 2017

Member

replace CFM with CQ

Show outdated Hide outdated docs/combine-queue.md
// Create a combine queue for computing word counts.
CombineQueue<String, Long> wcMap = CombineQueue.getInstance(ID, ctx.getAppConfiguration());
// Register observer that updates the CollisionsFreeMap

This comment has been minimized.

@mikewalch

mikewalch May 3, 2017

Member

use CombineQueue

@mikewalch

mikewalch May 3, 2017

Member

use CombineQueue

@mikewalch

This comment has been minimized.

Show comment
Hide comment
Member

mikewalch commented May 3, 2017

+1

Updated ExportQ and CFM to use new ObserverProvider API
Created a new CombineQueue API that replaces the CollisionFreeMap.  The new
combineQ uses the new Fluo Observer APIs exclusively.  The CollisionFreeMap
only uses the old Fluo Observer APIs and was deprecated.

The ExportQueue was modified to support a Fluent configuration mechanism. This
can only be used when using the new Observer API.

@asfgit asfgit merged commit 70a4239 into apache:master May 3, 2017

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

@keith-turner keith-turner deleted the keith-turner:observer-factory branch Jun 12, 2017

@keith-turner keith-turner modified the milestone: 1.1.0 Jun 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment