Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3x tweaks #1563

Closed
wants to merge 2 commits into from

Conversation

Projects
None yet
3 participants
@manolama
Copy link
Member

commented Mar 15, 2019

No description provided.

manolama added some commits Mar 15, 2019

ASYNCHBASE/BIGTABLE
- Fix a concurrency issue wherein Trove maps are not thread safe and the ID resolution
  was returning sporadic results.
COMMON:
- Add QuerySinkCallback and SerdesCallback to avoid deferreds with the new
  pipeline. Save objects!
- Add the data() method and remove iterator() from PartialTimeSeries.
- Add a complete() method to PartialTimeSeriesSet so upstream knows when a
  set is finished.
- Remove onComplete(PartialTimeSeriesSet) from the QueryNode and QuerySink
  since we can now use just the one callback. Cleaner.

CORE:
- Add the NumericLongArrayType which is a more CPU cache friendly array of
  timestamps and values. It means we're moving away from iterators *sniff*.
  Makes things uglier but faster.
- AbstractQueryPipelineContext now keeps track fo the partial time series and
  callbacks from sinks so we can determine when to close the query. This since
  code is pretty finicky.
  TODO - still more work to do here around exceptions and handling multiple
  sinks.
- Fix serialization in JsonV3QuerySerdes for the PartialTimeSeries.
- Fix MockDataStore for pushing data and make sure it works multi-threaded
  as AsyncHBase will have to.
  TODO - move the NumericLongArrayType code to a static class.
@gskchaitanya
Copy link
Collaborator

left a comment

👍

@manolama

This comment has been minimized.

Copy link
Member Author

commented Mar 15, 2019

Merged in a35c3d4

@manolama manolama closed this Mar 15, 2019

if (this.callback == null) {
synchronized (this) {
if (this.callback == null) {
this.callback = callback;

This comment has been minimized.

Copy link
@SeanPMiller

SeanPMiller Mar 18, 2019

DCLP here means the this.callback member must be marked volatile to prevent operation reordering.

This comment has been minimized.

Copy link
@manolama

manolama Mar 18, 2019

Author Member

Thanks for the catch.

}

if (this.callback != callback) {
// TODO WTF?

This comment has been minimized.

Copy link
@SeanPMiller

SeanPMiller Mar 18, 2019

This will be true if onNext() is called multiple times on a given object with different callback arguments. Might be a logic error?

This comment has been minimized.

Copy link
@manolama

manolama Mar 18, 2019

Author Member

Yeah there should only ever be the one callback.

@@ -670,13 +674,19 @@ void close() {
* clean up quicker. */
private void clear() {
if (keys_to_ids != null) {
keys_to_ids.clear();
synchronized (keys_to_ids) {

This comment has been minimized.

Copy link
@SeanPMiller

SeanPMiller Mar 18, 2019

Seeing three successive critical sections makes me nervous, but I don't see any issues...

}
}
}
// if (set.set == null) {

This comment has been minimized.

Copy link
@SeanPMiller
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.