Skip to content

ISSUE #931,#907: Add option to track task execution time#932

Closed
athanatos wants to merge 1 commit intoapache:masterfrom
athanatos:forupstream/issue-931
Closed

ISSUE #931,#907: Add option to track task execution time#932
athanatos wants to merge 1 commit intoapache:masterfrom
athanatos:forupstream/issue-931

Conversation

@athanatos
Copy link

@athanatos athanatos commented Jan 2, 2018

Fixes a bug in OrderedScheduler introduced in
e33ec10 which failed to track execution
time with some calls and adds an option to enable it in the bookie. Also
fixes a bug with task_queued duration.

Add a simple mock for remembering stats long enough to verify that
counters are actually used and sensible in unit tests and bake it into
BookKeeperClusterTestCase so that we can write tests to ensure that the
stats are actually counted and make sense. Use said mock to add simple
tests for top level read and write stats validating this fix.

(@bug W-4276826@)
(@bug W-4268290@)
Signed-off-by: Samuel Just sjust@salesforce.com

Copy link
Member

@sijie sijie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice change. I left some minor comments

}
}

@Test(timeout = 6000)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Drop timeouts we have now in surefire config

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep.

"BookieLongPollThread-" + serverCfg.getBookiePort(), OrderedScheduler.NO_TASK_LIMIT);
"BookieLongPollThread",
OrderedScheduler.NO_TASK_LIMIT,
NullStatsLogger.INSTANCE);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we pass the statslogger here as well?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, no reason not to. Will fix.

++toRemoveIndex;
}
if (toRemove != null) {
stopAutoRecoveryService(toRemove);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line seems to related to the issue

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for typo, it does not seem to be related to the issue

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stopAutoRecoveryService gets called as part of of killBookie. I refactored it a little to make the bs/bsLoggers state maintenance slightly less error prone.

Fixes a bug in OrderedScheduler introduced in
e33ec10 which failed to track execution
time with some callsand adds an option to enable it in the bookie.  Also
fix bug with task_queued duration.

Add a simple mock for remembering stats long enough to verify that
counters are actually used and sensible in unit tests and bake it into
BookKeeperClusterTestCase so that we can write tests to ensure that the
stats are actually counted and make sense.  Use said mock to add simple
tests for top level read and write stats validating this fix.

(@bug W-4276826@)
(@bug W-4268290@)
Signed-off-by: Samuel Just <sjust@salesforce.com>
@athanatos athanatos force-pushed the forupstream/issue-931 branch from b00a3de to dd15a5d Compare January 3, 2018 18:15
@athanatos
Copy link
Author

@eolivelli I think I've addressed your other comments.

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 lgtm. Thank you

@sijie sijie added this to the 4.7.0 milestone Jan 5, 2018
@sijie sijie closed this in 477ff2b Jan 5, 2018
sijie pushed a commit that referenced this pull request Jan 5, 2018
Fixes a bug in OrderedScheduler introduced in
e33ec10 which failed to track execution
time with some calls and adds an option to enable it in the bookie.  Also
fixes a bug with task_queued duration.

Add a simple mock for remembering stats long enough to verify that
counters are actually used and sensible in unit tests and bake it into
BookKeeperClusterTestCase so that we can write tests to ensure that the
stats are actually counted and make sense.  Use said mock to add simple
tests for top level read and write stats validating this fix.

(bug W-4276826)
(bug W-4268290)
Signed-off-by: Samuel Just <sjustsalesforce.com>

Author: Samuel Just <sjust@salesforce.com>

Reviewers: Enrico Olivelli <eolivelli@gmail.com>, Sijie Guo <sijie@apache.org>

This closes #932 from athanatos/forupstream/issue-931, closes #931, closes #907
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants