Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: send attempt and timestamp in headers #935

Merged
merged 14 commits into from Oct 29, 2021
Merged

Conversation

mutianf
Copy link
Contributor

@mutianf mutianf commented Aug 4, 2021

Add attempt number and client timestamp to request headers so on the server side we can collect these client metrics.
BigtableTracer will record attempt number when #attemptStart(int attemptNubmer) is called. Added a getter to get this attemptNumber so we can access it from ApiCallContext in the callable chain.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

@product-auto-label product-auto-label bot added the api: bigtable label Aug 4, 2021
@google-cla google-cla bot added the cla: yes label Aug 4, 2021
@@ -152,4 +156,8 @@ public void batchRequestSent(long elementCount, long requestSize) {
child.batchRequestSent(elementCount, requestSize);
}
}

public int getAttempt() {
Copy link
Contributor

@igorbernstein2 igorbernstein2 Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doc please...without context this is very confusing :)

@Override
public ApiFuture futureCall(RequestT request, ApiCallContext apiCallContext) {
int attemptCount = -1;
if (apiCallContext.getTracer() instanceof CompositeTracer) {
Copy link
Contributor

@igorbernstein2 igorbernstein2 Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when would this be false?

Copy link
Contributor Author

@mutianf mutianf Aug 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't be false, just a sanity check, and so that the EnhancedBigtableStubTest#testCreateReadRowsRawCallable test won't fail

import java.util.List;
import java.util.Map;

/** A callable that injects client timestamp and attempt count to request headers. */
Copy link
Contributor

@igorbernstein2 igorbernstein2 Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a bit about about the other pieces that it interacts with. ie It colborates with the Bigtable/CompositeTracer to identify the retry attempt

Copy link
Contributor

@igorbernstein2 igorbernstein2 Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, please clarify the attempt numbering scheme...is the first attempt 0 or 1?

Copy link
Contributor

@igorbernstein2 igorbernstein2 Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and what does -1 imply?

Copy link
Contributor Author

@mutianf mutianf Aug 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this part to stub/metrics/Util

Added comments to multiple places, attempt starts from 0.

ResponseObserver<ResponseT> responseObserver,
ApiCallContext apiCallContext) {
int attemptCount = -1;
if (apiCallContext.getTracer() instanceof CompositeTracer) {
Copy link
Contributor

@igorbernstein2 igorbernstein2 Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when is this false?

Arrays.asList(String.valueOf(attemptCount)),
EnhancedBigtableStub.TIMESTAMP_HEADER_KEY.name(),
Arrays.asList(String.valueOf(System.currentTimeMillis())));
ApiCallContext newCallContext = apiCallContext.withExtraHeaders(headers);
Copy link
Contributor

@igorbernstein2 igorbernstein2 Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would extract both the key constants and extraction of the api tracer and the creation of the map into a helper class. This way this feature lives in one place and the unary/streaming callables just use it

@mutianf mutianf marked this pull request as ready for review Aug 10, 2021
@mutianf mutianf requested review from as code owners Aug 10, 2021
@mutianf mutianf added the kokoro:force-run label Aug 11, 2021
@yoshi-kokoro yoshi-kokoro removed kokoro:force-run labels Aug 11, 2021
@mutianf mutianf added the kokoro:force-run label Aug 11, 2021
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run label Aug 11, 2021
static final Metadata.Key<String> ATTEMPT_HEADER_KEY =
Metadata.Key.of("attempt", Metadata.ASCII_STRING_MARSHALLER);
static final Metadata.Key<String> TIMESTAMP_HEADER_KEY =
Metadata.Key.of("client-timing", Metadata.ASCII_STRING_MARSHALLER);
Copy link
Contributor

@igorbernstein2 igorbernstein2 Sep 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont love the name client-timing. How would you feel about attempt-timestamp? or attempt-epoch?

Copy link
Contributor

@igorbernstein2 igorbernstein2 Sep 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these be prefixerd with x-bigtable-?

Copy link
Contributor Author

@mutianf mutianf Sep 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to attempt-epoch. Will update the cl too

attemptCount = ((BigtableTracer) apiCallContext.getTracer()).getAttempt();
}
ImmutableMap.Builder headers = ImmutableMap.<String, List<String>>builder();
if (attemptCount != -1) {
Copy link
Contributor

@igorbernstein2 igorbernstein2 Sep 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont think adding magic value here is useful. I think it would be cleaner to restructure as:

ImmutableMap.Builder headers = ImmutableMap.<String, List<String>>builder();

if (apiCallContext.getTracer() instanceof BigtableTracer) {
  BigtableTracer bigtableTracer = (BigtableTracer) apiCallContext.getTracer();
  int attemptNumber = bigtableTracer.getAttempt();
  headers.put(ATTEMPT_HEADER_KEY.name(), Arrays.asList(String.valueOf(attemptNumber)));
}

Copy link
Contributor Author

@mutianf mutianf Sep 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated.. I don't know why I wrote it that way 😓

callContext
.getTracer()
.attemptStarted(externalFuture.getAttemptSettings().getOverallAttemptCount());
callContext.getTracer().attemptStarted(externalFuture.getAttemptSettings().getAttemptCount());
Copy link
Contributor

@igorbernstein2 igorbernstein2 Sep 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might be a typo?

Copy link
Contributor Author

@mutianf mutianf Sep 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I reverted the change

@@ -81,6 +88,7 @@ public void attemptStarted(int attemptNumber) {
for (ApiTracer child : children) {
child.attemptStarted(attemptNumber);
}
this.attempt = attemptNumber;
Copy link
Contributor

@igorbernstein2 igorbernstein2 Oct 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should probably happen first

static final Metadata.Key<String> ATTEMPT_HEADER_KEY =
Metadata.Key.of("bigtable-attempt", Metadata.ASCII_STRING_MARSHALLER);
static final Metadata.Key<String> ATTEMPT_EPOCH_KEY =
Metadata.Key.of("bigtable-client-attempt-epoch", Metadata.ASCII_STRING_MARSHALLER);
Copy link
Contributor

@igorbernstein2 igorbernstein2 Oct 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can drop client-, I dont think there is ambiguity who set this header

Copy link
Contributor

@igorbernstein2 igorbernstein2 Oct 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if its better, but have you considered sending the timestamp as binary?

Copy link
Contributor Author

@mutianf mutianf Oct 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will keep the naming and timestamp for since the change on the server side depends on this and is already released. We can update them later if we see any problems.

* number starts from 0.
*/
@InternalApi("For internal use only")
public final class ExtraHeadersSeverStreamingCallable<RequestT, ResponseT>
Copy link
Contributor

@igorbernstein2 igorbernstein2 Oct 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: missing r in server

Copy link
Contributor

@igorbernstein2 igorbernstein2 Oct 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rename it to StatsHeader....

* number starts from 0.
*/
static Map<String, List<String>> createExtraHeaders(ApiCallContext apiCallContext) {
ImmutableMap.Builder headers = ImmutableMap.<String, List<String>>builder();
Copy link
Contributor

@igorbernstein2 igorbernstein2 Oct 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please out type params on the variable type

Copy link
Contributor

@igorbernstein2 igorbernstein2 left a comment

lgtm after comments

@mutianf mutianf added the kokoro:force-run label Oct 28, 2021
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run label Oct 28, 2021
@mutianf mutianf merged commit de3b476 into googleapis:main Oct 29, 2021
17 checks passed
@mutianf mutianf deleted the headers1 branch Oct 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable cla: yes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants