google-auth-library-nodejs v3.0 failed Google Analytics Data Import from Google Cloud Storage file

[lushu]

Environment details

• OS: MacOS
• Node.js version: v10.15.3
• npm version: 6.9.0
• google-auth-library version: v2.0.2 vs. v.3.1.2

Steps to reproduce

1. Create a Google Cloud Project and enable Google Analytics API
2. Create a service account and download the key file
3. In Google Analytics, create a user account based on the email of the service account and grant the 'Edit' access.
4. Create a 'data import' entry in Google Analytics
5. Prepare a CSV file and test through the GA UI Data Import. Make sure it works and upload to Google Cloud Storage.
6. Create a local node project, with dependency to googleapis and @google-cloud/storage (could be the latest version) and google-auth-library(v2.0.2). Write code based on googleapis to do a Google Analytics Data Import based on (1) local file fs.createReadStream; (2) Google Cloud Storage file createReadStream. They both work.
7. Change the google-auth-library version to v3.1.2 and update the library, then run the same two functions, the GCS one will failed with the error:

Error: Media type 'application/json' is not supported. Valid media types: [application/octet-stream]
at Gaxios.

8. Personal thought, it looks like gaxios didn't take care the readstream from GCS very well. It will send out other request overwrite the content-type as 'json' instead of the value which is 'application/octet-stream' in the code.

Thanks!
Shu

=====Full response of the failed request=====

{ Error: Media type 'application/json' is not supported. Valid media types: [application/octet-stream]
at Gaxios. (/usr/local/google/home/lushu/workspace/intheair/nodejs-standard/common/node_modules/gaxios/build/src/gaxios.js:72:27)
at Generator.next ()
at fulfilled (/usr/local/google/home/lushu/workspace/intheair/nodejs-standard/common/node_modules/gaxios/build/src/gaxios.js:16:58)
at process._tickCallback (internal/process/next_tick.js:68:7)
response:
{ config:
{ url:
'https://www.googleapis.com/upload/analytics/v3/management/accounts/2620914/webproperties/UA-2620914-4/customDataSources/_gTi9XfWTzS68BPXpaYqpA/uploads?uploadType=media',
method: 'POST',
paramsSerializer: [Function],
data: [DestroyableTransform],
headers: [Object],
params: [Object],
validateStatus: [Function],
body:
'{"_readableState":{"objectMode":false,"highWaterMark":16384,"buffer":{"head":null,"tail":null,"length":0},"length":0,"pipes":null,"pipesCount":0,"flowing":null,"ended":false,"endEmitted":false,"reading":false,"sync":false,"needReadable":true,"emittedReadable":false,"readableListening":false,"resumeScheduled":false,"destroyed":false,"defaultEncoding":"utf8","awaitDrain":0,"readingMore":false,"decoder":null,"encoding":null},"readable":true,"_events":{},"_eventsCount":3,"_writableState":{"objectMode":false,"highWaterMark":16384,"finalCalled":false,"needDrain":false,"ending":false,"ended":false,"finished":false,"destroyed":false,"decodeStrings":true,"defaultEncoding":"utf8","length":0,"writing":false,"corked":0,"sync":true,"bufferProcessing":false,"writecb":null,"writelen":0,"bufferedRequest":null,"lastBufferedRequest":null,"pendingcb":0,"prefinished":false,"errorEmitted":false,"bufferedRequestCount":0,"corkedRequestsFree":{"next":null,"entry":null}},"writable":true,"allowHalfOpen":true,"_transformState":{"needTransform":false,"transforming":false,"writecb":null,"writechunk":null,"writeencoding":null},"_destroyed":false}',
responseType: 'json' },
data: { error: [Object] },
headers:
{ 'alt-svc': 'quic=":443"; ma=2592000; v="46,44,43,39"',
connection: 'close',
'content-length': '4622',
'content-type': 'application/json; charset=UTF-8',
date: 'Mon, 06 May 2019 05:48:00 GMT',
server: 'UploadServer',
vary: 'Origin, X-Origin',
'x-google-apiary-auth-expires': '1557121980000',
'x-google-apiary-auth-scopes': 'https://www.googleapis.com/auth/analytics',
'x-google-apiary-auth-user': '783549991279',
'x-google-backends':
'/bns/sf/borg/sf/bns/apps-upload/apps-upload.uploader/205,acsydb10-v6:443',
'x-google-dos-service-trace': 'main:apps-upload',
'x-google-gfe-backend-request-info': 'eid=kMrPXOblKdbVrAHJ8JnoDQ',
'x-google-gfe-request-trace':
'acsydb10-v6:443,/bns/sf/borg/sf/bns/apps-upload/apps-upload.uploader/205,acsydb10-v6:443',
'x-google-gfe-response-code-details-trace': 'response_code_set_by_backend',
'x-google-gfe-service-trace': 'apps-upload-migration100',
'x-google-netmon-label': '/bns/sf/borg/sf/bns/apps-upload/apps-upload.uploader/205',
'x-google-security-signals': 'FRAMEWORK=HTTPSERVER2',
'x-google-service': 'apps-upload-migration100',
'x-google-session-info':
'CO-S4PnmFhoCGAYoATpAChlhbmFseXRpY3MtZGF0YS1pbXBvcnQtYXBpEglhbmFseXRpY3MiFTEwMTk3NTkwMTIxMjY4ODI4ODE5ODDUFkqfARKCAXlhMjkuYy5FbG9CQjlhdnJ4elJOdmRWd08yQ0k0Qlc5QlRPczBVZXFUZThRcG5Od1lISzdwWlVfMzQtRXJDeC1TVTc0ay1qUlVId2c4MWtjZjRqSTlWSWxPX0t0T0NxUThob3FCQ2c2c0lyLXpLa1BNSkk3Y1BkeWVVUldCNWRuSTQwBDoWTk9UX0FfUEVSU0lTVEVOVF9UT0tFTg',
'x-google-shellfish-status': 'CA0gBEBG',
'x-guploader-customer': 'apiary_single_post_uploads',
'x-guploader-request-result': 'agent_rejected',
'x-guploader-upload-result': 'agent_rejected',
'x-guploader-uploadid':
'AEnB2Urwk7jV7LfkvjgT283UtZrkXBuWDPJr0mJkQdBkSFXh6fWuarzgoCRHWB64o3furDvM9P9Y_s0GX4cPC4epL_OFvLRQAQ' },
status: 400,
F** statusText: 'Bad Request' },
config:
{ url:
'https://www.googleapis.com/upload/analytics/v3/management/accounts/2620914/webproperties/UA-2620914-4/customDataSources/_gTi9XfWTzS68BPXpaYqpA/uploads?uploadType=media',
method: 'POST',
paramsSerializer: [Function],
data:
DestroyableTransform {
_readableState: [ReadableState],
readable: true,
_events: [Object],
_eventsCount: 3,
_maxListeners: undefined,
_writableState: [WritableState],
writable: true,
allowHalfOpen: true,
_transformState: [Object],
_destroyed: false,
_transform: [Function: noop],
_read: [Function],
_write: [Function] },
headers:
{ 'Content-Type': 'application/json',
'Accept-Encoding': 'gzip',
'User-Agent': 'google-api-nodejs-client/0.7.2 (gzip)',
Authorization:
'Bearer ya29.c.EloBB9avrxzRNvdVwO2CI4BW9BTOs0UeqTe8QpnNwYHK7pZU_34-ErCx-SU74k-jRUHwg81kcf4jI9VIlO_KtOCqQ8hoqBCg6sIr-zKkPMJI7cPdyeURWB5dnI4',
Accept: 'application/json' },
params: [Object: null prototype] { uploadType: 'media' },
validateStatus: [Function],
body:
'{"_readableState":{"objectMode":false,"highWaterMark":16384,"buffer":{"head":null,"tail":null,"length":0},"length":0,"pipes":null,"pipesCount":0,"flowing":null,"ended":false,"endEmitted":false,"reading":false,"sync":false,"needReadable":true,"emittedReadable":false,"readableListening":false,"resumeScheduled":false,"destroyed":false,"defaultEncoding":"utf8","awaitDrain":0,"readingMore":false,"decoder":null,"encoding":null},"readable":true,"_events":{},"_eventsCount":3,"_writableState":{"objectMode":false,"highWaterMark":16384,"finalCalled":false,"needDrain":false,"ending":false,"ended":false,"finished":false,"destroyed":false,"decodeStrings":true,"defaultEncoding":"utf8","length":0,"writing":false,"corked":0,"sync":true,"bufferProcessing":false,"writecb":null,"writelen":0,"bufferedRequest":null,"lastBufferedRequest":null,"pendingcb":0,"prefinished":false,"errorEmitted":false,"bufferedRequestCount":0,"corkedRequestsFree":{"next":null,"entry":null}},"writable":true,"allowHalfOpen":true,"_transformState":{"needTransform":false,"transforming":false,"writecb":null,"writechunk":null,"writeencoding":null},"_destroyed":false}',
responseType: 'json' },
code: 400,
errors:
[ { domain: 'global',
reason: 'badContent',
message:
'Media type 'application/json' is not supported. Valid media types: [application/octet-stream]',
debugInfo:
'com.google.api.server.core.Fault: Builder{base=UNSUPPORTED_CONTENT, category=USER_ERROR, domain=global, httpHeaders={}, httpStatus=badRequest, internalReason=Builder{createdByBackend=false, unnamedArguments=[]}, message=Media type 'application/json' is not supported. Valid media types: [application/octet-stream], reason=badContent, rpcCode=400} Media type 'application/json' is not supported. Valid media types: [application/octet-stream]\n\tat com.google.api.server.media.uploader.filter.ContentTypeFilter.filter(ContentTypeFilter.java:48)\n\tat com.google.api.server.media.uploader.filter.AbstractMediaFilter.filter(AbstractMediaFilter.java:29)\n\tat com.google.api.server.media.uploader.filter.SizeFilter.filter(SizeFilter.java:84)\n\tat com.google.api.server.media.uploader.filter.AbstractMediaFilter.filter(AbstractMediaFilter.java:29)\n\tat com.google.api.server.media.uploader.filter.MetadataFilter.filter(MetadataFilter.java:54)\n\tat com.google.api.server.media.MediaPreprocessingInterceptor.processUploadRequest(MediaPreprocessingInterceptor.java:205)\n\tat com.google.api.server.media.MediaPreprocessingInterceptor.processRequest(MediaPreprocessingInterceptor.java:102)\n\tat com.google.api.server.core.intercept.AroundInterceptorWrapper.processRequest(AroundInterceptorWrapper.java:20)\n\tat com.google.api.server.stats.StatsBootstrap$InterceptorStatsRecorder.processRequest(StatsBootstrap.java:281)\n\tat com.google.api.server.core.inte*rcept.Interceptions$AroundInterception.processRequest(Interceptions.java:159)\n\tat com.google.api.server.core.intercept.Interceptions$AroundInterception.invoke(Interceptions.java:135)\n\tat com.google.api.server.media.AbstractMediaClosure.continueRequest(AbstractMediaClosure.java:64)\n\tat com.google.api.server.media.batch.BatchAgent.continueRequest(BatchAgent.java:141)\n\tat com.google.api.server.media.batch.BatchAgent.onRequestReceived(BatchAgent.java:74)\n\tat com.google.uploader.agent.ScottyBatchAgent$ServiceParameters$1.handleRequest(ScottyBatchAgent.java:418)\n\tat com.google.net.rpc3.impl.server.RpcServerInternalContext.runRpcInApplication(RpcServerInternalContext.java:602)\n\tat com.google.net.rpc3.impl.server.RpcServerChannel$3.apply(RpcServerChannel.java:921)\n\tat com.google.net.rpc3.impl.server.RpcServerChannel$3.apply(RpcServerChannel.java:909)\n\tat com.google.common.util.concurrent.AbstractTransformFuture$TransformFuture.doTransform(AbstractTransformFuture.java:242)\n\tat com.google.common.util.concurrent.AbstractTransformFuture$TransformFuture.doTransform(AbstractTransformFuture.java:232)\n\tat com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118)\n\tat com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:453)\n\tat com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:460)\n\tat io.grpc.Context.run(Context.java:565)\n\tat com.google.tracing.CurrentContext.runInContext(CurrentContext.java:204)\n\tat com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:319)\n\tat com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:311)\n\tat com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:457)\n\tat com.google.common.util.concurrent.MoreExecutors$5$1.run(MoreExecutors.java:1046)\n\tat com.google.api.server.thread.ThreadTrackers$ThreadTrackingRunnable.run(ThreadTrackers.java:126)\n\tat com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:453)\n\tat com.google.api.server.server.CommonModule$ContextCarryingExecutorService$1.runInContext(CommonModule.java:802)\n\tat com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:460)\n\tat io.grpc.Context.run(Context.java:565)\n\tat com.google.tracing.CurrentContext.runInContext(CurrentContext.java:204)\n\tat com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:319)\n\tat com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:311)\n\tat com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:457)\n\tat com.google.gse.internal.DispatchQueueImpl$WorkerThread.run(DispatchQueueImpl.java:403)\n' } ] }

[lushu]

I don't think it's a document issue. I have found the root cause and posted here