Error: decoded.map is not a function

[walshie4]

Environment details

• OS: CoreOS
• Node.js version: v8.9.4
• npm version: v6.0.1
• @google-cloud/spanner version: v1.4.1

Steps to reproduce

1. ???
2. ???

This error is occasionally happening when running the same query as in #197.

SELECT * FROM <TABLE> ORDER BY id LIMIT 10000 OFFSET @offset

const [rows] = await spannerDb.run({
  sql: 'SELECT * FROM <table> ORDER BY id LIMIT @limit OFFSET @offset',
  params: {
    limit: 10000,
    offset: 0,
  },
}, { readTimestamp: timestampBound });
return rows.map(row => row.toJSON()); // Causes error from scrambled data

TypeError: decoded.map is not a function

        at decodeValue_ (/opt/app/node_modules/@google-cloud/spanner/src/codec.js:145)
        at Object.decode (/opt/app/node_modules/@google-cloud/spanner/src/codec.js:179)
        at (/opt/app/node_modules/@google-cloud/spanner/src/partial-result-stream.js:183)
        at Array.map (<anonymous>)
        at Function.partialResultStream.formatRow_ (partial-result-stream.js:179)
        at (/opt/app/node_modules/@google-cloud/spanner/src/partial-result-stream.js:175)
        at Array.map (<anonymous>)
        at Function.partialResultStream.formatRow_ (partial-result-stream.js:173)
        at DestroyableTransform._transform (/opt/app/node_modules/@google-cloud/spanner/src/partial-result-stream.js:99)
        at DestroyableTransform.Transform._read (_stream_transform.js:184)
        at DestroyableTransform.Transform._write (_stream_transform.js:172)

[stephenplusplus]

This is likely #180 as well, sadly.

[walshie4]

Re-opening this following the discussion in #180. The issue is currently resulting in an unhandled rejection with the above error which seems to cause the promise from the library to never resolve or reject causing our tasks to freeze up.

[crwilcox]

@walshie4
Sorry for a bit of delay between responses. I have rewritten the core logic of how we stream rows. It is possible it has fixed this issue also now (even though I believe this is not related to the other). If you want to try npm install googleapis/nodejs-spanner#rewrite-partial-result-stream you can.

[walshie4]

🙇 Thank you for the branch @crwilcox. I've tested it next to my current latest and can confirm it does in fact resolve this issue.

[crwilcox]

Quick Update: We are still testing this branch and handling a few edge cases.

[crwilcox]

Branch has been updated as we have corrected some corner cases that regressed. As soon as we finish authoring some new tests this fix will be merged.

[crwilcox]

A patch release is published that includes this fix. https://www.npmjs.com/package/@google-cloud/spanner

[sourcec0de]

@crwilcox I am still encountering this exact same issue using 1.5.0

@google-cloud/spanner/src/codec.js:235
api_1    |         decoded = decoded.map(function(value) {
api_1    |                           ^
api_1    | 
api_1    | TypeError: decoded.map is not a function
api_1    |     at decodeValue_ (/opt/app/node_modules/@google-cloud/spanner/src/codec.js:235:27)
api_1    |     at Object.decode (/opt/app/node_modules/@google-cloud/spanner/src/codec.js:260:10)
api_1    |     at Function.RowBuilder.formatValue (/opt/app/node_modules/@google-cloud/spanner/src/row-builder.js:74:18)
api_1    |     at values.map (/opt/app/node_modules/@google-cloud/spanner/src/row-builder.js:196:27)
api_1    |     at Array.map (<anonymous>)
api_1    |     at rows.map.values (/opt/app/node_modules/@google-cloud/spanner/src/row-builder.js:191:31)
api_1    |     at Array.map (<anonymous>)
api_1    |     at RowBuilder.toJSON (/opt/app/node_modules/@google-cloud/spanner/src/row-builder.js:190:15)
api_1    |     at DestroyableTransform._transform (/opt/app/node_modules/@google-cloud/spanner/src/partial-result-stream.js:88:35)
api_1    |     at DestroyableTransform.Transform._read (/opt/app/node_modules/readable-stream/lib/_stream_transform.js:184:10)
api_1    |     at DestroyableTransform.Transform._write (/opt/app/node_modules/readable-stream/lib/_stream_transform.js:172:83)
api_1    |     at DestroyableTransform.obj.(anonymous function) [as _write] (/opt/app/node_modules/stubs/index.js:26:26)
api_1    |     at doWrite (/opt/app/node_modules/readable-stream/lib/_stream_writable.js:428:64)
api_1    |     at writeOrBuffer (/opt/app/node_modules/readable-stream/lib/_stream_writable.js:417:5)
api_1    |     at DestroyableTransform.Writable.write (/opt/app/node_modules/readable-stream/lib/_stream_writable.js:334:11)
api_1    |     at Pumpify.ondata (/opt/app/node_modules/readable-stream/lib/_stream_readable.js:619:20)

[stephenplusplus]

@sourcec0de Thanks for reporting. Do you have any code, and/or information about the data and query, that we could use to try to reproduce?

[sourcec0de]

@stephenplusplus I have been banging my head on the wall trying to resolve this issue for a few days. I honestly thought it may have been a conflict with other installations of @google-cloud/* libraries.

I've been using babel-node to take advantage of es6 module's until node supports it out of the box (without a flag).
After installing and upgrading all my @google-cloud libraries I found out that babel-node was the issue all along.

Switching to ts-node instead eliminated the issue.

Should have waited a little longer before reporting.

Thank you for being so prompt, though. It's great to see you guys active in here.