Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception: Data read but end of buffer not reached #21

Closed
devtrix opened this issue Dec 10, 2021 · 6 comments
Closed

Exception: Data read but end of buffer not reached #21

devtrix opened this issue Dec 10, 2021 · 6 comments

Comments

@devtrix
Copy link

devtrix commented Dec 10, 2021

I am trying to decode a cbor string. I used TextEncoder to convert it to Uint8Array and used the decode function when I get this error.
index.js:138 Uncaught (in promise) Error: Data read, but end of buffer not reached
at checkedRead (index.js:138)
at Object.decode (index.js:86)

Please help.

@kriszyp
Copy link
Owner

kriszyp commented Dec 10, 2021

This generally means that the input data is not valid. You created a valid CBOR binary to pass into decode? Here is an example of how you would create a (valid) CBOR encoding using TextEncoder:

let te = new TextEncoder();
let bytes = new Uint8Array(64);
bytes[0] = 0x78; // one-byte string length encoding
// encode the string, starting at the third byte and store the length:
let stringBytes = te.encodeInto('Hello, World!', bytes.subarray(2)).written;
bytes[1] = stringBytes; // store the length in the string prefix
decode(bytes.subarray(0, stringBytes + 2)); -> 'Hello, World!'

Do you have a sample of your code that is failing?

@devtrix
Copy link
Author

devtrix commented Dec 11, 2021

I definitely was not doing this. So let me explain what I am trying to do here. I am working with a crypto wallet and it gives me a 124 char hex encoded cbor string. I need to decode that string so parts of it can be used for validation. So I tried to convert it to a Uint8Array as your decode method didnt work with a string.
Here is the hex encoded cbor string:
821a00159282a1581c00b96929c38ef48107f18e3122873b43de6182c9355aae4110ce3d33a15654657374436f6c6c656374696f6e3030566f6c33343101

This is not sensitive info, just FYI.
Any help is appreciated.

@kriszyp
Copy link
Owner

kriszyp commented Dec 11, 2021

You would need to first convert the hex string to binary bytes (there are lot of different types of string/binary encodings). If you are working in NodeJS you could just decode the hex string to a buffer by specifying it as the string encoding:

let bytes = Buffer.from('821a00159282a1581c00b96929c38ef48107f18e3122873b43de6182c9355aae4110ce3d33a15654657374436f6c6c656374696f6e3030566f6c33343101', 'hex');
decode(bytes); // bytes is a valid CBOR and can be decoded

If you are not working Node, there are plenty of examples of how to convert a hex to a Uint8Array like:

const fromHexString = hexString =>
  new Uint8Array(hexString.match(/.{1,2}/g).map(byte => parseInt(byte, 16)));

I don't really know what the returned CBOR structure means, but I assume it means something to you.

@devtrix
Copy link
Author

devtrix commented Dec 11, 2021

So what I am getting back is
[ 1413762, { "0,185,105,41,195,142,244,129,7,241,142,49,34,135,59,67,222,97,130,201,53,90,174,65,16,206,61,51": { "84,101,115,116,67,111,108,108,101,99,116,105,111,110,48,48,86,111,108,51,52,49": 1 } } ]
My question is, what is this an array(?) of?: "0,185,105,41,195,142,244,129,7,241,142,49,34,135,59,67,222,97,130,201,53,90,174,65,16,206,61,51"?

I need code to do something like what happens on cbor.me.
I paste the hex string on the right side, select "as text" checkbox and click the left arrow button [<--] where it displays the word Bytes. I want my object (output) to look like the left side.
Like this:
[1413762, {h'00B96929C38EF48107F18E3122873B43DE6182C9355AAE4110CE3D33': {'TestCollection00Vol341': 1}}]

Thanks in advance.

@kriszyp
Copy link
Owner

kriszyp commented Dec 11, 2021

Your CBOR structure is a map where the keys are binary data. This doesn't really have a clear JS equivalent, because JS requires that object keys be strings (or symbols or numbers), not typed arrays (like Uint8Array). So in the browser, if you are using the default setting where maps are mapped to objects, the typed array just gets coerced to a string ("0,185,105,41,195,...").

My question is, what is this an array(?) of?: "0,185,105,41,195,142,244

I have no idea, this is the binary data for the key that came from your data. It is the decimal representation of that binary data (and 00B96929C38EF48... is the hexadecimal representation of that same binary data).

If you want to preserve the original keys, you may want to consider mapping the CBOR map structures to JS Maps, which will preserve the original binary data:

let decoder = new CBOR.Decoder({ mapsAsObjects: false });
let decoded = decoder.decode(bytes);
let firstKey = Array.from(decoded[1].keys())[0] -> first key as a Uint8Array
let secondKey = Array.from(decoded[1].get(firstKey).keys())[0] -> second key as Uint8Array
new TextDecoder().decode(secondKey) -> decode second to a string

@devtrix
Copy link
Author

devtrix commented Dec 12, 2021

Thanks a million. You are the best! Thank you for your patience, kindness and absolute brilliance. Needless to say your very quick responses. My hats off to you.

As you can probably tell I am not a JS expert but I am also in the middle of a ton of things so I greatly appreciate all your help.
Thanks again! I am closing this.

@devtrix devtrix closed this as completed Dec 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants