Coerce toJSON'ed Buffer to real Buffer #2954
Comments
QuentinFarizon
changed the title
QuentinFarizon
changed the title
|
Can you provide your hapi configuration as well? I'm not aware of it transmitting buffers as objects. |
|
Hello @Marsup |
|
Is this is part of a multipart where you expect a file buffer on that route? Buffers initiated by hapi itself are actual buffers, and you should not have to do this, so unless you're actually serializing buffers as JSON (which would be bad), there must be something else interfering with the payload. |
|
Ok I understand that having defined my route as taking JSON, it is expected to receive it as a JSON object. This is part of a larger JSON object. Still, I think it should be parsed as a native Buffer. Or am I missing something on how to mix Joi.binary inside of a JSON payload ? |
|
JSON doesn't have semantics for serializing buffers, how is it supposed to pass it as such? |
|
Well, passing it as { type: 'Buffer, data: } worked with my fix for parsing it, and seemed legit :) I could instead define pass this Buffer as a |
|
This is very inefficient to pass a buffer serialized as JSON, you shouldn't do that, it should be in a multipart field, or as a plain string if doable. |
|
This is a serial number of length 4 to 7, so efficiency is not really an issue. I'm not sure how to pass it as a string, since there could be 0 in it that would be considered as null-terminators |
@Marsup Buffer itself encodes to a format automatically: https://nodejs.org/api/buffer.html#buftojson as part of the Node Build-In API for it. However, there is no automatic function that automatically hooks into JSON parsing and converts it back the other way. The example code in the documentation does show that you can use Buffer.from(value) to reinterpret the resulting object that was encoded in Javascript to covert it back. Since the binary type checker already uses the Buffer API, My suggestion would be to adapt the binary type coerce method to be something like: coerce(value, { schema }) {
if (typeof value === 'string') {
try {
return { value: Buffer.from(value, schema._flags.encoding) };
}
catch (ignoreErr) { }
}
if (typeof value === 'object' && value.type === 'Buffer') {
try {
return { value: Buffer.from(value, schema._flags.encoding) };
}
catch (ignoreErr) { }
}
return { value };
},This follows the same logic presented in the code sample in the Node.js documentation of the buf.toJSON() I put together a minimal reproduction of this issue that utilizes node.js Buffer API: const Joi = require('joi');
// Setup Schema that requires a binary value.
const schema = Joi.object().keys({
name: Joi.string().required(),
content: Joi.binary().required(),
});
// test PNG graphic from Wikipedia PNG article
const testPng = Buffer.from('89504E470D0A1A0A0000000D49484452' +
'00000001000000010802000000907753' +
'DE0000000C4944415408D763F8CFC000' +
'000301010018DD8DB00000000049454E' +
'44AE426082', "hex");
// This creates a test object
const testObj = {
name: "test.png",
content: testPng
};
// Output validate() return prior to JSON encode/decode
console.log("Validation of Input", schema.validate(testObj));
// Covert object to JSON
const jsonOutput = JSON.stringify(testObj);
// Output JSON that was generated
console.log("JSON", jsonOutput);
// Convert JSON back to object
const testObjAfterJSON = JSON.parse(jsonOutput);
// Output restored object validate() return
console.log("Validation of Output", schema.validate(testObjAfterJSON));Output of test using Joi 17.9.2 and node v20.0.0 without above change: Output of test using Joi 17.9.2 and node v20.0.0 with above change: Note that the type attribute isn't present in other JSON encoded values of other Node built in API objects. It appears to be unique to the Buffer API. Keep in mind Buffer objects are a node.js API, and thus do not exist in browser (i.e. Chrome, Firefox, Safari, etc) based APIs. I don't think this is an issue for Joi since it already uses the Buffer API in the binary type check. |
|
@QuentinFarizon In researching this a bit more, if you have control over the schema, this can be worked around by using alternatives with an object with custom casting. Joi.alternatives(
Joi.binary(),
Joi.object({
type: 'Buffer',
data: Joi.array().items(Joi.number().min(0).max(255))
}).and('type','data').custom(Buffer.from)
)Now Joi 17.9.2 and node v20.0.0 running: const Joi = require('joi');
// Setup Schema that requires a binary value.
const schema = Joi.object().keys({
name: Joi.string().required(),
content: Joi.alternatives(
Joi.binary(),
Joi.object({
type: 'Buffer',
data: Joi.array().items(Joi.number().min(0).max(255))
}).and('type','data').custom(Buffer.from)
).required(),
});
// test PNG graphic from Wikipedia PNG article
const testPng = Buffer.from('89504E470D0A1A0A0000000D49484452' +
'00000001000000010802000000907753' +
'DE0000000C4944415408D763F8CFC000' +
'000301010018DD8DB00000000049454E' +
'44AE426082', "hex");
// This creates a test object
const testObj = {
name: "test.png",
content: testPng
};
// Output validate() return prior to JSON encode/decode
console.log("Validation of Input", schema.validate(testObj));
// Covert object to JSON
const jsonOutput = JSON.stringify(testObj);
// Output JSON that was generated
console.log("JSON", jsonOutput);
// Convert JSON back to object
const testObjAfterJSON = JSON.parse(jsonOutput);
// Output restored object validate() return
console.log("Validation of Output", schema.validate(testObjAfterJSON));Results in the output of: Which causes it to at least return the buffer how the application expects it. While I personally think that the type coerce function should be adapted to handle the buffer in object format, this at least gives a workaround for those who need this capability now or if the repository maintainers disagree. Either way! Great library with great documentation! Thank you! |
|
Implemented in #2960, soon to be released. |
Hi! 馃憢
Firstly, thanks for your work on this project! 馃檪
I am using joi with Hapi, and I have a route with a payload field defined as
serialNumber: Joi.binary()But when inspecting payload before validation, I see that I receive the data not as a Buffer instance, but as an object with { type: "Buffer", data: } (probably Hapi calling toJSON() on it). I think it's valid to coerce it to a Buffer instance
Here is the diff that solved my problem:
The text was updated successfully, but these errors were encountered: