You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a JSON structure coming in and it has string representations of Booleans, Ints, Longs. I believe that avsc has the capability to infer if given the right options on the parse function. However, I'm not entirely sure how to go about doing this and would greatly appreciate any assistance.
const type = avro.parse(await schema);
// Add additional fields needed for pub/sub
message.IngestionTimestamp = new Date(Date.now()).toISOString();
// this accepts UPSERT and DELETE, but I expect that DELETEs should not come through.
message._CHANGE_TYPE = 'UPSERT';
// this needs to be done before we change the finished_at value
message._CHANGE_SEQUENCE_NUMBER = Math.ceil(message.finished_at);
// these come in as floats but we want to change them to a TIMESTAMP string
message.finished_at = new Date(Math.ceil(message.finished_at)).toISOString();
message.started_at = new Date(Math.ceil(message.started_at)).toISOString();
// avsc doesn't change a string boolean to a boolean object
message.agent_expired = Boolean(message.agent_expired);
console.log(type.toString(message));
The error message I get is this:
Error: invalid ["null","long"]: "1258"
at throwInvalidError (/Users/H476998/Desktop/work-items/digital-uptime-results/node_modules/avsc/lib/types.js:3042:9)
at UnwrappedUnionType._copy (/Users/H476998/Desktop/work-items/digital-uptime-results/node_modules/avsc/lib/types.js:1383:7)
at RecordType._copy (/Users/H476998/Desktop/work-items/digital-uptime-results/node_modules/avsc/lib/types.js:2481:26)
at ArrayType._copy (/Users/H476998/Desktop/work-items/digital-uptime-results/node_modules/avsc/lib/types.js:2052:31)
at RecordType._copy (/Users/H476998/Desktop/work-items/digital-uptime-results/node_modules/avsc/lib/types.js:2481:26)
at Type.toString (/Users/H476998/Desktop/work-items/digital-uptime-results/node_modules/avsc/lib/types.js:678:30)
at publishMessage (/Users/H476998/Desktop/work-items/digital-uptime-results/avro-helper.js:19:22)
at publishAvroRecords (/Users/H476998/Desktop/work-items/digital-uptime-results/index.js:69:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
The text was updated successfully, but these errors were encountered:
Hi @sabbysabs. avsc doesn't expose an option to auto-convert types to their string representation. You could achieve it with the combination of a logical type to do the transformation and a type hook to apply it where it makes sense, but I would not recommend it as first option. If possible, it would be better to update the schema to reflect the true types, or explicitly convert the fields that need it.
I have a JSON structure coming in and it has string representations of Booleans, Ints, Longs. I believe that avsc has the capability to infer if given the right options on the parse function. However, I'm not entirely sure how to go about doing this and would greatly appreciate any assistance.
My sample JSON:
input.json
My Schema:
runscope.json
My code:
The error message I get is this:
The text was updated successfully, but these errors were encountered: