union type error

[asarubbi]

given this AVSC:

{
  "type" : "record",
  "name" : "event_root",
  "namespace" : "com.millicom.digital.events.schema",
  "fields" : [ {
    "name" : "apiproxy",
    "type" : {
      "type" : "record",
      "name" : "apiproxy_type",
      "fields" : [ {
        "name" : "name",
        "type" : "string"
      }, {
        "name" : "revision",
        "type" : "string"
      } ]
    }
  }, {
    "name" : "client",
    "type" : {
      "type" : "record",
      "name" : "client_type",
      "fields" : [ {
        "name" : "cn",
        "type" : [ "null", "string" ]
      }, {
        "name" : "country",
        "type" : [ "null", "string" ]
      } ]
    }
  } ]
}

and this JSON:

{
  "schema": "tigo_mobile_gt_upselling_v1_Subscriber_Balances",
  "apiproxy": {
    "name": "tigo_mobile_gt_upselling_v1",
    "revision": "4"
  },
  "application": {
    "basepath": null
  },
  "client": {
    "cn": null,
    "country": null
  }
}

and using permutations between

• tag country in the JSON
• data type for country in the AVSC

AVSC Type	JSON Value	Result	Status	Comment
"type" : [ "null", "string" ]	null	parsing ok	expected
"type" : [ "null", "string" ]	"US"	parsing error (see below)	unexpected	the union type is supposed to allow either null or an string value
"type" : "string"	"US"	parsing ok	expected
"type" : "string"	null	parsing error	expected

** Parsing Error **

/home/asarubbi/Development/repo/avrotest/node_modules/avsc/lib/schemas.js:809
    keys = Object.keys(val);
                  ^
TypeError: Object.keys called on non-object
    at Function.keys (native)
    at UnionType._write (/home/asarubbi/Development/repo/avrotest/node_modules/avsc/lib/schemas.js:809:19)
    at RecordType.writeclient_type [as _write] (eval at <anonymous> (/home/asarubbi/Development/repo/avrotest/node_modules/avsc/lib/schemas.js:1620:10), <anonymous>:4:6)
    at RecordType.writeevent_root [as _write] (eval at <anonymous> (/home/asarubbi/Development/repo/avrotest/node_modules/avsc/lib/schemas.js:1620:10), <anonymous>:4:6)
    at RecordType.Type.toBuffer (/home/asarubbi/Development/repo/avrotest/node_modules/avsc/lib/schemas.js:247:8)
    at Object.<anonymous> (/home/asarubbi/Development/repo/avrotest/index.js:9:18)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)

my code:

var avsc = require('avsc');

var parser = avsc.parse("./schemas/event_root.avsc");

var evt = require("./input.json");

var buf = parser.toBuffer(evt);

console.log(buf);

var obj = parser.fromBuffer(buf);

console.log(obj);

[mtth]

The error is expected, this record isn't valid (see below). The error message isn't explicit, but this is a purposeful trade-off (encoding is designed to be as fast as possible; you can always use isValid when that is needed).

Your record is invalid because unions are by default deserialized to objects with single key the type of the chosen branch (similar to their JSON representation). This isn't ideal but the best way to handle all unions properly in JavaScript. You can take a look here for more information and even a way to implement the behavior you initially expected.

TL;DR: Your record should have looked like:

{
  "schema": "tigo_mobile_gt_upselling_v1_Subscriber_Balances",
  "apiproxy": {
    "name": "tigo_mobile_gt_upselling_v1",
    "revision": "4"
  },
  "application": {
    "basepath": null
  },
  "client": {
    "cn": null,
    "country": {"string": "US"}
  }
}

[asarubbi]

just to understand the trade off you mention:

• my JSON is VALID exactly as is. actually, it's being generated outside my control so i have no inference on the format.
• my AVSC is VALID
• the library cannot parse a perfectly valid json with a perfectly valid schema because it would slow down the parser and will break the purpose of the whole library.

so either i tell my client to change the way they generate the json or i find myself another library?

[mtth]

my AVSC is VALID

Right.

my JSON is VALID exactly as is.

It depends what you mean. It's valid JSON but it isn't a valid JSON encoding of a record according to your schema (see Avro's JSON encoding specification). For example, {"foo": 2} is a valid JSON object, but isn't a valid encoding of a record with schema {"type": "record", "fields": [{"name": "foo", "type": "string"}], /* ... */}.

the library cannot parse a perfectly valid json with a perfectly valid schema because it would slow down the parser and will break the purpose of the whole library.

No. See above.

Finally, even if the data is generated out of your control, in this case you can still conveniently convert it into valid records by using clone's wrapUnions option (example here).

Hope this makes sense.

[asarubbi]

make sense and clone method with wrapUnions did the trick preprocessing the json to convert to something acceptable for the parser.
thanks for the support.

[edulop91]

Is it intended that clone's, wrapUnions not replace missing fields with their default value?

var schema = {
  type: 'record',
  name: 'test',
  fields: [
    { name: 'test', type: ['null', 'string'], default: null },
    { name: 'foo', type: ['null', 'string'], default: null }
  ]
};

var testType = avsc.parse(schema);

var incomplete = {foo: {string: 'bar'} }
var complete = {foo: {string: 'bar'}, test: null }

// Works as expected
testType.clone(complete, { wrapUnions: true });
// Throws error: invalid ["null","string"]: undefined
testType.clone(incomplete, { wrapUnions: true });

[mtth]

@edulop91 - It should, I'll add this.

[mtth]

@edulop91 - Fixed in 98eb097 and released in 3.1.2.

[edulop91]

@mtth Awesome! Thanks for the quick response!

[RPS044]

@mtth @tboothman @asarubbi @edulop91

is it solved ?

invalid union index: 4032 [object Object]"