Add integration tests against reference data

[acmiyaguchi]

The test suite currently has a hand-curated set of expected outputs for the schemas. However, the resulting schema may not be valid against certain edge-cases. For each of the test cases under tests/resources/, there should be a json document that passes validation. This can be used for validating that transformations are working correctly.

This testing script was useful for validating sampled data from the aws pipeline against mozilla-pipeline-schemas.

jsonschema-transpiler/scripts/mps-generate-avro-data-helper.py

Lines 35 to 77 in a7ab358

	def convert(data, schema):
	if schema.type == "string":
	if not isinstance(data, str):
	return json.dumps(data)
	if schema.type == "record":
	# iterate over all keys
	out = {}
	if not data:
	return out
	for key, value in data.items():
	# apply the appropriate transformations on the key
	key = format_key(key)
	field = schema.field_map.get(key)
	if not field:
	continue
	out[key] = convert(value, field.type)
	return out
	if schema.type == "union":
	for sub in schema.schemas:
	if sub.type == "null":
	continue
	out = convert(data, sub)
	return out
	if schema.type == "array":
	out = []
	if not data:
	return out
	for item in data:
	out.append(convert(item, schema.items))
	return out
	if schema.type == "map":
	out = {}
	for key, value in data.items():
	out[key] = convert(value, schema.values)
	return out
	# terminal node, do nothing
	return data