Body is not validated against JSON Schema

[marker452]

In our API Blueprint file, we use JSON Schema under +Schema and an example under +Body. Is there any way to get Dredd to validate the example body against the JSON Schema? We could add this kind of checking separately, but it'd be awesome if it were native to dredd.

[honzajavorek]

This is exactly what Dredd should do. Could you please share an example of what doesn't work?

[marker452]

Just to clarify, Dredd is working when run against the actual server, but it doesn't seem to validate against the actual + Body section in the API Blueprint.

There is an example here: https://gist.github.com/marker452/1221d7669b6a2fe740b19ca9eed225ee

I am running dredd with $ dredd --language python --hookfiles=./hooks/*.py. The response from the actual server validates against the schema in the Gist, but I don't think the body in the Gist would validate against that schema. However, when I run the command, I get no failures.

[GiritharanR]

@marker452 - one suggestion here, to validate the blueprint with json schema we can use api-blueprint-validator or grunt blueprint-validator. This will does the API blueprint validation.

[honzajavorek]

@marker452 Ah, I understand now! Thanks for clarifying. So this is something Dredd doesn't do as of now. There might be some other tools providing the feature, check out suggestions from @GiritharanR or scan the list at https://apiblueprint.org/tools.html.

Validating body samples against JSON Schema samples might be an interesting feature of Dredd, but I personally see it fitting more to a concept of something like "API Blueprint linter". Not sure whether there are some Open Source ones, but in Apiary we provide SaaS one as part of our plans.

Either way, will leave this open for people to comment and let @netmilk decide whether this makes sense as built-in part of Dredd.

[dhoffman34]

In case anyone else finds it useful to add + Body validation to dredd, it can easily be done as a hook (python example):

@hooks.before_each
def validate_body(transaction):
    try:
        body = transaction['expected'].get('body')
        schema = transaction['expected'].get('bodySchema')

        if body and schema:
            jsonschema.validate(json.loads(body), json.loads(schema))
    except (ValidationError, ValueError) as e:
        transaction['fail'] = str(e)

[honzajavorek]

@dhoffman34 Thanks for sharing this! This could go to http://dredd.readthedocs.io/en/latest/hooks-nodejs/#examples, actually.

[honzajavorek]

I actually think this is something Dredd could (and should) do. It's relevant for both API Blueprint and Swagger and I think it is a common mistake, which is hard to debug. The question would be, whether it makes sense to have this directly in the parser tooling? @apiaryio/adt

[kylef]

Linking in some prior relation suggestion (from before I worked at Apiary): apiaryio/snowcrash#154. This was for JSON validation, I'd see JSON Schema as next step. Although I am not sure it makes sense in the parser itself, perhaps additional tooling between the parser(s) and Dredd.

Thinking further, isn't this a style guide rule? 😉

[honzajavorek]

It is! But it's so low hanging and straightforward, that I think it's a matter of user experience to offer this out of the box.

[pksunkara]

I would be strictly against it being in the parser. It should be a downstream tool.

[Almad]

I would be strictly against it being in the parser. It should be a downstream tool.

I agree it shouldn't be in snowcrash, but any reason why it shouldn't be part of fury toolchain?