Test suite to ensure compliance to spec

[jerel]

As a maintainer of a JSON API adapter it would be helpful to us to have an external test suite that ensures we are in compliance with the spec. Otherwise I'm afraid the json-api standard might be more of a "standard" as implementers accidentally deviate or don't completely implement the spec.

I found issue #789 which was promptly closed with a link to endpoints/endpoints#125 which appears to have been abandoned in June.

Are there any further developments or is there a reason why a universal test suite shouldn't happen?

[tkellen]

Endpoints has not been abandoned, but it is OSS and neither I or the other contributors have had time to continue work on recently it due to other obligations.

Put simply, the reason why the universal test suite doesn't exist is because nobody has written it. We're all doing this in our free time.

If you'd like to take a stab writing one we can point at any implementation, here is a clear roadmap for what needs to be tested:
https://github.com/json-api/json-api/blob/gh-pages/_format/1.0/normative-statements.json

[jerel]

Endpoints has not been abandoned, but it is OSS and neither I or the other contributors have had time to continue work on recently it due to other obligations.

Put simply, the reason why the universal test suite doesn't exist is because nobody has written it. We're all doing this in our free time.

Right, I understand. This issue is not an attack on the Endpoints project or anyone's availability but just to restart the discussion and/or raise visibility to new contributors that might be able to help.

[tkellen]

👍

[gr0uch]

Just want to add a +1.

I don't want to have to keep maintaining my own tests.

[masterspambot]

👍

[tenodi]

👍

[jnslxndr]

👍

[ieugen]

:+1 I felt the neet when working on integrationg katharsis https://github.com/katharsis-project/katharsis-core/issues

[hglattergotz]

@jerel Do you have any concrete ideas about how to structure something like this to make it language agnostic?
I found the json-schema project that has such a test suite: https://github.com/json-schema/JSON-Schema-Test-Suite.

I share your sentiment and think it is a great idea. Takes a lot of the guess work out for lib maintainers.
A single source of truth would be great!

[cziegenberg]

I created a project to validate JSON API documents. It supports version 1.0 and is ready to also support version 1.1 with profile extensions. It can detect the exact position of the error and provide a JSON Pointer for it (for the error object). I hope to publish it soon - it's written in PHP7 that will hopefully be released in two weeks...

[Perni1984]

@ziege: how is your project coming along, can we eventually help you with it?

[cziegenberg]

Sorry, it's paused.

The first problem is, that my boss told me to focus on other things, because we are unhappy with the development and not sure about using JSON API anymore. The second problem is that he didn't agree to publish the things I already created (perhaps later, but I'm not sure).

There are several reasons for this decision:

• No extension support. I got a lot of time to work on this, to ensure that JSON API 1.1 will support an extension system that fulfills some important requirements for a business service (API reliability, extensibility, flexibility,...).
• Most suggestions have been denied because it makes things too complex. That could have been discussed and solved, but it looks like only very simple (error-prone) solutions are accepted and there is no focus on business services.
• We have to accept this, so that's why we are looking for an alternative format that we can support instead.

[Perni1984]

@ziege: Sorry to hear that. Eventually he changes his mind so you can publish your work in some time in future.