Encoder might skip intermediate resources when fieldset filters are applied

[hglattergotz]

At the application level a developer is presented with the request that might look like this:

GET /sites/1?include=posts.comments,comments.author&fields[comments]=body HTTP/1.1
Accept: application/vnd.api+json

From the test example EncodeIncludedObjectsTest we can see that the include and fields parameters need to be combined (in this example comments):

    public function testEncodeWithRecursiveIncludedObjects()
    {
        $this->author->{Author::LINK_COMMENTS} = $this->comments;

        $actual = Encoder::instance([
            Author::class  => AuthorSchema::class,
            Comment::class => CommentSchema::class,
            Post::class    => PostSchema::class,
            Site::class    => SiteSchema::class,
        ], $this->encoderOptions)->encodeData($this->site, new EncodingParameters(
            // include only this relation (according to the spec intermediate will be included as well)
            [Site::LINK_POSTS . '.' . Post::LINK_COMMENTS],
            // include only these attributes and links
            [
                'comments' => [Comment::ATTRIBUTE_BODY, Comment::LINK_AUTHOR],
                'posts'    => [Post::LINK_COMMENTS],
                'sites'    => [Site::LINK_POSTS],
            ]
        ));
        :
        :
    }

Given that the url will contain the includes and fields in separate parameters, it is up to the application level to correctly parse posts.comments,comments.author and figure out to create the following for the $includePaths:

    [Site::LINK_POSTS . '.' . Post::LINK_COMMENTS], // posts.comments

and

    'comments' => [Comment::ATTRIBUTE_BODY, Comment::LINK_AUTHOR /*author*/],

for $fieldSets.

Would it not be better to put this logic in the library?

[neomerx]

Do you mean to have separate parameters for them? Can you post a code piece how its usage could look like?

[hglattergotz]

Looking through the code base and tests a bit more I found Neomerx\JsonApi\Parameters\ParametersParser.

Using the example in this issue

GET /sites/1?include=posts.comments,comments.author&fields[comments]=body HTTP/1.1
Accept: application/vnd.api+json

Parsing this with an instance of ParametersParser I believe you would end up with a Parameters instance with the following private member values:

$this->includePaths = ['posts.comments','comments.author'];
$this->fieldSets = ['comments' => ['body']]

You would NOT get

$this->includePaths = ['posts.comments'];
$this->fieldSets = ['comments' => ['body', 'author']]

What I was asking originally is if there is code in this library that will build the EncodingParameters object for me from the query string. The answer to that is YES: ParametersParser
But, I think that ParametersParser does not output an instance of EncodingParameters/Parameters that would be needed to make the test pass because the parser does not add links to the field sets (it will not create ['comments' => ['body', 'author'], but instead will create ['comments' => ['body'] which will cause the Encode not to include the Author).

[lindyhopchris]

I think @hglattergotz interpretation is correct, however the include links shouldn't be added to the fields asked for by the client. (It's unsafe to start adding to what the client has actually asked for.) Instead the encoder needs to process what fields to encode independently from what included resources need to be encoded.

The definition of fields is:

A resource object's attributes and its relationships are collectively called its "fields".

So the example request has asked for comments to only have the body field included, but has not asked for the author relationship field to be included. So

$this->includePaths = ['posts.comments','comments.author'];
$this->fieldSets = ['comments' => ['body']]

is correct based on the request received. The request has not asked for the comments.author relationship field to be included. (Remember the relationship field is the identifier contained within the comments resource object - not the related resource.)

What should now happen is that because comments.author has been requested as an include, the encoder needs to include that related resource in the returned response even though any encoded comments resource will not have their author relationship field encoded.

I.e. the logic as to whether a related resource is included in the encoded response is independent from whether the relationship field is being encoded.

So your comment would look like this:

{
    "type": "comments",
    "id": "123",
    "attributes": {
        "body": "The comment body"
    }
}

But the included resources should included the comment's author resource.

It does not matter that the comments.author relationship field is not included in the returned compound document because:

The only exception to the full linkage requirement is when relationship fields that would otherwise contain linkage data are excluded via sparse fieldsets.

[hglattergotz]

@lindyhopchris It did not even occur to me that relationship names could be included in the sparse field set parameter or the fields[type] url parameter. I though this was a neomerx/json-api'ism.

A resource object's attributes and its relationships are collectively called its "fields".

The above does not explicitly say either way if this applies to the fields[type] url parameter (IMO). But various discussions on this subject seem to indicate that relationships can be both in include and fields[type].

Your summary seems on point (@lindyhopchris).

So it seems like because of my lack of understanding I inadvertently found an issue. ;-)

[lindyhopchris]

@hglattergotz

The quote definitely does refer to the fields parameter. The spec says:

A client MAY request that an endpoint return only specific fields in the response on a per-type basis by including a fields[TYPE] parameter.
http://jsonapi.org/format/#fetching-sparse-fieldsets

The word fields in that quote is hyper-linked to the definition of fields, which is:

A resource object's attributes and its relationships are collectively called its "fields".
http://jsonapi.org/format/#document-resource-object-fields

I'd agree, you seem to have found an issue!

[neomerx]

Can you please explain the issue in plain English?

[lindyhopchris]

Say there's this set of data:

GET /posts/1?include=comments.author

{
    "data": {
        "type": "posts",
        "id": "1",
        "attributes": {
            "title": "My post",
            "body": "Post body...",
            "date": "2015-01-01"
        },
        "relationships": {
            "comments": {
                "data": [
                    {
                        "type": "comments",
                        "id": "123"
                    }
                ]
            }
        }
    },
    "included": [
        {
            "type": "comments",
            "id": "123",
            "attributes": {
                "body": "My comment...",
                "date": "2015-01-01",
                "likes": 5
            },
            "relationships": {
                "author": {
                    "data": {
                        "type": "people",
                        "id": "345"
                    }
                }
            }
        },
        {
            "type": "people",
            "id": "345",
            "attributes": {
                "name": "Foo",
                "surname": "Bar"
            }
        }
    ]
}

If only the comment's body field is requested, then the result should be:

GET /posts/1?include=comments.author&fields[comments]=body

{
    "data": {
        "type": "posts",
        "id": "1",
        "attributes": {
            "title": "My post",
            "body": "Post body...",
            "date": "2015-01-01"
        },
        "relationships": {
            "comments": {
                "data": [
                    {
                        "type": "comments",
                        "id": "123"
                    }
                ]
            }
        }
    },
    "included": [
        {
            "type": "comments",
            "id": "123",
            "attributes": {
                "body": "My comment..."
            }
        },
        {
            "type": "people",
            "id": "345",
            "attributes": {
                "name": "Foo",
                "surname": "Bar"
            }
        }
    ]
}

I.e. author 345 should still be in the included member even though the comments.relationships.author field is not included in the comment.

@hglattergotz correct me if I've got it wrong, but I believe you're saying that the encoder would actually not include the author resource in the included member because the comments.author field has not been requested?

[neomerx]

@lindyhopchris thanks for explanation

feature/issue98 branch created