New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate data and metadata in records. #256

Open
almet opened this Issue Nov 4, 2015 · 7 comments

Comments

Projects
None yet
4 participants
@almet
Member

almet commented Nov 4, 2015

I'm opening this for discussion in Kinto, but it's entierly possible that it should live in Cliquet instead.

Currently, records have some metadata attributes directly returned alongside the business attributes. These are "id" and "last_modified".

{
  'data': {'id': '1234', 'last_modified': 123456, 'foo': 'bar'}
}

It could make sense to separate these metadata attributes from the data. e.g.

{
  'id': '1234',
  'last_modified': 123456,
  'data': {'foo': 'bar'}
}

One of the challenges with this approach is that record lists should also contain the associated metadata. I'll let @leplatrem expend on this as I don't see clearly where the problem lies.

Other than that, this change would break the API in an incompatible way, and thus the clients would need to be updated accordingly. Validation features (Schema) and permissions would also be impacted by this approach.

Related to #158

@leplatrem

This comment has been minimized.

Contributor

leplatrem commented Nov 4, 2015

Very related to #174

@leplatrem leplatrem added the question label Nov 4, 2015

@Natim

This comment has been minimized.

Member

Natim commented Nov 4, 2015

Actually what I understand of your proposal, taking care of @leplatrem concerns, is to rather do:

{
  "data": {
      "id": "1234",
      "last_modified": 123456,
      "data": {"foo": "bar"}
    }
}

So adding a new level to put all records data in it.

Then the collection_get will look like :

{
    "data": [{
      "id": "1234",
      "last_modified": 123456,
      "data": {"foo": "bar"}
    }]
}

The question is, will permission be part of this data or not?

{
  "data": {
      "id": "1234",
      "last_modified": 123456,
      "data": {"foo": "bar"},
      "permissions": {}
    }
}

or

{
  "data": {
      "id": "1234",
      "last_modified": 123456,
      "data": {"foo": "bar"}
    },
    "permissions": {}
}
@Natim

This comment has been minimized.

Member

Natim commented Nov 4, 2015

If we go furthermore in the proposal and look at collections, will it look like that:

{
    "data": {
        "id": "tasks",
        "last_modified": 1446646942107,
        "data": {"fingerprint": "1a4089078fda74bc20441f62e621b057a92cffb6ace35ccde0c32b6394949801"},
        "schema": {"JSONSchema": "in it"}
    }
}

or like that?

{
    "data": {
        "id": "tasks",
        "last_modified": 1446646942107,
        "data": {
            "fingerprint": "1a4089078fda74bc20441f62e621b057a92cffb6ace35ccde0c32b6394949801",
            "schema": {"JSONSchema": "in it"}
        }
    }
}

Depending of if we consider the schema being part of the collection attributes (that we want to take into account when signing the collection object).

@Natim

This comment has been minimized.

Member

Natim commented Nov 4, 2015

If we want to get closer to what jsonapi is doing, it should probably look like this:

{
  "data": {
      "id": "1234",
      "last_modified": 123456,
      "attributes": {"foo": "bar"}
    }
}
@leplatrem

This comment has been minimized.

Contributor

leplatrem commented Nov 4, 2015

The question is, will permission be part of this data or not?

From what I understand I guess it should. Like schema and other metadata.

One of the challenges with this approach is that record lists should also contain the associated metadata.

In terms of implementation, this would probably mean to get rid of the multiple backends and put everything into one. Otherwise, joining data and permissions during the list construction might not be efficient at all. Especially if we want stuff like #123

I'll let @leplatrem expend on this as I don't see clearly where the problem lies.

As I said privately, this could be quite a change for the whole ecosystem, and I see no emergency in doing this. I personally would rather keep metadata mixed up with data and put the efforts into other challenges that bring more value to the end users (docs, install, demos, integrity, ...)

@almet

This comment has been minimized.

Member

almet commented Nov 5, 2015

@Natim I don't understand what we get by having two different levels of "data" (or data/attributes). Having data alongside the other metadata (last modified etc) seems cleaner to me.

Permissions, schema etc should go alongside the rest. Eg.

{
  'id': '1234',
  'last_modified': 123456,
  'data': {'foo': 'bar'},
  'permissions': {},
  'schema': 'blah'
}

@leplatrem leplatrem added the protocol label Jan 28, 2016

@smuriu

This comment has been minimized.

smuriu commented Feb 29, 2016

I'm very much in favour of the proposal by @almet. I only just recently came across kinto but already thinking of things I could do with it. One question I have, related to the kinto-attachment plugin, would this proposal if implemented add the attachment attribute as metadata alongside id, permissions, ...? Seems a lot cleaner that way IMO.

On a side note, has anyone proposed a related records plugin? In my mind, it would add the key relations which is an array of a relation objects:

'relations': [
  {
    'relation': {
      'collection': 'collection_x',
      'record': 'record_x'
    }
  },
  {
    'relation': {
      'collection': 'collection_x',
      'record': 'record_y'
    }
  }
]

Again, this would feel nice if added as metadata alongside id, ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment