Store metadata in storage in push/pull?

[roll]

For now we do not store any metadata in the storage (bigquery, sql etc). It means pull_datapackage works only using reflection of the database. So additional metadata can't be restored (description, user defined fields etc).

As a solution we can store additional table with stringified datapackage.json.

---

pwalsh

@roll excellent observation. Seeing as we can't rely on a backend supporting, for example, JSON storage, and neither can we expect any particular fields on any given data package, I agree that we can just have a table with a stringified datapackage.json, but it might have other columns too, that for example point to the tables used for the DP in question (Imagine a storage backend that holds data from many datapackages - we might expect a common meta table that points out to the various tables for each package).

[rufuspollock]

I'm not sure we'd want to push full metadata into the backend storage would we? Normally we just want to push the data in order to do some analysis, not as permanent storage for the data packages.

[roll]

Here on JTS level - frictionlessdata/tableschema-py#70 - I'm also leaning to the idea to not store metadata in backend to do not complicate things.

[roll]

CLOSED FOR NOW (will be solved with other approach at jts level)

[rufuspollock]

@roll what's the referencing issue for closing this - if it is going to be closed in JTS somewhere could you reference the issue there?

[roll]

@rgrp
Sorry here it is - frictionlessdata/tableschema-py#70 - it will fix problem with types on JTS level. On DP level based on your words we don't need to store metadata like licence etc.