Make field level metadata searchable and link common fields across the catalog #640

Open
philipashlock opened this Issue May 21, 2015 · 7 comments

Comments

Projects
None yet
4 participants
@philipashlock
Member

philipashlock commented May 21, 2015

We should be able to expose machine readable data dictionaries describing each field or variable in a dataset and index this as part of the catalog in order to more easily discover and understand each variable and provide linkages across disparate datasets.

This could also source metadata from resources linked from the describedBy field such as Swagger API specs and associated JSON Schema files. See https://project-open-data.cio.gov/v1.1/api/

Some precedent with CKAN examples like http://catalogue.data.gov.bc.ca/dataset/mta-mineral-placer-and-coal-titles and related work with Data Packager http://ckan.org/2014/06/09/the-open-knowledge-data-packager/ but ideally this wouldn't require data to be hosted within CKAN (DataStore) in order to provide the field level metadata.

This also relates to project-open-data/project-open-data.github.io#465 and uscensusbureau/citysdk#100

See also http://www.w3.org/blog/data/2015/04/16/csv-on-the-web-seeking-comments-and-implementations/

@dportnoy

This comment has been minimized.

Show comment
Hide comment
@dportnoy

dportnoy May 26, 2015

+1 for HealthData.gov. Looking to work together on a solution.

Discussed: http://david.portnoy.us/field-level-data-dictionaries-for-open-data/

+1 for HealthData.gov. Looking to work together on a solution.

Discussed: http://david.portnoy.us/field-level-data-dictionaries-for-open-data/

@philipashlock

This comment has been minimized.

Show comment
Hide comment
@philipashlock

philipashlock Mar 28, 2016

Member

We'd probably want to build on this extension - https://github.com/cmuphillycapstone/ckanext-dictionary

Member

philipashlock commented Mar 28, 2016

We'd probably want to build on this extension - https://github.com/cmuphillycapstone/ckanext-dictionary

@dportnoy

This comment has been minimized.

Show comment
Hide comment
@dportnoy

dportnoy Apr 4, 2016

@philipashlock, agreed! I've been talking with the author of this extension on adding this functionality.

dportnoy commented Apr 4, 2016

@philipashlock, agreed! I've been talking with the author of this extension on adding this functionality.

@philipashlock

This comment has been minimized.

Show comment
Hide comment
@philipashlock

philipashlock Dec 15, 2017

Member

Just adding this to track more recent discussion and documentation around this as a feature available with CKAN core (#3414) https://stackoverflow.com/questions/47783069/data-dictionary-feature-of-ckan-2-7-x

Member

philipashlock commented Dec 15, 2017

Just adding this to track more recent discussion and documentation around this as a feature available with CKAN core (#3414) https://stackoverflow.com/questions/47783069/data-dictionary-feature-of-ckan-2-7-x

@jalbertbowden

This comment has been minimized.

Show comment
Hide comment
@jalbertbowden

jalbertbowden Mar 1, 2018

ckan next dictionary looks neat. are there any known live examples to poke around on?

ckan next dictionary looks neat. are there any known live examples to poke around on?

@jalbertbowden

This comment has been minimized.

Show comment
Hide comment
@jalbertbowden

jalbertbowden Mar 2, 2018

that is neat and almost a solution to this thread, i think. html is the perfect format in this situation, the ckan dictionary just needs some microformats/microdata to give it the desired structures.
is there a list of examples or desired fields/variables that i can tinker with?
should mostly be just a matter of adding attributes and/or elements.

that is neat and almost a solution to this thread, i think. html is the perfect format in this situation, the ckan dictionary just needs some microformats/microdata to give it the desired structures.
is there a list of examples or desired fields/variables that i can tinker with?
should mostly be just a matter of adding attributes and/or elements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment