DCAT support for all datasets

[pietercolpaert]

feature request

Allow people and machines to download a datadump which describes all datasets in a CKAN instance using the DCAT vocabulary. This datadump should have an option to be licensed according to the open knowledge definition.

why

• Open meta-data!
• This way people will be able to test generic tools on a lot of data

I know that we can already request a single dataset's meta-data as RDF using for instance http://datahub.io/dataset/thesesfr.rdf. To get a list of all the datasets we could query the API and then perform a lot of requests to get all the rdf. When trying this on data.gov, the fun ends quickly as http://catalog.data.gov/api/3/action/package_list gives a time out (see #1137)

[rossjones]

You can schedule your own RDF dump using the rdf-export command-line tool (I suggest daily) the code for which is at https://github.com/okfn/ckan/blob/master/ckan/lib/cli.py#L488

The required command line args is in the doc at the link above.

[pietercolpaert]

@rossjones but this is a tool for CKAN administrators on the server itself? I cannot use this tool to get a dump from data.gov right?

[rossjones]

You're correct, you'd have to get the admins there to do it but it'll be a lot more efficient way of getting the data than 10s of thousands of API calls. At least until there is VoID support.

[pietercolpaert]

Is this something that might get implemented quickly or is this a wontfix?

[rossjones]

We have an open ticket for it on data.gov.uk, and then it depends whether it gets accepted into core CKAN. It isn't likely to be days until this is done, most likely weeks, and it won't be realtime (we have 9.5k datasets, the US has a lot more). And again, would need the administrator to set up the background task.

Easiest approach at present is to ask the data.gov team to enable the RDF dumps I think.

[pietercolpaert]

Can you link us to that issue?

[rossjones]

Afraid it isn't on github/openly accessible.

[kindly]

https://github.com/okfn/ckanext-dcat

This is being addressed here.