Harvester for old website data

[sirex]

There is a similar tool:

https://github.com/sirex/ckan-ivpk-import

But in this case we will be synchronizing data directly from MySQL database.

Code for this script is hosted here (based on the mentioned similar tool):

https://github.com/ivpk/opendata.gov.lt-mysql-import

Todo

• Data packages (datasets).
• Add cron job to do the update continuously.
• Package tags.
• Contact information.
• Organizations.
• Synchronize groups.
• Investigate, maybe it is possible to import CSV resources into CKAN.

Extra tasks:

• Possibility to update groups if for example name has changed.
• Delete groups if groups where deleted in IVPK database.

[sirex]

It was decided to refactor whole synchronization thing to a harvester and only synchronizing datasets.

[sirex]

@JustNindze I'm attaching list of links used in opendata.gov.lt packages, use these links to extract all package resources for each link.

rinkmenos.csv.gz