Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data.json harvest improvements #38

Open
kvuppala opened this issue Mar 24, 2017 · 0 comments
Open

data.json harvest improvements #38

kvuppala opened this issue Mar 24, 2017 · 0 comments

Comments

@kvuppala
Copy link

  • currently we check the individual dataset json object hash to identify if there are any new updates to the dataset.

  • as a first step, we should check the entire json file content hash to make sure there are some updates compared to the previous version, and the next step would be to go into the individual datasets

This additional check improves the harvesting process, and going into the individual dataset hash wont be required if the whole file hash didn't change.

@kvuppala kvuppala changed the title data.json harvest improvements data.json harvest improvements Mar 24, 2017
@kvuppala kvuppala added Review and removed Backlog labels Apr 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant