Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the sanitized dataset properties to the resulting data.json file #118

Closed
3 tasks done
nightsh opened this issue Apr 14, 2020 · 0 comments
Closed
3 tasks done
Assignees

Comments

@nightsh
Copy link
Collaborator

nightsh commented Apr 14, 2020

As part of #109 we started sanitizing the data obtained by scraping. The sanitized values are stored in an extra key named _clean_data, which needs to be processed by the datajson transformer before the final output.

Tasks:

  • amend the datajson transformer to include the _clean_data values
  • run a harvester on a processed file to make sure everything works

Acceptance criteria:

  • the values in _clean_data are used in the final data.json output
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants