Addresses #437, Access.Earth Importer#635
Conversation
joshi1983
left a comment
joshi1983
left a comment
There was a problem hiding this comment.
The importers/access.earth/data_backup.json file is over 13MB which could add a lot to the download size of our git repository. Can you copy it to a publicly available Google Drive link instead? Also, squash or fixup the delete of the file into the commit that originally added it.
It is nice to have the raw data available for others but making it part of our repository could nearly double the data size. For now, the main repository is roughly 18MB and it could become 30MB if you commit that file and it compresses poorly.
| data=requests.get(url) | ||
| if len(data.text) < 10: | ||
| return "" | ||
| #print(data.json()[0]) |
There was a problem hiding this comment.
Remove this commented code line.
| json.dump(bigList,saveData) | ||
| saveData.close() | ||
| fd.close() | ||
| if __name__ == "__main__": |
There was a problem hiding this comment.
Put at least 1 empty line after the main function to break it from code after the function.
|
@joshi1983 Google Drive link to data_backup.json is here |
…s 'school, university, college' in name to add to Education
M4ttoF
changed the title
2 participants
Run 'generate_csv' to get the csv file. 'generate_csv' must be in the same directory as cleanData.json when it is run.