Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data base conception #2

Closed
5 tasks done
freezed opened this issue Jul 21, 2018 · 3 comments
Closed
5 tasks done

Data base conception #2

freezed opened this issue Jul 21, 2018 · 3 comments
Assignees
@freezed
Projects
[Initial] - Evaluated work

Comments

@freezed
Copy link
Owner

freezed commented Jul 21, 2018 •
edited

  • Recover [Open Food Facts][https://fr.openfoodfacts.org/data] data in CSV format [out of scope]
  • Purge unkept data [out of scope]
  • Create the physical data model
  • Create the local database
  • Simplify the PDM & DB
@freezed freezed added the TODO Get to work label Jul 21, 2018
@freezed freezed added this to To do in [Initial] - Evaluated work via automation Jul 21, 2018
@freezed freezed removed the TODO Get to work label Jul 21, 2018
@freezed freezed self-assigned this Jul 21, 2018
@freezed freezed changed the title Data reflexion Data base conception Jul 21, 2018
@freezed freezed moved this from To do to In progress in [Initial] - Evaluated work Jul 22, 2018
freezed added a commit that referenced this issue Jul 22, 2018
@freezed
Imports & reduces data to a mono-table MySQL DB #2
b698d3c 
`sample.csv` is a small test-file based on the HUDGE OpenFF CSV file :

        head --lines=100 fr.openfoodfacts.org.products.csv > sample.csv
        tail --lines=100 fr.openfoodfacts.org.products.csv >> sample.csv

Script keep :

- `'countries_fr':'France'` tagged rows
- selected fields (see documentation)
@freezed freezed mentioned this issue Jul 23, 2018
Closed
freezed added a commit that referenced this issue Jul 23, 2018
@freezed
Syncs fields between documentation and script #2
6b6e03f
freezed added a commit that referenced this issue Jul 23, 2018
@freezed
Adjuts field type to not loosing data #2
5a2a97a
freezed added a commit that referenced this issue Jul 24, 2018
@freezed
Adds Physical Data Model #2 & creates doc folder
7f4ea9b 
Updates README (URL's links), move docs files in new doc folder
freezed added a commit that referenced this issue Jul 24, 2018
@freezed
Adds Physical Data Model #2 & creates doc folder
92a3f2b 
Updates README (URL's links), move docs files in new doc folder
freezed added a commit that referenced this issue Jul 26, 2018
@freezed
Adds MySQL script to create local DB. #2:done
f3421e3 
Renames legacy script with DB name
@freezed
Copy link
Owner Author

freezed commented Jul 26, 2018

Done with f3421e3

@freezed freezed closed this as completed Jul 26, 2018
[Initial] - Evaluated work automation moved this from In progress to Done Jul 26, 2018
@freezed
Copy link
Owner Author

freezed commented Jul 31, 2018

Physical data model too complex, just using 2 tables (product & category).
No association tables, keep one category per product.

@freezed freezed reopened this Jul 31, 2018
[Initial] - Evaluated work automation moved this from Done to In progress Jul 31, 2018
freezed added a commit that referenced this issue Aug 2, 2018
@freezed
Updates physical data model #2
a900bca
freezed added a commit that referenced this issue Aug 2, 2018
@freezed
Updates SQL for DB creation script #2
ae687d8 
- Adds UNIQUE KEY
- Deletes association table
- Adds product's url
freezed added a commit that referenced this issue Aug 2, 2018
@freezed
Adds developpement requests (INSERT & DB size) #2
005a25a
@freezed
Copy link
Owner Author

freezed commented Aug 2, 2018

Closed by ae687d8

@freezed freezed closed this as completed Aug 2, 2018
[Initial] - Evaluated work automation moved this from In progress to Done Aug 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@freezed freezed

Labels
None yet
Projects
[Initial] - Evaluated work
  
Done
Milestone
No milestone
Development

No branches or pull requests
1 participant
@freezed