sebastienhamel/recipe_extractor
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
Hello!
I have recently developped an obsession with a type of dessert found in Québec, Canada: the galette.
Historically, most types of food found in Québec can be traced to the nations who invaded the territory
a couple of hundred years ago: France and Great Britain. Even if traditional food from Québec is not
directly found in these nation, you can often see the roots of it. For example, meat pies are pretty common
in Québec and can also be found in British traditional cooking.
Galettes (and obviously poutine) seem the be an exception.
This projects aim to scrape data from a popular recipe website in Québec and find if maybe the origin of the
galette could be not from colonizing nations but from practicality. Were the galettes invented in Québec because
they used ingredients more readily available? Were the galettes invented because they require less precision
than say a cake maybe?
These are the questions I am trying to answer.
Currently, the project is incomplete. These are my planned steps:
- Find all recipe related to sibling deserts in a popular Québec recipe website (cakes, cookies and obviously
galettes) (done)
- Scrape the recipes (work in progress)
- Dockerize the scripts so they can run on Microsoft Azure (work in progress)
- Normalize the data
- Perform statistical analysis on the data
- Answer my question
Here are the steps you should follow in order to build the container:
- setup a file at the root of your project, containing the following:
- the full url to the database: DATABASE_URL=mysql+mysqlconnector://username:password@mysql-db:3306?charset=utf8mb4
- MYSQL_ROOT_PASSWORD=yourPassword
- MYSQL_DATABASE=recipeextractor
- MYSQL_USER=recipe_scraper
- MYSQL_PASSWORD=yourOtherPassword
- This file should have the .env extension and will be used in the docker run command below.
- create a custom network container by running: docker network create scraper-network
- run your containers using: docker compose up
- This will instanciate three containers: the database, the redis container and the scraper container.
A couple of notes:
1. After the first start of the celery beats, it is normal to have this error: "dbm.error: db type is dbm.gnu,
but the module is not available". At the second beat, this error will go away.
2. If you want to test your connection to the database, you can use: docker exec -it mysql-db mysql -u root -p. Then enter the root password from you .env file.