Skip to content

Data Engineering Project aiming to create a comprehensive data model for the big US Retailer Walmart

Notifications You must be signed in to change notification settings

m-p-esser/mysql-walmart-data-model

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Hits

MySQL Walmart Data Model

🚧 Info

This Project is currently on hold

What is this Project about

Training data modeling techniques using MySQL

Start MySQL from Docker container

docker run -d --name $MYSQL_CONTAINER_NAME -e MY_SQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD -v $MYSQL_VOLUME

Prepare CSV Files

python src/process_data.py

The IP of container changes. To find out the current IP adress, run the following Docker commands

docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <container-name>

Create Tables

mysql -h 172.17.0.2 -u root -p < scripts/create_tables.sql > logs/create_tables.log

Load Data from CSV Files

mysql -h 172.17.0.2 -u root -p --local-infile=1 < scripts/ingest_data.sql > logs/ingest_data.log

Now log into MySQL shell to check tables

mysql -h 172.17.0.2 -u root -p

About

Data Engineering Project aiming to create a comprehensive data model for the big US Retailer Walmart

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages