Skip to content

Load retail demo data using compressed file#271

Merged
kmax12 merged 9 commits intomasterfrom
load-retail-zip
Oct 2, 2018
Merged

Load retail demo data using compressed file#271
kmax12 merged 9 commits intomasterfrom
load-retail-zip

Conversation

@WillKoehrsen
Copy link
Contributor

Changed load_retail to load a gz compressed version of the retail detail from S3. This reduces the size of the downloaded file from 43 MB to 7.3 MB.

@codecov-io
Copy link

codecov-io commented Oct 1, 2018

Codecov Report

Merging #271 into master will decrease coverage by 0.02%.
The diff coverage is 60%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #271      +/-   ##
==========================================
- Coverage   94.47%   94.45%   -0.03%     
==========================================
  Files          71       71              
  Lines        7696     7700       +4     
==========================================
+ Hits         7271     7273       +2     
- Misses        425      427       +2
Impacted Files Coverage Δ
featuretools/demo/retail.py 85% <60%> (-8.75%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8498bb6...8903c05. Read the comment docs.

* Update to reference Spark

* Added reference to Spark scaling

Included references to spark notebook and article for using Featuretools on a cluster.
Tries to load `gz` compressed and defaults to uncompressed `csv`  if error.
Doc string now refers to both `gz` compressed and uncompressed versions of csv data file.
@WillKoehrsen
Copy link
Contributor Author

Added a fallback to uncompressed version. This might be necessary if pandas.read_csv() cannot read the gz compressed version.

@kmax12
Copy link
Contributor

kmax12 commented Oct 2, 2018

Looks good. Merging

@kmax12 kmax12 changed the title Load retail gz compressed Load retail demo data as compressed file Oct 2, 2018
@kmax12 kmax12 changed the title Load retail demo data as compressed file Load retail demo data using compressed file Oct 2, 2018
@kmax12 kmax12 merged commit da09b20 into master Oct 2, 2018
@WillKoehrsen WillKoehrsen deleted the load-retail-zip branch October 3, 2018 13:40
@rwedge rwedge mentioned this pull request Oct 31, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants