New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sas7bdat: Check if the SAS file has zero variables #18184

Merged
merged 1 commit into from Nov 10, 2017

Conversation

Projects
None yet
3 participants
@AbdealiJK
Contributor

AbdealiJK commented Nov 9, 2017

If the given SAS file has 0 rows, throw an error for the EmptyData file.
When reading, check that the column information is available. If not,
throw an error.

  • closes #xxxx
  • tests added / passed
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry
@pep8speaks

This comment has been minimized.

pep8speaks commented Nov 9, 2017

Hello @AbdealiJK! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on November 10, 2017 at 03:11 Hours UTC
@jreback

can you add a whatsnew note for 0.22 (can be in bug fixes). what did this do before with an empty file?

@@ -174,3 +176,9 @@ def test_date_time():
df0 = pd.read_csv(fname, parse_dates=['Date1', 'Date2', 'DateTime',
'DateTimeHi', 'Taiw'])
tm.assert_frame_equal(df, df0)
def test_zero_variables():
dirpath = tm.get_data_path()

This comment has been minimized.

@jreback

jreback Nov 9, 2017

Contributor

can you add the issue number (if no issue, then this PR number) as a comment

def test_zero_variables():
dirpath = tm.get_data_path()
fname = os.path.join(dirpath, "zero_variables.sas7bdat")
with pytest.raises(EmptyDataError):

This comment has been minimized.

@jreback

jreback Nov 9, 2017

Contributor

can you test with the zero observations file as well (is it the same result)? you can parametrize if that'st the case

This comment has been minimized.

@AbdealiJK

AbdealiJK Nov 9, 2017

Contributor

The zero observations file gives a None value while reading an empty the csv file gives an empty dataframe. I think this is a bug but have not fixed it right now

dirpath = tm.get_data_path()
fname = os.path.join(dirpath, "zero_variables.sas7bdat")
with pytest.raises(EmptyDataError):
df = pd.read_sas(fname)

This comment has been minimized.

@jreback

jreback Nov 9, 2017

Contributor

don't need the df = here

@jreback jreback added the IO SAS label Nov 9, 2017

@AbdealiJK

This comment has been minimized.

Contributor

AbdealiJK commented Nov 9, 2017

@jreback - Thanks for the comments. Updated.

@AbdealiJK

This comment has been minimized.

Contributor

AbdealiJK commented Nov 9, 2017

can you add a whatsnew note for 0.22 (can be in bug fixes). what did this do before with an empty file?

Can you help me with which file I should be updating for this ?

@jreback

This comment has been minimized.

Contributor

jreback commented Nov 9, 2017

doc/source/whatsnew/v0.22.0.txt

@AbdealiJK

This comment has been minimized.

Contributor

AbdealiJK commented Nov 9, 2017

@jreback Thanks. its done

@jreback jreback added this to the 0.22.0 milestone Nov 9, 2017

@@ -90,6 +90,7 @@ Bug Fixes
- Bug in ``pd.read_msgpack()`` with a non existent file is passed in Python 2 (:issue:`15296`)
- Bug in ``DataFrame.groupby`` where key as tuple in a ``MultiIndex`` were interpreted as a list of keys (:issue:`17979`)
- Bug in :func:`pd.read_csv` where a ``MultiIndex`` with duplicate columns was not being mangled appropriately (:issue:`18062`)
- Bug in :func:`pd.read_sas` where a file with 0 variables gives an invalid error. Now it gives and ``EmptyDataError``

This comment has been minimized.

@jreback

jreback Nov 9, 2017

Contributor

what was the previous error? can you re-word this a bit.

This comment has been minimized.

@AbdealiJK

AbdealiJK Nov 10, 2017

Contributor

Done

sas7bdat: Check if the SAS file has zero variables
If the given SAS file has 0 rows, throw an error for the EmptyData file.
When reading, check that the column information is available. If not,
throw an error.
@codecov

This comment has been minimized.

codecov bot commented Nov 10, 2017

Codecov Report

Merging #18184 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18184      +/-   ##
==========================================
+ Coverage    91.4%   91.41%   +0.01%     
==========================================
  Files         163      163              
  Lines       50064    50067       +3     
==========================================
+ Hits        45759    45767       +8     
+ Misses       4305     4300       -5
Flag Coverage Δ
#multiple 89.22% <100%> (+0.02%) ⬆️
#single 40.35% <0%> (-0.07%) ⬇️
Impacted Files Coverage Δ
pandas/io/sas/sas7bdat.py 91.05% <100%> (+0.28%) ⬆️
pandas/io/gbq.py 25% <0%> (-58.34%) ⬇️
pandas/core/frame.py 97.8% <0%> (-0.1%) ⬇️
pandas/core/indexes/datetimes.py 95.48% <0%> (+0.09%) ⬆️
pandas/plotting/_converter.py 65.2% <0%> (+1.81%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca737ac...90d100c. Read the comment docs.

@codecov

This comment has been minimized.

codecov bot commented Nov 10, 2017

Codecov Report

Merging #18184 into master will increase coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18184      +/-   ##
==========================================
+ Coverage    91.4%   91.41%   +0.01%     
==========================================
  Files         163      163              
  Lines       50064    50067       +3     
==========================================
+ Hits        45759    45767       +8     
+ Misses       4305     4300       -5
Flag Coverage Δ
#multiple 89.22% <100%> (+0.02%) ⬆️
#single 40.35% <0%> (-0.07%) ⬇️
Impacted Files Coverage Δ
pandas/io/sas/sas7bdat.py 91.05% <100%> (+0.28%) ⬆️
pandas/io/gbq.py 25% <0%> (-58.34%) ⬇️
pandas/core/frame.py 97.8% <0%> (-0.1%) ⬇️
pandas/core/indexes/datetimes.py 95.48% <0%> (+0.09%) ⬆️
pandas/plotting/_converter.py 65.2% <0%> (+1.81%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca737ac...90d100c. Read the comment docs.

@jreback jreback merged commit 150f6d4 into pandas-dev:master Nov 10, 2017

3 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@jreback

This comment has been minimized.

Contributor

jreback commented Nov 10, 2017

thanks @AbdealiJK

No-Stream added a commit to No-Stream/pandas that referenced this pull request Nov 28, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment