Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure parquet tests are skipped if fastparquet and pyarrow not installed #5217

Merged
merged 4 commits into from Aug 5, 2019

Conversation

@jrbourbeau
Copy link
Member

commented Aug 2, 2019

This PR updates test_append_overlapping_divisions and test_append_different_columns in dask/dataframe/io/tests/test_parquet.py to ensure they are skipped when fastparquet and pyarrow are not installed.

xref #5193

  • Tests added / passed
  • Passes black dask / flake8 dask
@jrbourbeau

This comment has been minimized.

Copy link
Member Author

commented Aug 2, 2019

The AppVeyor failures are unrelated to the changes here and being tracked in #4994

@jrbourbeau

This comment has been minimized.

Copy link
Member Author

commented Aug 2, 2019

The test suite is passing now without fastparquet and pyarrow installed 🎉 I'll go ahead and add them back in now

@@ -733,7 +733,7 @@ def test_append_wo_index(tmpdir, engine):
assert_eq(df.set_index("f"), ddf3)


def test_append_overlapping_divisions(tmpdir):
def test_append_overlapping_divisions(tmpdir, engine):

This comment has been minimized.

Copy link
@martindurant

martindurant Aug 3, 2019

Member

So this is the critical change - before it would try with the default engine, and error when there were none, but now it skips if one or other engine is missing, correct?

This comment has been minimized.

Copy link
@jrbourbeau

jrbourbeau Aug 3, 2019

Author Member

Correct. The test is now run twice, once with fastparquet as engine and another time with pyarrow. In both cases, the test will be skipped if the corresponding engine is not installed.

@martindurant

This comment has been minimized.

Copy link
Member

commented Aug 5, 2019

OK, sounds good to me

@jrbourbeau jrbourbeau merged commit 973e8eb into dask:master Aug 5, 2019

1 of 2 checks passed

continuous-integration/appveyor/pr AppVeyor build failed
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@jrbourbeau

This comment has been minimized.

Copy link
Member Author

commented Aug 5, 2019

Thanks for the review @martindurant!

@jrbourbeau jrbourbeau deleted the jrbourbeau:skip-parquet-tests branch Aug 5, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.