Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better windows support #644

Merged
merged 35 commits into from Sep 25, 2020
Merged

Better windows support #644

merged 35 commits into from Sep 25, 2020

Conversation

lhoestq
Copy link
Member

@lhoestq lhoestq commented Sep 18, 2020

There are a few differences in the behavior of python and pyarrow on windows.

For example there are restrictions when accessing/deleting files that are open

Fix #590

@lhoestq lhoestq changed the title Windows support [WIP] Better windows support Sep 18, 2020
Copy link
Member

@thomwolf thomwolf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about one modification (maybe it's related to the CI failure)

src/datasets/builder.py Outdated Show resolved Hide resolved
@lhoestq lhoestq changed the title [WIP] Better windows support Better windows support Sep 25, 2020
@lhoestq
Copy link
Member Author

lhoestq commented Sep 25, 2020

This PR is ready :)
It brings official support for windows.

Some tests AWSDatasetTest are failing.
This is because I had to fix a few datasets that were not compatible with windows.
These test will pass once they got merged on master :)

Copy link
Member

@thomwolf thomwolf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly I mostly trust you here :)

@@ -26,6 +30,41 @@ jobs:
- run: pip install pyarrow==1.0.0
- run: HF_SCRIPTS_VERSION=master python -m pytest -sv ./tests/


run_dataset_script_tests_pyarrow_0p17_WIN:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Youpi!

@lhoestq lhoestq merged commit e86a2a8 into master Sep 25, 2020
@lhoestq lhoestq deleted the fix-tests-on-windows branch September 25, 2020 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The process cannot access the file because it is being used by another process (windows)
2 participants