Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix python tests on Windows #2530

Merged
merged 3 commits into from Aug 3, 2020

Conversation

peterychang
Copy link
Collaborator

Fixes a few things.

  1. deleting the vw object in python did not result in its destruction
  2. finish_example cannot be called on an example that has not been used. Doing so on Windows will cause it to crash (this behavior appears to have existed for a while, an incorrect test simply exposed it)
  3. Creating a DataFrame with int values will generate a DataFrame containing int64s. However, the default conversion of int->pandas_dtype on Windows is int->int32. This causes type mismatch errors when performing our type checks in DFtoVW.

I'm not completely sure of the ramifications of the vw deletion change though; on the surface it makes sense and all of the tests pass. However there may be some subtle interactions with how we implement the python wrappers or how they're being used that could cause issues.

@@ -59,7 +59,7 @@ vw_ptr my_initialize(std::string args)
{ if (args.find_first_of("--no_stdin") == std::string::npos)
args += " --no_stdin";
vw*foo = VW::initialize(args);
return boost::shared_ptr<vw>(foo, dont_delete_me);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we leave a comment in case things go downhill in the future?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

particularly bc we're not sure of the impact and we also had some memory leak reported so there's something quirky going on with the python bindings. maybe a comment would be helpful as a breadcrumb in a future investigation

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think leaving the rationale here would be ok. The code makes sense, and if a problem comes up, the commit history will eventually lead back to this PR. Sound reasonable?

@peterychang
Copy link
Collaborator Author

Commenting on the change to the vw shared_ptr deleter function:
It looks like the original no-op deleter was added as part of the initial commit for the python wrapper. It appears to have been added either as a mistake or possibly to deal with some peculiarity in the way vw cleanup was being done at the time. One piece of evidence pointing to the former case is the comment left here

void my_finish(vw_ptr all)
{ VW::finish(*all, false);  // don't delete all because python will do that for us!
}

This line incorrectly comments that python will delete the object; in actuality the object would be leaked because the default shared pointer deleter was replaced with the no-op deleter.

@jackgerrits jackgerrits added this to the 8.9.0 milestone Aug 3, 2020
@peterychang peterychang merged commit 5d4c23a into VowpalWabbit:master Aug 3, 2020
lalo pushed a commit to lalo/vowpal_wabbit that referenced this pull request Aug 12, 2020
* Fix python tests on Windows

* move import to top

Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>
olgavrou pushed a commit to Sharad24/vowpal_wabbit that referenced this pull request Nov 12, 2020
* Fix python tests on Windows

* move import to top

Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants