-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH - Gets SciKeras script working #394
base: main
Are you sure you want to change the base?
Conversation
WIP still need to fix one test
It looks like all the pytest tests are failing with |
Should be addressed by #398 |
@lazarust Could you please solve the merge conflict? Regarding the uncovered new line: Would that be solved by adding tests for scikeras? |
@BenjaminBossan I've fixed the merge conflict. Yeah, I believe it would be but I was unsure if we wanted to have tests that included another library like that. Should I add one? |
Yes, it would be good, since that was the initial reason for the change. We have external library tests, see here: https://github.com/skops-dev/skops/blob/main/skops/io/tests/test_external.py For scikeras, it won't be possible to add a comprehensive coverage of all possible models, so a simple example should be enough. If it's possible to add a unit test independently of scikeras that explicitly tests weakrefs, that would also be good, not sure how easy it is to do. Regarding the failing tests, at first glance, it appears to be a change in the model repr in the latest sklearn version, so not related to the changes here. |
@BenjaminBossan Sorry this took me so long to get back to. I've added a test to hit that line. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks so much for the added scikeras tests. I just have a small request to improve them a little bit.
skops/io/tests/test_external.py
Outdated
|
||
pipeline = Pipeline([("classifier", clf)]) | ||
|
||
dump(clf, "keras-test.skops") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of just dumping, could we please do a cycle of dumps
and loads
, similar to the other tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done! I now have it dumping the model, loading it back in and comparing the results.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we get rid of dump
completely in favor of dumps
? That way, we also don't need to care about cleaning up any files created during the test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed! Sorry about that. I didn't realize the difference between dumps
and dump
🤯
@adrinjalali The tests for sklearn nightly are failing because the model repr was changed (not sure why). Normally, we could fix that by having the test check the sklearn verison, but this is a doctest. Any idea how it can be fixed, short of skipping it whole? |
skops/io/tests/test_external.py
Outdated
|
||
X, y = make_classification(1000, 20, n_informative=10, random_state=0) | ||
clf.fit(X, y) | ||
dumped = dumps(clf, "keras-test.skops") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dumped = dumps(clf, "keras-test.skops") | |
dumped = dumps(clf) |
2nd argument to dumps
is the compression level. Honestly, I'm surprised that this didn't raise an error.
Ugh, the list of trusted modules is giant now :D I guess it's related to the tensorflow change. Could you please explain why that was necessary? Also, we now get this error on CI:
|
@BenjaminBossan Yeah, sorry I realized I had marked the test method as a |
@BenjaminBossan After staring at this all day, I could use some help lol. For some reason, there's some infinite recursion happening when constructing the tree that I can't figure out why. Initially, I thought it was due to the |
I'll have to check when I'm back. Still off till end of November. But I remember |
information for CachedNodes I'm wodering after looking at the types of a lot of the CachedNodes if there's something weird happening with `None`
198195f
to
ab530f3
Compare
5babada
to
0d8efca
Compare
67c2025
to
cc2aead
Compare
@adrinjalali I got it working for older versions of Python other than Windows on 3.8 (I think the Ubuntu 3.9 failure was just a codecov issue). Could you help me figure out how to get it working? Downgrading tensorflow to 2.11 just breaks more versions. |
One of the main reasons we support 3.8 was colab, which now supports 3.10. I think we can drop 3.8 alltogether. Would you be kind enough to submit a PR to remove 3.8 support? |
@adrinjalali I created this PR to remove 3.8 support #418, but I'm running into an issue where |
MacOS issue could be this: actions/setup-python#850 |
Intelex is very brittle, and I don't think it's used much if at all. I'd be in favor of removing it. It was only to handle the inference side and HF inference is not that well maintained anyway. @BenjaminBossan any objections to remove it? |
Works for me. |
@lazarust you've been very kind here. Would you remove |
@adrinjalali Done! I removed |
@lazarust let me know when the CI is green and ready for review here again. |
@adrinjalali Sounds good, it may take me a couple of days to get to it. It looks like there's something wrong in |
The errors seem like a bunch of deprecation warnings due to some updates in dependency packages maybe? |
@adrinjalali I created an issue in tensorflow to discuss this and make sure I wasn't missing anything tensorflow/tensorflow#68194. It seems like the best way forward is to ignore the warnings for now until Tensorflow supports |
Ignoring warnings in cases where we know why they're happening and with a comment as when the ignore statement should be removed is a normal practice indeed. Thanks for the follow up work on the TF side. |
d43cdd7
to
ce11bf0
Compare
@adrinjalali This should be ready for you to look at again. The failing tests seem to just be a blip with codecov |
WIP still need to fix one test
Reference Issues/PRs
Fixes #388
What does this implement/fix? Explain your changes.
This fixes a recursion error that was happening when dumping a scikeras model.
Any other comments?