remove engine overwrite in pandas TextFileReader patch #791
Conversation
|
Hm, that is strange. The patch was needed because with the "c" backend, filesystem calls cannot be patched. If I comment it out locally, the respective tests ( |
|
I'm still not quite sure what is going on, but in the CI tests the patched code is never called - it seems to go another path that uses the "python" engine by default. So in this case the change has no effect. If the code is called (as in my local tests), the tests fail with the change, so I don't see what the PR would achieve. |
|
Ok, I now understand what is going on, and your patch makes sense, after all - just a bit changed. Sorry for the confusion... You can adapt the PR accordingly, if you want (will add a couple of comments in a moment), or I can make the changes myself. |
|
|
||
| class TextFileReader(parsers.TextFileReader): | ||
| def __init__(self, *args, **kwargs): | ||
| kwargs["engine"] = "python" |
There was a problem hiding this comment.
The whole patching of pandas (all what is currently inside the if parsers is not None conditions) can be removed for pandas >= 1.2. You can do something like this for the new condition:
import pandas as pd
...
patch_pandas = (
parsers is not None and [int(v) for v in pd.__version__.split(".")] < [1, 2, 0]
)Also, please add an entry to the release notes - thanks!
|
Oh, and I'm afraid that this will not help you with version 1.1.5 - this still needs the patch to work correctly. Though if you don't need that functionality, you can always switch off that patching, as mentioned before. |
pyfakefs/patched_packages.py
Outdated
| if parsers is not None: | ||
| # From pandas v 1.2 onwards the python fs functions are used even when the engine selected is "c". | ||
| # This means that we don't explicitly have to change the engine. | ||
| patch_pandas = ( |
There was a problem hiding this comment.
There are two other places above with use if parsers is not None - these also have to be adapted to use patch_pandas instead.
There was a problem hiding this comment.
Got it, let me update.
|
Thanks for all your help !! |
2 participants
Describe the changes
TextFileReader currently overwrites the engine as python when patching for pandas. I am not sure why is it needed since the default engine is "c" and "python" engine doesn't expects certain params like "float_precision" which results in it failing with the version of pandas that we have (1.1.5).
Tasks