-
-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CLN/ENH/BLD: Remove need for 2to3 for Python 3. #4384
Conversation
I think GitHub might explode if I click on the files changed tab 💥 |
I think |
@cpcloud it is...still coming up on my list :P |
@cpcloud actually, only used in one place in library and it's used as an iterator anyways. still need to check for itertools changes. |
great. i think i might use |
@cpcloud just put |
cool thanks! pandas is going to rule the world soon. |
i hope python 3 testing is going to be faster after this .... those 2to3 runs seem like they take FOREVER |
@cpcloud I think it makes the entire test suite a little faster...not sure. |
nice looks like travis build times are 3-4 mins faster that is cool |
@cpcloud does Cython have any changes for Python 3? |
I believe all of those issues are taken care of during translation from the Cython to C. I don't see why any Cython would need to be changed. Are you getting compilation errors? |
@cpcloud guess I can empirically see that that's not true...just gone through so many changes (and I can't get py3 pandas to work on my mac). |
@cpcloud I'm getting all these weird stata reader errors. Do you have any ideas on what might be causing them? |
@jtratner can u link me to a build with those errors? |
@cpcloud check out the first few errors on here: https://travis-ci.org/jtratner/pandas/jobs/9563671 |
have you tried stepping with a debugger? |
@cpcloud I've been having trouble getting pandas with Python 3 to install on my mac, which is why I haven't been able to step through. |
@cpcloud but eventually I'll figure out how to get it to work |
ah ok... it's weird because i think you actually have to |
@cpcloud well, the current build has 2to3 disabled, so you don't actually need to do that. I'm trying a separate build with 2to3 enabled to see if it resolves the problem. |
indeed, i hadn't pulled down your PR in my py3.2 venv yet 😄 |
anyway my dev version works so just keep at it i think you'll get it ... i would offer to clone my env but i'm not using mac |
@cpcloud okay, pretty sure I can figure it out. It doesn't fail with any of the stata errors with 2to3, so that means I just have to apply some of those fixes and it will work :). |
for some reason the |
(in |
@cpcloud thanks. And I found all the non-stata errors are resolved by 2to3'ing pandas/core. |
ahh so the problem is actually not in |
that's in the |
i love getting to the bottom of things! |
sorry i wrote that on the wrong thread...see the eval thread! |
rebases all worked (with a little TLC..) except for 1 thing:
|
Maybe I put it in as future warning by accident? Or did we lose the
|
@jreback oh didn't see your last line, I have an idea...let me look. |
@jreback which branch? |
msgpack3 (simpler one) |
Why does it require pytest? ======================================================================
ERROR: Failure: ImportError (No module named 'pytest')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/failure.py", line 38, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/loader.py", line 413, in loadTestsFromName
addr.filename, addr.module)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/imp.py", line 160, in load_module
return load_source(name, filename, file)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/imp.py", line 109, in load_source
return _LoadSourceCompatibility(name, pathname, file).load_module(name)
File "<frozen importlib._bootstrap>", line 586, in _check_name_wrapper
File "<frozen importlib._bootstrap>", line 1023, in load_module
File "<frozen importlib._bootstrap>", line 1004, in load_module
File "<frozen importlib._bootstrap>", line 562, in module_for_loader_wrapper
File "<frozen importlib._bootstrap>", line 869, in _load_module
File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
File "/Users/jtratner/projects/python/pandas3/pandas/tests/test_msgpack/test_except.py", line 4, in <module>
from pytest import raises
ImportError: No module named 'pytest'
======================================================================
ERROR: Failure: ImportError (No module named 'pytest')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/failure.py", line 38, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/loader.py", line 413, in loadTestsFromName
addr.filename, addr.module)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/imp.py", line 160, in load_module
return load_source(name, filename, file)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/imp.py", line 109, in load_source
return _LoadSourceCompatibility(name, pathname, file).load_module(name)
File "<frozen importlib._bootstrap>", line 586, in _check_name_wrapper
File "<frozen importlib._bootstrap>", line 1023, in load_module
File "<frozen importlib._bootstrap>", line 1004, in load_module
File "<frozen importlib._bootstrap>", line 562, in module_for_loader_wrapper
File "<frozen importlib._bootstrap>", line 869, in _load_module
File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
File "/Users/jtratner/projects/python/pandas3/pandas/tests/test_msgpack/test_obj.py", line 4, in <module>
from pytest import raises
ImportError: No module named 'pytest'
======================================================================
ERROR: Failure: ImportError (No module named 'pytest')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/failure.py", line 38, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/loader.py", line 413, in loadTestsFromName
addr.filename, addr.module)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/imp.py", line 160, in load_module
return load_source(name, filename, file)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/imp.py", line 109, in load_source
return _LoadSourceCompatibility(name, pathname, file).load_module(name)
File "<frozen importlib._bootstrap>", line 586, in _check_name_wrapper
File "<frozen importlib._bootstrap>", line 1023, in load_module
File "<frozen importlib._bootstrap>", line 1004, in load_module
File "<frozen importlib._bootstrap>", line 562, in module_for_loader_wrapper
File "<frozen importlib._bootstrap>", line 869, in _load_module
File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
File "/Users/jtratner/projects/python/pandas3/pandas/tests/test_msgpack/test_pack.py", line 6, in <module>
from pytest import raises, xfail
ImportError: No module named 'pytest'
======================================================================
ERROR: Failure: ImportError (No module named 'pytest')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/failure.py", line 38, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/loader.py", line 413, in loadTestsFromName
addr.filename, addr.module)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/imp.py", line 160, in load_module
return load_source(name, filename, file)
File "/Users/jtratner/.virtualenvs/pandas3/lib/python3.3/imp.py", line 109, in load_source
return _LoadSourceCompatibility(name, pathname, file).load_module(name)
File "<frozen importlib._bootstrap>", line 586, in _check_name_wrapper
File "<frozen importlib._bootstrap>", line 1023, in load_module
File "<frozen importlib._bootstrap>", line 1004, in load_module
File "<frozen importlib._bootstrap>", line 562, in module_for_loader_wrapper
File "<frozen importlib._bootstrap>", line 869, in _load_module
File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
File "/Users/jtratner/projects/python/pandas3/pandas/tests/test_msgpack/test_sequnpack.py", line 7, in <module>
from pytest import raises
ImportError: No module named 'pytest'
======================================================================
FAIL: test_iterkv_deprecation (pandas.tests.test_frame.TestDataFrame)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/jtratner/projects/python/pandas3/pandas/tests/test_frame.py", line 10316, in test_iterkv_deprecation
self.mixed_float.iterkv()
File "/usr/local/Cellar/python3/3.3.0/Frameworks/Python.framework/Versions/3.3/lib/python3.3/contextlib.py", line 55, in __exit__
next(self.gen)
File "/Users/jtratner/projects/python/pandas3/pandas/util/testing.py", line 1058, in assert_produces_warning
% expected_warning.__name__)
AssertionError: Did not see expected warning of class 'DeprecationWarning'.
----------------------------------------------------------------------
Ran 3537 tests in 166.243s |
its there from where when msgpack tests originally created; been meaning to take it out |
@jreback easy to put raises-equivalent into Also, you should change this line: unpacker = Unpacker(six.BytesIO(b'foobar'), read_size=3) to the |
what about the rest of them that directly import from six? |
hold on, I have a diff for you. |
I fixed them all....how do I do raw bytes? |
yeah, that's cross-compatible. I just posted a commit on jtratner/msgpack3 that fixes everything. Main thing is you need to pass string encoding in test_seq.py and wrap around |
(hadn't seen your note before that...) |
|
nvm, my example doesn't work for that |
actually assert_raises is in |
look at test_msgpack/test_seq.py I had to hack a bit, encoding in py3, but passing str in py2...seems to work, but never could figure out why |
@jreback well, in Python 3, everything is unicode, whereas in py2 everything is encoded by default, so using bytes requires setting explicit coding. In python 2 |
@jtratner I resolved that issue with assert_produces_warning.... I was calling IOW I was calling iterkv accidently, which produced a warning (but I think globally warnings are somehow turned off for DeprecationWarning)..... then when it hit the specific test, warnings were still turned off! weird |
warnings are incredibly finicky. I'm -1 on checking for things like |
agreed.....I in fact usually when I test something like this I make it raise while I am testing (which in this case would have found the cases where I didn't fix the deprecated routine).... so maybe need something that is a sub-class of DeprecationWarning that assert_produces_warning can really 'catch' (but in normal usage just spews a warning)? just thinking out loud here |
there's no runtime switch to turn on deprecation warnings? |
http://ondrejcertik.blogspot.com/2013/08/how-to-support-both-python-2-and-3.html?m=1 I guess we had an easier time than SymPy |
There were two particularly hard parts:
|
Fixes #4375 and #4372.
Many changes to make codebase compatible in 2 and 3. For
range
,zip
,map
, etc. tried to favor iterators over needing to use lists.Changes:
dateutil
for handling unicode when version <= 2.0 + 1.5 dateutil build in TravisSwitches everything to use special iteritems, so can deprecate as discussed in #4372.
This iteritems use "iteritems" method if available and otherwise uses items.
Checklist of packages to run through 2to3 to check:
Other tasks:
__bool__
to__nonzero__