Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stage 2 of Porting Python 2 Code to Python 3 #40

Merged
merged 5 commits into from
Mar 20, 2019

Conversation

roaldnefs
Copy link
Contributor

@roaldnefs roaldnefs commented Feb 2, 2019

This PR is part of porting the Python 2 code to Python 3 without losing compatibility for Python 2.7 according to https://python-future.org/futurize.html.

Note: this PR is still work in progress!

Description:

Stage 1 will apply fixes to modernize the Python 2 code without changing the effect of the code. The goal of this stage is to create most of the diff for the entire porting process, but without introducing any bugs.

Stage 2 will add a dependency on the future package. The goal for stage 2 is to make further mostly safe changes to the Python 2 code to use Python 3 style code that still runs on Python 2 with the help of the appropriate builtins and utilities in future.

In stage 3 we will be separating text from bytes manually by prefixing all the string literals with either b or u accordingly. To ensure that these type behave similarly on Python 2 as on Python 3, we will wrap byte-strings or text in the bytes and str types from future.

After running futurize, we start by testing compatibility on Python 3 and making further code changes until it's fully compatible with Python 3. The next step is to make manual tweaks to the code to re-enable Python 2 compatibility.

Action Items:

  • Drop support for Python 2.6 and older (Drop support for Python 2.6 and older #33)
  • Stage 1 of futurize (except for vstruct/*)
  • Manually fix commented code and exception messages that aren't correctly ported by futurize.
  • Merge stage 1 changes into master (most of the diff, minimal to no impact) (Intial Porting of Python 2 Code to Python 3 #34)
  • Stage 2 of futurize (except for vstruct/*)
  • Merge stage 2 changes into master
  • Stage 1 and 2 of futurize in vstruct/*
  • Merge futurize changes in vstruct/* into master
  • Separating text from bytes manually
  • Test & make the code pass on Python 3
  • Manually tweak to re-enable Python 2 compatibility
  • Merge changes into master

Additional Information:

This PR is part of fixing #15.

Add dependency on the future package, required for stage 2 of
futurize.
Stage 2 of futurize in `rflib/*.py`. The goal of stage 2 is to make
further mostly safe changes to the Python 2 code to use Python
3-style code that still runs on Python 2 with the help of the
appropriate builtins and utilities in `future`.
@roaldnefs roaldnefs changed the title Stage 2 of Porting Python 2 Code to Python 3 WIP: Stage 2 of Porting Python 2 Code to Python 3 Feb 2, 2019
@roaldnefs
Copy link
Contributor Author

Automatically run stage 2 of futurize (except for vstruct/*) on all Python files. Futurize did not rewrite all divisions, and I think we we need to fix the long to int changes manually.

Note: this PR is still work in progress!

The Python 2 integer types int and long have been unified in Python
3. So there is now only one type, int.
Import cPickle on Python 2 and pickle on Python 3.
@atlas0fd00m
Copy link
Owner

you waiting on me? or am i waiting on you? :) bump if i fall off the face of the planet.

@roaldnefs
Copy link
Contributor Author

@atlas0fd00m, you are waiting on my! I am currently very busy, hope to get some work done on the PR next weekend.

Update chr() to bytes([]) after Futurize stage 2 caused errors. For
more information see: https://python-future.org/compatible_idioms.html#chr
@roaldnefs roaldnefs changed the title WIP: Stage 2 of Porting Python 2 Code to Python 3 Stage 2 of Porting Python 2 Code to Python 3 Mar 17, 2019
@roaldnefs
Copy link
Contributor Author

@atlas0fd00m Finally got some spare time to work on this PR, so it's now ready to be merged. Unfortunately I don't have much time to work on this in the coming weeks...

aa.sort()
return aa

def minaddr(self):
'''Get minimal address of HEX content.
@return minimal address or None if no data
'''
aa = self._buf.keys()
aa = list(self._buf.keys())
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i notice a lot of this. is there a problem accessing the list returned from keys() in Py3? or is this a stylistic thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Python 3 the dict.keys() will return a dict_keys object instead of a list, e.g.:

>>> my_dict = {'first': 1, 'second': 2} 
>>> my_dict.keys()
dict_keys(['first', 'second'])
>>> list(my_dict.keys())
['first', 'second']

For more information see: PEP 3106 -- Revamping dict.keys(), .values() and .items().

@atlas0fd00m atlas0fd00m merged commit 7068761 into atlas0fd00m:master Mar 20, 2019
@roaldnefs roaldnefs mentioned this pull request Dec 23, 2019
@cutaway
Copy link

cutaway commented Mar 11, 2020

@roaldnefs where do we stand on this? How can I help?

Also, do you have a recommended Python ENV setup so I'm working from the same place as you, @atlas0fd00m, and others

@roaldnefs
Copy link
Contributor Author

@cutaway, the description of this PR is up to date with my latest changes. You may also take a look at #15, this issue contains some extra information about achieving Python 3 compatibility.

I just use virtualenv to create a virtual environment for both Python 2 and 3.

Tip: python-future.org is a greate resource!

@atlas0fd00m
Copy link
Owner

atlas0fd00m commented Jul 13, 2020

@roaldnefs, @cutaway how are we doing on the next phase of py3 migration? :)
any progress to speak of? and plans?
thank you again for your hard work so far!
@

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants