New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port code to Python 3 #93

Closed
inquisb opened this Issue Jul 16, 2012 · 13 comments

Comments

Projects
None yet
5 participants
@inquisb
Member

inquisb commented Jul 16, 2012

Currently there is no pressure on Python projects to switch to the new version of Python interpreter, as the process of switching, especially on larger projects can be cumbersome (due to the few backward incompatibilities). The switch will take place eventually, but currently it is a very low priority task.

@stamparm

This comment has been minimized.

Show comment
Hide comment

@chym chym referenced this issue Nov 10, 2012

Closed

error-based false #231

@stamparm

This comment has been minimized.

Show comment
Hide comment
@stamparm

stamparm Dec 8, 2012

Member

As I've collected information regarding this over time, this will be maybe an impossible task to do because of one feature: "unicode strings". We currently extensively use unicode everywhere together with encoding/decoding for page responses. Problem is that Python 2 has a clear distinction between unicode (e.g. decoded page response using stated charset) and non-unicode data (e.g. raw page response), while Python 3 doesn't have.

Good reference: http://lucumr.pocoo.org/2010/1/7/pros-and-cons-about-python-3/

Member

stamparm commented Dec 8, 2012

As I've collected information regarding this over time, this will be maybe an impossible task to do because of one feature: "unicode strings". We currently extensively use unicode everywhere together with encoding/decoding for page responses. Problem is that Python 2 has a clear distinction between unicode (e.g. decoded page response using stated charset) and non-unicode data (e.g. raw page response), while Python 3 doesn't have.

Good reference: http://lucumr.pocoo.org/2010/1/7/pros-and-cons-about-python-3/

@inquisb inquisb closed this Feb 15, 2013

@ghost ghost assigned stamparm Jul 17, 2013

@badbob

This comment has been minimized.

Show comment
Hide comment
@badbob

badbob Jul 16, 2015

Is there any news for this issue? Does Python 3.4 have required functionality?

badbob commented Jul 16, 2015

Is there any news for this issue? Does Python 3.4 have required functionality?

@stamparm

This comment has been minimized.

Show comment
Hide comment
@stamparm

stamparm Jul 16, 2015

Member

@badbob I would say no, but that's my opinion. Only small projects are prone to conversion from 2 to 3. I have a feeling that this is a futile task for sqlmap.

Member

stamparm commented Jul 16, 2015

@badbob I would say no, but that's my opinion. Only small projects are prone to conversion from 2 to 3. I have a feeling that this is a futile task for sqlmap.

@badbob

This comment has been minimized.

Show comment
Hide comment
@badbob

badbob Jul 18, 2015

Looks like the only way to reuse sqlmap code in Python 3 solutions is a REST API. Thanks for developers sqlmap has it. I can't see any other variant.

badbob commented Jul 18, 2015

Looks like the only way to reuse sqlmap code in Python 3 solutions is a REST API. Thanks for developers sqlmap has it. I can't see any other variant.

@Flimm

This comment has been minimized.

Show comment
Hide comment
@Flimm

Flimm Feb 15, 2017

Problem is that Python 2 has a clear distinction between unicode (e.g. decoded page response using stated charset) and non-unicode data (e.g. raw page response), while Python 3 doesn't have.

I would highly disagree with this statement. In Python 2, strings represented either textual data or raw data (often raw data of textual data in an assumed or unknown encoding), while unicode objects represented textual data. Strings in Python 2 were by nature ambiguous. In Python 3, bytes objects represent raw data, and string objects represent textual data. It's crystal clear in Python 3, and it's enforced with exceptions when mixing strings with bytes objects. This crystal clear distinction exposes bad design choices in certain systems (for instance, the ambiguous encodings of URLs, or of filenames), and it causes the programmer to have to consider these problems upfront. Python 2 just let you stuff everything into a string, and it gives you runtime exceptions sometime in the future when you try anything not in ASCII.

I'm not trying to argue that sqlmap should necessarily be ported to Python 3. I just felt someone needed to speak up and defend Python 3's explicitness when it comes to strings and bytes.

Python 2.7 will no longer be supported past 2020, as advertised by this website: https://pythonclock.org/ Many scientific libraries for Python have announced that they will also no longer support Python 2.x http://www.python3statement.org/

If the owners of this project would welcome efforts to port this project to Python 3, perhaps create an issue saying so, with the tag help-wanted.

Flimm commented Feb 15, 2017

Problem is that Python 2 has a clear distinction between unicode (e.g. decoded page response using stated charset) and non-unicode data (e.g. raw page response), while Python 3 doesn't have.

I would highly disagree with this statement. In Python 2, strings represented either textual data or raw data (often raw data of textual data in an assumed or unknown encoding), while unicode objects represented textual data. Strings in Python 2 were by nature ambiguous. In Python 3, bytes objects represent raw data, and string objects represent textual data. It's crystal clear in Python 3, and it's enforced with exceptions when mixing strings with bytes objects. This crystal clear distinction exposes bad design choices in certain systems (for instance, the ambiguous encodings of URLs, or of filenames), and it causes the programmer to have to consider these problems upfront. Python 2 just let you stuff everything into a string, and it gives you runtime exceptions sometime in the future when you try anything not in ASCII.

I'm not trying to argue that sqlmap should necessarily be ported to Python 3. I just felt someone needed to speak up and defend Python 3's explicitness when it comes to strings and bytes.

Python 2.7 will no longer be supported past 2020, as advertised by this website: https://pythonclock.org/ Many scientific libraries for Python have announced that they will also no longer support Python 2.x http://www.python3statement.org/

If the owners of this project would welcome efforts to port this project to Python 3, perhaps create an issue saying so, with the tag help-wanted.

@stamparm

This comment has been minimized.

Show comment
Hide comment
@stamparm

stamparm Feb 15, 2017

Member

I'll just leave this here. It is not an argument for anything, just to get the clear picture of current situation in Python world.

3

Reference: https://hynek.me/articles/python3-2016/

Member

stamparm commented Feb 15, 2017

I'll just leave this here. It is not an argument for anything, just to get the clear picture of current situation in Python world.

3

Reference: https://hynek.me/articles/python3-2016/

@stamparm

This comment has been minimized.

Show comment
Hide comment
@stamparm

stamparm Feb 15, 2017

Member

@Flimm sqlmap currently has around 64K LoC (excluding comments and blank lines). Running simple 2to3 is not an option here (if you think that 2to3 is enough then you are utterly wrong). As I can see from other projects, estimates for conversion goes from 3-6 months on projects of this size. As I've solely done the unicode support here, I believe that I have a right to say that switching of unicode related stuff from Python 2 to Python 3 is one of the biggest obstacles for this whole switch. This is not only my opinion. Just Google around.

Let us all together wait for the 2020. I have a feeling that it will pass as any other (Python) year. I bet that nothing will change as there were couple of similar (failed) attempts in past couple of years.

Member

stamparm commented Feb 15, 2017

@Flimm sqlmap currently has around 64K LoC (excluding comments and blank lines). Running simple 2to3 is not an option here (if you think that 2to3 is enough then you are utterly wrong). As I can see from other projects, estimates for conversion goes from 3-6 months on projects of this size. As I've solely done the unicode support here, I believe that I have a right to say that switching of unicode related stuff from Python 2 to Python 3 is one of the biggest obstacles for this whole switch. This is not only my opinion. Just Google around.

Let us all together wait for the 2020. I have a feeling that it will pass as any other (Python) year. I bet that nothing will change as there were couple of similar (failed) attempts in past couple of years.

@Flimm

This comment has been minimized.

Show comment
Hide comment
@Flimm

Flimm Feb 15, 2017

So you wouldn't welcome efforts to port this to Python 3?

Flimm commented Feb 15, 2017

So you wouldn't welcome efforts to port this to Python 3?

@stamparm

This comment has been minimized.

Show comment
Hide comment
@stamparm

stamparm Feb 15, 2017

Member

@Flimm efforts of porting sqlmap in one go? Impossible. sqlmap is a dynamic project with steady commits. Making a parallel branch and keeping up the pace with HEAD Python 2 changes would be a Sisyphean task. Also, there are parts of sqlmap that maybe couple of non-sqlmap developers would understand. If a converter (person doing the conversion) would get stuck inside, he'll have a hell of a time.

My conclusion. Let's wait for the 2020. If Python 2 will die I promise that I'll do the whole conversion thing - as I believe that nobody else would be able to do it.

Member

stamparm commented Feb 15, 2017

@Flimm efforts of porting sqlmap in one go? Impossible. sqlmap is a dynamic project with steady commits. Making a parallel branch and keeping up the pace with HEAD Python 2 changes would be a Sisyphean task. Also, there are parts of sqlmap that maybe couple of non-sqlmap developers would understand. If a converter (person doing the conversion) would get stuck inside, he'll have a hell of a time.

My conclusion. Let's wait for the 2020. If Python 2 will die I promise that I'll do the whole conversion thing - as I believe that nobody else would be able to do it.

@Flimm

This comment has been minimized.

Show comment
Hide comment
@Flimm

Flimm Feb 15, 2017

OK. Fair enough. I trust your experience in this project.

Flimm commented Feb 15, 2017

OK. Fair enough. I trust your experience in this project.

@liwt31

This comment has been minimized.

Show comment
Hide comment
@liwt31

liwt31 Jun 20, 2018

I'm also not arguing something, just to leave another picture, which I found in https://hynek.me/articles/python3-2016/:
image

liwt31 commented Jun 20, 2018

I'm also not arguing something, just to leave another picture, which I found in https://hynek.me/articles/python3-2016/:
image

@stamparm

This comment has been minimized.

Show comment
Hide comment
@stamparm

stamparm Jun 20, 2018

Member

@liwt31 that article is excellent. Everybody should take 5 minutes and read it. IMO your picture is way out of context :). Article explains in a great way what's the status of old and newly started projects.

One important part from me. Python3 has been introduced in a bad way. It has split the community and introduced more problems than benefits. Now, whoever forces changes of big Python2 projects to Python3 is IMO not a programmer as programmers wouldn't do that kind of stuff to other fellow programmers. It's like forcing usage of objects and C++ to established C Linux/FOSS projects.

Already made a statement. If Python2 is going to be killed I'll convert/rewrite everything to Python3. Up until then new Python programmers can live in a lie that Python3 HAD to be hard forked from Python2

Member

stamparm commented Jun 20, 2018

@liwt31 that article is excellent. Everybody should take 5 minutes and read it. IMO your picture is way out of context :). Article explains in a great way what's the status of old and newly started projects.

One important part from me. Python3 has been introduced in a bad way. It has split the community and introduced more problems than benefits. Now, whoever forces changes of big Python2 projects to Python3 is IMO not a programmer as programmers wouldn't do that kind of stuff to other fellow programmers. It's like forcing usage of objects and C++ to established C Linux/FOSS projects.

Already made a statement. If Python2 is going to be killed I'll convert/rewrite everything to Python3. Up until then new Python programmers can live in a lie that Python3 HAD to be hard forked from Python2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment