Python 3 support? #64

Closed
nfd opened this Issue Jun 20, 2011 · 11 comments

Comments

Projects
None yet
4 participants
@nfd

nfd commented Jun 20, 2011

I've been a longtime user of python-markdown2 and like the way it just works. I'd like to start using it in a Python 3-based project, but it seems as if there is no goal of porting to Python 3 (I last checked it out a year or so ago -- at that time I did my own port).

Would you like me to contribute the port I have made, or is there no real motivation to support Python 3 at this time?

@trentm

This comment has been minimized.

Show comment Hide comment
@trentm

trentm Jul 8, 2011

Owner

@nfd: Sorry for the long time to respond. I'd definitely like to support Python 3. I suck that I haven't gotten to it in so long. I'm not sure how hairy the port will be. Ideally I'd really want to have a single markdown2.py that worked with both Python 2 and Python 3... but that depends on how gross that is (e.g. the hacks for supporting the different except: clause syntax, etc.).

Owner

trentm commented Jul 8, 2011

@nfd: Sorry for the long time to respond. I'd definitely like to support Python 3. I suck that I haven't gotten to it in so long. I'm not sure how hairy the port will be. Ideally I'd really want to have a single markdown2.py that worked with both Python 2 and Python 3... but that depends on how gross that is (e.g. the hacks for supporting the different except: clause syntax, etc.).

@flying-sheep

This comment has been minimized.

Show comment Hide comment
@flying-sheep

flying-sheep Aug 18, 2011

maybe you’ll want to hack 2to3 to do stuff it doesn’t do normally, i.e. specific fixes for your code.

so that running python2 setup.py install just installs the code and python3 setup.py install runs your modified 2to3 and then installs it.

sqlalchemy uses this approach and it works flawlessly.

maybe you’ll want to hack 2to3 to do stuff it doesn’t do normally, i.e. specific fixes for your code.

so that running python2 setup.py install just installs the code and python3 setup.py install runs your modified 2to3 and then installs it.

sqlalchemy uses this approach and it works flawlessly.

@bobuk

This comment has been minimized.

Show comment Hide comment
@bobuk

bobuk Apr 13, 2012

Today I was tried to port markdown2 with 2to3.
And looks like it works for everything excluding this part:

@@ -122,9 +122,9 @@
SECRET_SALT = str(uuid.uuid4())
def _hash_ascii(s):
#return md5(s).hexdigest() # Markdown.pl effectively does this.

  • return 'md5-' + md5(SECRET_SALT + s).hexdigest()
  • return 'md5-' + md5((SECRET_SALT + s).encode('utf-8')).hexdigest()
    def _hash_text(s):
  • return 'md5-' + md5(SECRET_SALT + s.encode("utf-8")).hexdigest()
  • return 'md5-' + md5((SECRET_SALT + s).encode("utf-8")).hexdigest()

bobuk commented Apr 13, 2012

Today I was tried to port markdown2 with 2to3.
And looks like it works for everything excluding this part:

@@ -122,9 +122,9 @@
SECRET_SALT = str(uuid.uuid4())
def _hash_ascii(s):
#return md5(s).hexdigest() # Markdown.pl effectively does this.

  • return 'md5-' + md5(SECRET_SALT + s).hexdigest()
  • return 'md5-' + md5((SECRET_SALT + s).encode('utf-8')).hexdigest()
    def _hash_text(s):
  • return 'md5-' + md5(SECRET_SALT + s.encode("utf-8")).hexdigest()
  • return 'md5-' + md5((SECRET_SALT + s).encode("utf-8")).hexdigest()
@trentm

This comment has been minimized.

Show comment Hide comment
@trentm

trentm Apr 13, 2012

Owner

Sorry guys for non-response on this. I try to poke away at this. My strong strong preference would be to have one markdown2.py file that worked with both Python 2 and 3. Typically I've found that possible, if a little awkward (with the unicode string literals and except-clause syntax differences).

Perhaps if we limited to Python 3.3 as a base for 3.x then we could use http://docs.python.org/dev/whatsnew/3.3.html#pep-414-explicit-unicode-literals to simplify the port. Would Python 3.3 as a base be good enough for you guys? Can probably get by either way.

I suspect the main work will be sorting out buglets related to str vs. bytes usage. The diff from 2to3 isn't that big so shouldn't take too long.

Owner

trentm commented Apr 13, 2012

Sorry guys for non-response on this. I try to poke away at this. My strong strong preference would be to have one markdown2.py file that worked with both Python 2 and 3. Typically I've found that possible, if a little awkward (with the unicode string literals and except-clause syntax differences).

Perhaps if we limited to Python 3.3 as a base for 3.x then we could use http://docs.python.org/dev/whatsnew/3.3.html#pep-414-explicit-unicode-literals to simplify the port. Would Python 3.3 as a base be good enough for you guys? Can probably get by either way.

I suspect the main work will be sorting out buglets related to str vs. bytes usage. The diff from 2to3 isn't that big so shouldn't take too long.

@flying-sheep

This comment has been minimized.

Show comment Hide comment
@flying-sheep

flying-sheep Apr 13, 2012

well, there are severyl ways to tackle this:

  1. use six
  2. use the subset of python 2 and 3 (very limiting)
  3. use 2to3 and make sure it does a good job on your code
  4. use a monkeypatched 2to3 (like sqlalchemy does)
  5. use a preprocessor and any combination of the above.

see also here.

i think that a combination of 3. and 4. ist the way to go, but i don’t have experience here, it’s just what seem the most sensible choice.

well, there are severyl ways to tackle this:

  1. use six
  2. use the subset of python 2 and 3 (very limiting)
  3. use 2to3 and make sure it does a good job on your code
  4. use a monkeypatched 2to3 (like sqlalchemy does)
  5. use a preprocessor and any combination of the above.

see also here.

i think that a combination of 3. and 4. ist the way to go, but i don’t have experience here, it’s just what seem the most sensible choice.

@trentm

This comment has been minimized.

Show comment Hide comment
@trentm

trentm Apr 13, 2012

Owner

Thanks for the breakdown!

  1. Might steal snippets from six if useful. I don't want any externals deps for markdown2.py.
  2. I'm hoping this will be sufficient.

For "3-5", what is the deployment scenario here? Do I have two copies of markdown2.py commited to my repo (not great, but not awful)? Do I have this 2to3 processing (or 3to2 if/when making Python 3 the primary target version) happen in setup.py during distutils/distutils2/distribute/setup.py/easy_install/pip/pypm install (yuk)?

Owner

trentm commented Apr 13, 2012

Thanks for the breakdown!

  1. Might steal snippets from six if useful. I don't want any externals deps for markdown2.py.
  2. I'm hoping this will be sufficient.

For "3-5", what is the deployment scenario here? Do I have two copies of markdown2.py commited to my repo (not great, but not awful)? Do I have this 2to3 processing (or 3to2 if/when making Python 3 the primary target version) happen in setup.py during distutils/distutils2/distribute/setup.py/easy_install/pip/pypm install (yuk)?

@flying-sheep

This comment has been minimized.

Show comment Hide comment
@flying-sheep

flying-sheep Apr 13, 2012

since your goal for markdown2 seems to be that it stays one file (which can be dropped in any project), runtime dependecies like six don’t seem to be an option.

but you can use 2to3 in the setup.py. that’s exactly what sqlalchemy does. the linked article is from the autor of sqlalcemy, so reading sqlalchemy’s sources or at least the article should get us started. in fact, the article tells us how to do it the way he did it with sqlalchemy, structured in a way that we can stop as sooon as the current step is already sufficient.

since your goal for markdown2 seems to be that it stays one file (which can be dropped in any project), runtime dependecies like six don’t seem to be an option.

but you can use 2to3 in the setup.py. that’s exactly what sqlalchemy does. the linked article is from the autor of sqlalcemy, so reading sqlalchemy’s sources or at least the article should get us started. in fact, the article tells us how to do it the way he did it with sqlalchemy, structured in a way that we can stop as sooon as the current step is already sufficient.

@trentm

This comment has been minimized.

Show comment Hide comment
@trentm

trentm Apr 18, 2012

Owner

Woot. Getting there:

$ py2 lib/markdown2.py foo.md 
<p><em>hi</em></p>
$ py3 lib/markdown2.py foo.md 
<p><em>hi</em></p>
Owner

trentm commented Apr 18, 2012

Woot. Getting there:

$ py2 lib/markdown2.py foo.md 
<p><em>hi</em></p>
$ py3 lib/markdown2.py foo.md 
<p><em>hi</em></p>
@trentm

This comment has been minimized.

Show comment Hide comment
@trentm

trentm Apr 18, 2012

Owner

https://gist.github.com/2411884
markdown2 passing in Python 2.5 - 3.3

Will have a release soon with this.

Owner

trentm commented Apr 18, 2012

https://gist.github.com/2411884
markdown2 passing in Python 2.5 - 3.3

Will have a release soon with this.

@flying-sheep

This comment has been minimized.

Show comment Hide comment
@flying-sheep

flying-sheep Apr 18, 2012

how did you do it? did you use a preprocessor for from __future__ imports and stuff like items/iteritems?

how did you do it? did you use a preprocessor for from __future__ imports and stuff like items/iteritems?

trentm added a commit that referenced this issue Apr 20, 2012

@trentm

This comment has been minimized.

Show comment Hide comment
@trentm

trentm Apr 20, 2012

Owner

This is in a python-markdown2 1.4.0 release (http://pypi.python.org/pypi/markdown2).

@flying-sheep: Yup, basically playing it safe with compat 2 and 3 syntax, a few compat defines at the top (e.g. 'bytes' and 'unicode' usage). I started with a 2to3 conversion and worked backwards to get Python 2 working again. Most of the work was in getting subtle unicode-y semantics working.

Note: It is possible I've broken Python 2.4 here as I don't have that Python version on my Mac. I'm going to try setting up travis-ci to see if that can test Python 2.4 for me. I also have Python 2.4 at work to play with.

Owner

trentm commented Apr 20, 2012

This is in a python-markdown2 1.4.0 release (http://pypi.python.org/pypi/markdown2).

@flying-sheep: Yup, basically playing it safe with compat 2 and 3 syntax, a few compat defines at the top (e.g. 'bytes' and 'unicode' usage). I started with a 2to3 conversion and worked backwards to get Python 2 working again. Most of the work was in getting subtle unicode-y semantics working.

Note: It is possible I've broken Python 2.4 here as I don't have that Python version on my Mac. I'm going to try setting up travis-ci to see if that can test Python 2.4 for me. I also have Python 2.4 at work to play with.

@trentm trentm closed this Apr 20, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment