python 3 support

[BurntSushi]

I think it's about time we have an issue for this. At some point, we need to press forward. I have a small hiatus coming up that I could use to push us to Python 3.

The good news is that all dependencies (pytz, beautifulsoup4, httplib2 and psycopg2 (for nfldb)) are supported on Python 3. So the only task is to make the actual nflgame module Python 3 compatible.

There are two ways forward. The first is to migrate to Python 3 and leave Python 2 in the wind. The second is to simultaneously support Python 2 and 3 by writing a subset of each that is compatible with both versions.

Writing Python 2/3 compatible code is difficult. I've done it with pdoc and it is not fun. Also, I don't find the prospect of debugging user problems with Python 2/3 code very enticing.

Writing code that is only compatible with Python 3 is much simpler and easier to maintain. However, it makes the library unusable to people who cannot run Python 3. It will also force everyone to upgrade their personal scripts that use nflgame to Python 3. That's a major bummer.

[BurntSushi]

I think I'll experiment with the Python 2/3 approach first. nflgame and nfldb don't do much with string data, so it should be relatively easy to handle the bytes/str/unicode trichotomy fiasco.

I think the thing that will trouble me the most are the subtle changes from lists to generators in several builtin functions/methods. For example, dict.items is a list in Python 2 but a generator in Python 3 (the generator in Python 2 is dict.iteritems). All of this will require some sort of wrapper.

I need to also make a decision on whether to use the six library or not. I didn't with pdoc, but I could get away with not caring much about the generator issue. With six, they expect us to use six.iteritems(dictionary) instead of dictionary.iteritems() which I find less than ideal.

Ug.

[BurntSushi]

I would very much love to have feedback from users on this issue. It could have a giant impact on the path I take. I think these are probably the most important questions:

• If Python 2 support was removed, what is the likelihood that you'll port your existing scripts to Python 3?
• How much work would it require?
• If we did Python 2/3 support for, say, the next season and then dropped Python 2 completely going forward, would that increase the likelihood that you'll move your scripts to Python 3?

cc @ochawkeye @front9tech @cminton @ersherr @weixiyen @teamfball @dfd

[BurntSushi]

This is blocking on #79. Making a change as substantial as this one without tests is lunacy.

[BurntSushi]

To be clear, "substantial change" refers to implementation. A port to Python 3 shouldn't result in any public API changes.

[cminton]

I am okay moving to Python 3. To be fair, I have not developed enough with nflgame, so the impact for me should be minimal.

Regards,
Charlie

On May 26, 2014, at 3:58 PM, Andrew Gallant notifications@github.com wrote:

To be clear, "substantial change" refers to implementation. A port to Python 3 shouldn't result in any public API changes.

—
Reply to this email directly or view it on GitHub.

[ersherr]

Quick answers re my specific project to the questions you asked:

1. I would definitely move my analysis scripts to Python 3 if the library
   moved
2. It doesn't seem like it would require much work
3. The library spending a year in a limbo/flexible state of 2/3 wouldn't
   affect my likelihood of moving either way

Thanks, awesome library + OK w/ whichever path!

On Mon, May 26, 2014 at 6:04 PM, Charlie Minton notifications@github.comwrote:

I am okay moving to Python 3. To be fair, I have not developed enough with
nflgame, so the impact for me should be minimal.

Regards,
Charlie

On May 26, 2014, at 3:58 PM, Andrew Gallant notifications@github.com
wrote:

To be clear, "substantial change" refers to implementation. A port to
Python 3 shouldn't result in any public API changes.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/78#issuecomment-44220607
.

[front9tech]

I haven't researched the differences in Python 3 much since I only really use Python for nflgame but moving my scripts entirely to Python 3 would be fine by me as well. I would continue to use the library regardless of any level of support for Python 2. Thanks for all of your work on this library.

[front9tech]

Here's an interesting discussion that just happens to be at the top of Hacker News right now - https://news.ycombinator.com/item?id=7801834

[BurntSushi]

@front9tech That's what prompted this issue. :-)

[front9tech]

Ah figures ;)

On Mon, May 26, 2014 at 8:22 PM, Andrew Gallant notifications@github.com
wrote:

@front9tech That's what prompted this issue. :-)

Reply to this email directly or view it on GitHub:
#78 (comment)

[albertlyu]

I've been watching this repo for some time now, and having just learned Python several months ago, I (perhaps naively) started out with Python 3.3. I've since installed Python 2.7 in preparation for creating a web app based off of your database, but I would welcome a port to Python 3 and wait on the completion of this issue before beginning to dig into your projects. I'll be watching your updates closely, as I'm curious as to what updates it takes to port Python 2 to 3 beyond converting all your print statements. Thanks again to the devs, especially @BurntSushi for continuing to maintain this project!

[BurntSushi]

@albertlyu This is the best guide for porting: http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ --- Basically, the key is to drop support for Python <=2.5 and Python 3.{0,1,2}. Then things get somewhat easier.

I would definitely encourage you to dig into nflgame and/or nfldb even with Python 2.7 though. If you're just writing small scripts, they will be easy to change to Python 3 going forward.

FYI, try using this in your Python 2.7 programs:

from __future__ import absolute_import, division, print_function, unicode_literals

Basically, it enables a few things from Python 3 in your Python 2 program. For example, it disables print as a statement and allows you to use it as a function. This will ease your transition greatly.

[teamfball]

As I’m still a python novice I would prefer to stay put. However, don’t let me stand in the way of progress. As long as ample time is given prior to the beginning of the season I’d do anything required to make the switch. I can say that with some comfort knowing you and others have always been accommodating to my questions.

If you decide to move to Python3 I wouldn’t waste any time supporting Python2.
That would only permit me to stay behind another year, and then I’ll never catch up.

[Noah0504]

I'm a little behind in responding, but I would very much welcome the move to Python 3. Oddly enough, the nflgame API is what prompted me to actually try and learn to code. I'm still quite a novice, but I just completed a course in Python 3, so my habits have been formed there.

[BurntSushi]

Well, this obviously didn't happen. I'd still like to get it done this season. However, if I do it during the season, then I'll make sure it stays Python 2 compatible so we don't have to deal with any major disruptions.

[samueljohn]

Just wanted to give you a shout that I'd really appreciate Python 3 support. And well, thanks for this lib - it is great for tinkering around with some statistics and machine learning libs (which I happen to have installed for Python 3.4).

[samueljohn]

I make a very WIP like pull request in #99. With the included json data that seems to work but not yet with getting the live update. //cc @wonbyte

[digitalmonarch]

Wanted to throw my hat in the ring too. I'm in favor of moving to Python 3. I'm new to python but not new to software development, and I would like to start by building some stuff that uses nflgame. I'm hesitant to learn on 2.7 given its age and some of the quirks that I've read about. It's hard to get motivated to learn on something that's deprecated :)

[BurntSushi]

@digitalmonarch Python 2.7 isn't deprecated. It's being supported until at least 2020, I think.

FWIW, most of the world is still using 2.7. The split is unfortunate, but it's not nearly as bad as you might think. The differences between 2.7 and 3.x are pretty small, but they are subtle enough to be a major pain. Otherwise, your day-to-day programming life isn't going to be impacted too much whether you're on 2 or 3.

[Croxy]

I'm curious, couldn't you fork your code and have two separate platforms? You could slowly start moving support over to Python 3... I don't know, I just found this repo today and am looking forward to playing with it!

[BurntSushi]

Maintaining two separate code bases is absolutely insane.

[Croxy]

Haha, fair enough. I was thinking you could EOL Python 2 support.

[BurntSushi]

Ah i see. Yes, that is more reasonable.

[sayeghr]

Perhaps it would be best if we were to start writing unit tests for nflgame until we get a large % of lines covered by our test code. Once we do that, we can start running the tests in Python 2 and in Python 3 and fixing the code base to be compatible with both versions.

[BurntSushi]

@sayeghr Yes, that would be nice. One might start with porting these tests from nfldb: https://github.com/BurntSushi/nfldb/blob/master/tests/test_query.py

[mlenzen]

FYI, I've ported nflgame to Python3: https://github.com/mlenzen/nflgame

It runs without error in 2.7.9 and 3.4.3 but I haven't compared outputs (yet) or written tests.

I also switched to requests instead of httplib2 because I was familiar with it and it handles content decoding and I didn't want to think about it.

[jsheedy]

I only started in with nflgame today, but wanted to give my vote for Python3. I have an existing tool which is dependent on the new async in Python3 that I'm hoping to try integrating nflgame into. I agree with the comments that maintaining support for 2 and 3 might be untenable, and a clean break in a major release might be an OK solution. I haven't tried the fork from @mlenzen yet, but I'll give it a shot. Thanks for all the great work!

[BurntSushi]

This really isn't a matter of voting. It's a matter of developing a test suite and being confident that the port works. Thank you @mlenzen for getting it started!

[mrconway]

You can go to Python 3 now. Make a break for it and force others to leave 2.X in their wake. Thanks for the awesome package!

[irish1986]

As a newcomer to coding and python in general, what are the realistic option if I desire to use Py3with NFLGame ?

Should I clone the project and run something like 2to3 or six on my own branch ?

How can I make my porting work beneficial for the greater good ? I don't really want to get into Py2 now and there are so many other cool Py3 package I wish to use.

[mlenzen]

@irish1986 I have a working python3 fork at https://github.com/mlenzen/nflgame

[derek-adair]

Whats he status on this? This is a must have for me.

[BurntSushi]

Nothing to report. I'm unlikely to work on this.

[derek-adair]

I've figured out a way around it for now... But it would simplify things for me in the long run.

What would it take to get https://github.com/mlenzen/nflgame merged in?