Build fails on Python 3.7.0b1

[mgedmin]

pip install pygame fails on Python 3.7.0b1 with the following error:

  building 'pygame.pypm' extension
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_REENTRANT -DENABLE_NEWBUF=1 -I/usr/X11R6/include -I/usr/include/SDL -I/usr/include -I/usr/include -I/home/mg/opt/python37/include/python3.7m -c src/pypm.c -o build/temp.linux-x86_64-3.7/src/pypm.o
  src/pypm.c: In function ‘__Pyx_ErrRestore’:
  src/pypm.c:4976:18: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
       if ((tstate->exc_type != NULL) & (tstate->exc_type != Py_None)) {
                    ^~~~~~~~
                    curexc_type
  src/pypm.c:4976:47: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
       if ((tstate->exc_type != NULL) & (tstate->exc_type != Py_None)) {
                                                 ^~~~~~~~
                                                 curexc_type
  src/pypm.c:4977:28: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
           tmp_type = tstate->exc_type;
                              ^~~~~~~~
                              curexc_type
  src/pypm.c:4978:29: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
           tmp_value = tstate->exc_value;
                               ^~~~~~~~~
                               curexc_value
  src/pypm.c:4979:26: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
           tmp_tb = tstate->exc_traceback;
                            ^~~~~~~~~~~~~
                            curexc_traceback
  src/pypm.c:4982:17: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
           tstate->exc_type = 0;
                   ^~~~~~~~
                   curexc_type
  src/pypm.c:4983:17: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
           tstate->exc_value = 0;
                   ^~~~~~~~~
                   curexc_value
  src/pypm.c:4984:17: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
           tstate->exc_traceback = 0;
                   ^~~~~~~~~~~~~
                   curexc_traceback
  At top level:
  src/pypm.c:2845:13: warning: ‘__pyx_doc_4pypm_5Input___dealloc__’ defined but not used [-Wunused-variable]
   static char __pyx_doc_4pypm_5Input___dealloc__[] = "Close midi device if still open when the instance is destroyed.";
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  src/pypm.c:2673:13: warning: ‘__pyx_doc_4pypm_5Input___init__’ defined but not used [-Wunused-variable]
   static char __pyx_doc_4pypm_5Input___init__[] = "Instantiate MIDI input stream object.";
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  src/pypm.c:1359:13: warning: ‘__pyx_doc_4pypm_6Output___dealloc__’ defined but not used [-Wunused-variable]
   static char __pyx_doc_4pypm_6Output___dealloc__[] = "Close midi device if still open when the instance is destroyed.";
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  src/pypm.c:1091:13: warning: ‘__pyx_doc_4pypm_6Output___init__’ defined but not used [-Wunused-variable]
   static char __pyx_doc_4pypm_6Output___init__[] = "Instantiate MIDI output stream object.";
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  error: command 'gcc' failed with exit status 1

[jamadden]

The source file pypm.c needs to be regenerated with Cython >= 0.27.3 for Python 3.7 support (it looks like it was last done with Cython 0.11.2 in 2011).

[takluyver]

There's a PR open to regenerate it (#345), but still not with a recent enough Cython.

I think it would be better, though, to remove pypm.c from the repo, and require Cython as part of the process to build from source. If anyone's got time to work on that, that would be great.

[jamadden]

#345 has been merged, so perhaps this can be verified and closed?

[takluyver]

Please do check, but from your earlier comment I think it's still not using a recent enough Cython (#345 used 0.26). So it may need to be updated again.

[jamadden]

Oh snap, I see where the comments mention 0.27.3 but no one actually does the update. I misremembered that as the update getting done. Apologies for the noise.

[illume]

I had to work around an issue with bdist_egg in #386. But now all tests pass for me locally. When we do another release, people who build from source will be able to use pip install on python37 again. Travis and apveyor don't support 3.7 yet of course, but we can start testing I guess. Python release notes aren't clear on which compiler they are doing the releases on for windows and such... hopefully it's not having to a recompile all the dependencies situation. Will have to try and see.

@jamadden Sorry I didn't update you here that I regenerated the pypm yesterday, but wanted to complete some testing with my midi gear first before pushing that.

[illume]

I couldn't find any 3.7 ppa yet, so here's a little script for xenial which worked-for-me.

# Sets up python3.7.0b1 and pygame for a vagrant ubuntu xenial install
# vagrant init ubuntu/xenial64
# vagrant up
# vagrant ssh

sudo apt-get -y update
sudo apt-get -y build-dep python3.5
sudo apt-get -y install python3-dev python3-setuptools python3-numpy python3-opengl     libav-tools libsdl-image1.2-dev libsdl-mixer1.2-dev libsdl-ttf2.0-dev libsmpeg-dev     libsdl1.2-dev libportmidi-dev libswscale-dev libavformat-dev libavcodec-dev     libtiff5-dev libx11-6 libx11-dev fluid-soundfont-gm timgm6mb-soundfont     xfonts-base xfonts-100dpi xfonts-75dpi xfonts-cyrillic fontconfig fonts-freefont-ttf
wget https://www.python.org/ftp/python/3.7.0/Python-3.7.0b1.tgz
tar xf Python-3.7.0b1.tgz 
cd Python-3.7.0b1/
./configure --prefix=/usr/local/ --enable-shared LDFLAGS="-Wl,--rpath=/usr/local/lib"
make clean
make -j4
sudo make install
cd ~
git clone git@github.com:pygame/pygame.git
cd pygame
python3.7 setup.py build
python3.7 setup.py install
sudo python3.7 setup.py install
SDL_VIDEODRIVER=dummy SDL_AUDIODRIVER=disk python3.7 -m pygame.tests --exclude opengl

[takluyver]

On Travis, you can add python: nightly to test with the development version of Python. It should also become possible at some point to test with the beta versions.

[mgedmin]

On Travis you can also explicitly ask for python: 3.7-dev if you don't want nightly to silently roll over into 3.8 at some unspecified point in the future.

[illume]

Oh yes. Thanks. A python: 3.7-dev build is on travis linux now.

[illume]

This should be fixed for pip users once we do a release soon. See #390 for status of that.