Jython support

[fabioz]

Right now, it seems doing a pip install scandir will fail on Jython with:

error: Compiling extensions is not supported on Jython

As it seems like the library should work without extensions, I think there should be no issues in running it with Jython 2.7 (provided that it could be installed with pip).

Full output:

λ pip install scandir
Downloading/unpacking scandir
  Downloading scandir-1.7.tar.gz
  Running setup.py (path:C:\Users\fabio\AppData\Local\Temp\pip_build_fabio\scandir\setup.py) egg_info for package scandir

Installing collected packages: scandir
  Running setup.py install for scandir
    building '_scandir' extension
    error: Compiling extensions is not supported on Jython
    Complete output from command C:\bin\jython2.7.0\bin\jython.exe -c "import setuptools, tokenize;__file__='C:\\Users\\fabio\\AppData\\Local\\Temp\\pip_build_fabio\\scandir\\setup.py';exec(compile
(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\Users\fabio\AppData\Local\Temp\pip-inf_f4-record\install-record.txt --single-version
-externally-managed --compile:
    running install

running build

running build_py

creating build

creating build\lib.java1.8.0_172-2.7

copying scandir.py -> build\lib.java1.8.0_172-2.7

running build_ext

building '_scandir' extension

error: Compiling extensions is not supported on Jython

----------------------------------------
Cleaning up...
Command "C:\bin\jython2.7.0\bin\jython.exe -c "import setuptools, tokenize;__file__='C:\\Users\\fabio\\AppData\\Local\\Temp\\pip_build_fabio\\scandir\\setup.py';exec(compile(getattr(tokenize, 'open
', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\Users\fabio\AppData\Local\Temp\pip-inf_f4-record\install-record.txt --single-version-externally-managed --co
mpile" failed with error code 1 in C:\Users\fabio\AppData\Local\Temp\pip_build_fabio\scandir
Storing debug log for failure in C:\Users\fabio\.pip\pip.log

[benhoyt]

Hi @fabioz -- thanks for the report. I have never used Jython and don't have the bandwidth to begin now, but I'd be happy to review a pull request if you can make this work in Jython as well as CPython. Presumably some if/else logic in setup.py?

[RonnyPfannschmidt]

as far as i understood the scandir package cannot be supported on jython as it currently requires the c extension

[benhoyt]

@RonnyPfannschmidt: scandir.py itself works fine without the C module installed, and falls back to a (slower!) Python version. That's handled here, for example: https://github.com/benhoyt/scandir/blob/master/scandir.py#L28-L31

But setup.py always tries to compile the extension module: https://github.com/benhoyt/scandir/blob/master/setup.py#L35

[fabioz]

@benhoyt I don't personally use Jython either, but I do tests for the pydev debugger using pytest on Jython which now relies on scandir (and for the most part it works -- i.e.: https://travis-ci.org/fabioz/PyDev.Debugger/jobs/410478296), but other than keeping the basic support going, I'm also not vested in Jython and don't really intend to fix scandir to work on Jython -- using pytest==3.6 is good enough for me (just thought it'd be nice to report since pytest was working until last week and now is no longer working due to scandir).

[RonnyPfannschmidt]

if setup.py could opt out of building on missing compilers and /or unsupported pythons that would be fabulous

pytest-dev/pytest#3750

[frenzymadness]

I think that a fix here should be really simple and I wanted to prepare a pull request but I cannot reproduce the original issue. I am using virtual environment with Jython 2.7.1 and pip 18.0.

When I try to build scandir manually, it just shows UserWarning and nothing more.

$ python setup.py build
running build
running build_py
creating build
creating build/lib.java9.0.4-2.7
copying scandir.py -> build/lib.java9.0.4-2.7
running build_ext
building '_scandir' extension
/usr/share/jython/Lib/distutils/jythoncompiler.py:19: UserWarning: Compiling extensions is not supported on Jython
  warnings.warn('Compiling extensions is not supported on Jython')

Pip doesn't even show any warning. Tested with installation from the local folder and also from PyPI.

$ pip install .
Processing /home/lbalhar/Software/scandir
Building wheels for collected packages: scandir
  Running setup.py bdist_wheel for scandir ... done
  Stored in directory: /tmp/pip-ephem-wheel-cache-831ANe/wheels/01/e5/1a/e8d1d5535b7a2c83cd99a27eb0440b4ef743a6ca7b82199146
Successfully built scandir
Installing collected packages: scandir
Successfully installed scandir-1.7

$ pip install scandir
Collecting scandir
  Downloading https://files.pythonhosted.org/packages/13/bb/e541b74230bbf7a20a3949a2ee6631be299378a784f5445aa5d0047c192b/scandir-1.7.tar.gz
Building wheels for collected packages: scandir
  Running setup.py bdist_wheel for scandir ... done
  Stored in directory: /home/lbalhar/.cache/pip/wheels/4a/ca/d7/26c3620234732f2d5b3ca86d7ccb0f59a21bd7712bffbbedc2
Successfully built scandir
Installing collected packages: scandir
Successfully installed scandir-1.7

scandir works and uses the Python implementation in all three cases

$ jython
Jython 2.7.1 (, Feb 8 2018, 02:08:29) 
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java9.0.4
Type "help", "copyright", "credits" or "license" for more information.
>>> import scandir
>>> scandir._scandir is None
True

[RonnyPfannschmidt]

@frenzymadness at first glance this looks to me as if jython simly started to ignore c extensions - which is kind of horrific

there should be a clear error if a extension is-requested but cant be build

[fabioz]

The steps I'm running to get the failure on travis (https://github.com/fabioz/PyDev.Debugger/blob/master/.travis.yml) are:

export JYTHON_URL=http://search.maven.org/remotecontent?filepath=org/python/jython-installer/2.7.0/jython-installer-2.7.0.jar

wget $JYTHON_URL -O jython_installer.jar
java -jar jython_installer.jar -s -d $HOME/jython
export PATH=$HOME/jython:$HOME/jython/bin:$PATH
pip install scandir

It seems that the pip being used in that case is some older (or maybe custom) version (1.6.dev1), so, I tried upgrading pip too with pip install --upgrade pip (which got it to pip 18.0), but trying to do a pip install scandir still gives the same error.

So, maybe there is some fix on Jython 2.7.1 itself -- although I can't really find it... the Jython download page: http://www.jython.org/downloads.html only points to 2.7.0.

[frenzymadness]

@fabioz I am using Jython 2.7.1 installed from distribution RPM on Fedora 28.

[fabioz]

@frenzymadness thanks... I was able to find a newer version on:

http://repo1.maven.org/maven2/org/python/jython-installer/2.7.1/jython-installer-2.7.1.jar

and with that version I get the same behavior you get (pip install scandir works) -- not sure why the Jython download points to the older version though (will ask the Jython guys... probably just an oversight).

[benhoyt]

Thanks folks. I've merged the commit to make extension optional and will follow up with a pip release when I get a chance.

[benhoyt]

Done and on PyPI