Skip to content

Release on PyPI #12

Closed
pfmoore opened this Issue Feb 18, 2014 · 16 comments

5 participants

@pfmoore
pfmoore commented Feb 18, 2014

It would be much easier to depend on scandir in my own code if it were registered on PyPI. Even better would be if you published wheels as well. Is there any reason not to release it?

@benhoyt
Owner
benhoyt commented Feb 18, 2014

The reason is that it's not really battle-tested yet, and the C code is far from complete (even if working at a basic level). I do hope to do some more work on it, but it may not be for a little while...

@poupas
poupas commented Feb 18, 2014

Ben, could you please elaborate a little on what is missing from the C code?

It might allow others to eventually contribute with some code :)

Thanks

@benhoyt
Owner
benhoyt commented Feb 18, 2014

Sure -- the main thing is the C version is currently kind of hacked in there and builds a list of results (name_stat tuples on Windows, name_type tuples of Linux) rather than returning a generator which yields results. The slowish Python version, on the other hand, yields results as it goes.

@tjguk
tjguk commented Feb 18, 2014
@benhoyt
Owner
benhoyt commented Feb 18, 2014

Thanks, Tim. Actually, it is quite a bit of use without that -- even when building a list, you still get all the speed advantages of scandir returning stat-like info directly from the OS, you just don't get the advantage of it being a generator.

@tjguk
tjguk commented Feb 18, 2014
@pfmoore
pfmoore commented Feb 18, 2014

The reason I got interested in this again is that I have a use case for a fast directory scan that returns stat data (a large directory tree comparison program) so maybe I'll be able to contribute some code. It's been a while since I did any Python/C API coding, should be fun :-)

@benhoyt
Owner
benhoyt commented Feb 18, 2014

Sure -- if it's better than what we've got, I'll definitely include it. Bonus points if it compiles for both Python 2.x and 3.x

@poupas
poupas commented Feb 18, 2014

Excellent :) It would also be awesome to clarify the whole unicode paths stuff.

I have a minor change in my fork to allow scandir to accept and return str (bytes) objects in Python 2, as it is needed to work around paths which are byte strings with unknown/invalid encoding (e.g. latin-1 paths mixed with utf-8 encoded paths...). This seems to be a Linux-only requirement. Windows works happily with unicode strings and OS X sanitizes the paths so that they are valid utf-8.

@tonyandrewmeyer

I don't think you need to have it "battle-tested" or the C code complete to make it available on PyPI - just clearly mark it as pre-release, so that people are aware that there may be issues, incompatible changes coming, and so on. It would really be a huge benefit for installation.

If you do really want more done first, would you be able to open issues for each thing you want done first and reference them here? Then I could submit PRs for each of them :)

@benhoyt
Owner
benhoyt commented May 30, 2014

This is now registered on PyPI: https://pypi.python.org/pypi/scandir -- I don't think I'll do anything like wheels till it's stabilized.

@benhoyt benhoyt closed this May 30, 2014
@tonyandrewmeyer

Is there anything that can be done to change your mind about this? (Like any sort of patch?).

You're already bumping version at various points - if you could just create a package whenever that happens and upload it to PyPI, that would mean that pip install would work (no need for wheels). Or if you'd prefer to host releases on GitHub, you can use the "urls" tab to tell PyPI where downloads are found (choose "do not extract" at the top, then add the URL manually, ensuring that the hash is present). That'll require "allow external" to be set, but would still make the package pip installable.

If the goal is inclusion in 3.5, then having the package available from PyPI will help, because that's where most people install things from, so you will better be able to show wide usage.

@benhoyt
Owner
benhoyt commented May 30, 2014

Yeah, that's a fair call. No, I'm not opposed to this -- I just don't want to learn wheels at this point. :-) So for pip install to work, I just need to do a "setup.py sdist" or similar and upload the zip to PyPI?

@benhoyt benhoyt reopened this May 30, 2014
@pfmoore
pfmoore commented May 30, 2014

Correct.

Just FYI, learning wheels isn't hard, just install the wheel project and do setup.py bdist_wheel. Then upload the .whl file. The only issue is that like any binary format, for a C extension you need to provide separate builds for each Python version and architecture. (win32/win64, wheels are mainly useful on Windows at this point).

@benhoyt
Owner
benhoyt commented May 30, 2014

Thanks. Yep, happy to upload a source zip to PyPI (I'll do that in the next couple of days). However, I'm going to leave the wheels / binary builds at this point.

@benhoyt
Owner
benhoyt commented May 31, 2014

Uploaded v0.4 files (.zip, .tar.gz, and Windows 64-bit installer just for fun) to PyPI.

@benhoyt benhoyt closed this May 31, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.