New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract subversion support into its own package #313

Closed
bb-migration opened this Issue Dec 25, 2014 · 17 comments

Comments

Projects
None yet
1 participant
@bb-migration

bb-migration commented Dec 25, 2014

Originally reported by: jaraco (Bitbucket: jaraco, GitHub: jaraco)


Given the waning adoption of subversion in favor of DVCSs and the fact that setuptools itself no longer uses subversion, I propose we move the subversion support into a third-party package, something that can be more easily tested and supported by that community independent of setuptools. It will continue to use the same entry point for file finders.


@bb-migration

This comment has been minimized.

bb-migration commented Dec 25, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Removed svn support from setuptools. Ref #313.

@bb-migration

This comment has been minimized.

bb-migration commented Dec 25, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


The tests currently fail. That's because the egg_info command fails to run when the file_finders hook (as defined in the system-installed setuptools) isn't present in the local checkout. This issue will affect development environments as well. I expect running the bootstrap script will correct the issue.

@bb-migration

This comment has been minimized.

bb-migration commented Dec 25, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Invoking bootstrap prior to running egg_info works around the issue.

@bb-migration

This comment has been minimized.

bb-migration commented Dec 26, 2014

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


@philip_thiem @J1M

I want to draw your attention to this issue and the 9.0b1 release in downloads. If you know others who depend on the subversion support, please also mention this to them. Thanks.

@bb-migration

This comment has been minimized.

bb-migration commented Dec 26, 2014

Original comment by j1m (Bitbucket: j1m, GitHub: j1m):


:) Thanks.

I personally hate having VCS support as it usually just got in my way and led to mysterious behaviors.

@bb-migration

This comment has been minimized.

bb-migration commented Dec 29, 2014

Original comment by philip_thiem (Bitbucket: philip_thiem, GitHub: Unknown):


I can't think of anything that would require it out of the box unless someone is vendoring it. Pip has unit tests which test SVN support via way of setuptools, though. I think the module is probably the best way also. I would even further wonder if it shouldn't be trigger-able via a commandline option simply because if one needs setuptools to pull out SVN information, than one is also likely building it from SVN and doesn't need it all the time. The automagic file searching I will admit feels brittle. Correction: had, I do not know if it still does.

@bb-migration

This comment has been minimized.

bb-migration commented Jan 5, 2015

Original comment by maurits (Bitbucket: maurits, GitHub: maurits):


Just a note: there is already the setuptools_subversion package:
https://pypi.python.org/pypi/setuptools_subversion

I don't know which one would be better currently. I have used setuptools_subversion without problem.

But the last few years I just use a MANIFEST.in in most of my projects, so a third party add-on is not needed for me.

@bb-migration

This comment has been minimized.

bb-migration commented Jan 5, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Released as 10.0. Thanks for the feedback.

@bb-migration

This comment has been minimized.

bb-migration commented Jan 5, 2015

Original comment by maurits (Bitbucket: maurits, GitHub: maurits):


Released as 1.0 you mean. :-)

Thanks, good to have this.

@bb-migration

This comment has been minimized.

bb-migration commented Jan 5, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Well, I mean setuptools 10.0 loses subversion support and setuptools_svn 1.0 implements that same interface.

@bb-migration

This comment has been minimized.

bb-migration commented Jan 5, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


I'm surprised I didn't previously link to the setuptools_svn project.

@bb-migration

This comment has been minimized.

bb-migration commented Jan 5, 2015

Original comment by philip_thiem (Bitbucket: philip_thiem, GitHub: Unknown):


Last I checked it does not support legacy SVN at all. Which might be fine SVN 1.6 and earlier is not exactly recent. (I.E. requires the XML output interface).

@bb-migration

This comment has been minimized.

bb-migration commented May 28, 2015

Original comment by navytux (Bitbucket: navytux, GitHub: navytux):


This change not only removed support for svn, but also removed "setuptools.file_finders" entry-point group registration completly:

https://bitbucket.org/pypa/setuptools/commits/f191c8a1225bd58a5fb5aa9abb31b06dc710f0b9#Lsetup.pyF175

though setuptools.file_finders is still used by setuptools code for invoking custom file-finders:

https://bitbucket.org/pypa/setuptools/src/6f91e08f283f1822c16b4bafdee73cd30da56794/setuptools/command/sdist.py?at=default#cl-16
https://bitbucket.org/pypa/setuptools/src/6f91e08f283f1822c16b4bafdee73cd30da56794/setuptools/command/egg_info.py?at=default#cl-323

I personally use setuptools.file_finders to find files registered with git:

https://lab.nexedi.cn/kirr/wendelin.core/blob/84f0bd25/setup.py#L164
https://lab.nexedi.cn/kirr/wendelin.core/blob/84f0bd25/setup.py#L144

So could you please restore setuptools.file_finders entry-point group registration to empty?

Thanks beforehand,
Kirill

@bb-migration

This comment has been minimized.

bb-migration commented May 28, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


@navytux I'm afraid I don't understand. Setuptools indeed no longer supplies a file_finder for cvs and svn, but it still supports other projects implementing those finders. That's by design.

I tried loading your referenced setup.py file, but I didn't get a response. You'll need to provide an example inline.

Can you file a new ticket (referencing this one) and explain in code what you propose and what benefit you get from that?

@bb-migration

This comment has been minimized.

bb-migration commented May 29, 2015

Original comment by navytux (Bitbucket: navytux, GitHub: navytux):


@jaraco thanks for your feedback. It looks maybe it is me who misunderstood initially - yes other project can register such entry points. My misunderstanding came from the fact I register it at runtime in the same setup.py of the project who uses it, and I've found a way to workaround (or better say to fix my runtime registration):

https://lab.nexedi.cn/kirr/wendelin.core/commit/11d130d1

For completness, here is the whole code inline:

# register `func` as entry point `entryname` in `groupname` in distribution `distname` on the fly
def register_as_entrypoint(func, entryname, groupname, distname):
    from pkg_resources import working_set, EntryPoint
    dist = working_set.by_key[distname]

    # workaround for setuptools 9.0 dropping `setuptools.file_finders` entrypoint
    # group registration:
    #   https://bitbucket.org/pypa/setuptools/commits/f191c8a1225bd58a5fb5aa9abb31b06dc710f0b9#Lsetup.pyF175
    #   https://bitbucket.org/pypa/setuptools/issue/313
    # issue reported back:
    #   https://bitbucket.org/pypa/setuptools/issue/313#comment-18430008
    #
    # register group if it is not yet registered
    # else dist.get_entry_map(groupname) returns just {} not connected to entry map
    entry_map = dist.get_entry_map()
    if groupname not in entry_map:
        entry_map[groupname] = {}

    entrypoint = EntryPoint(entryname, func.__module__, attrs=(func.__name__,),
                                    extras=(), dist=dist)
    group = dist.get_entry_map(groupname)
    assert entryname not in group
    group[entryname] = entrypoint

def git_lsfiles(dirname):
    # FIXME dirname is currently ignored
    # XXX non-ascii names, etc
    for _ in runcmd(['git', 'ls-files']).splitlines():
        yield _.decode('utf8')  # XXX utf8 hardcoded
    # XXX recursive submodules
    for _ in runcmd(['git', 'submodule', 'foreach', '--quiet',  \
                r'git ls-files | sed "s|\(.*\)\$|$path/\1|g"']).splitlines():
        yield _.decode('utf8')  # XXX utf8 hardcoded

# if we are in git checkout - inject git_lsfiles to setuptools.file_finders
# entry-point on the fly.
#
# otherwise, for released sdist tarball we do not - that tarball already
# contains generated SOURCES.txt and sdist, if run from unpacked tarball, would
# reuse that information.
#
# (to setuptools - beause we have to use some distribution and we already
#  depend on setuptools)
if os.path.exists('.git'):  # FIXME won't work if we are checked out as e.g. submodule
    register_as_entrypoint(git_lsfiles, 'git', 'setuptools.file_finders', 'setuptools')

P.S. the site I've referenced was 500, but now it is online. Sorry.

@bb-migration

This comment has been minimized.

bb-migration commented May 30, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


@navytux, What you're doing there is unsupported. You're monkey-patching the entry points advertised by setuptools. What you should do instead is use the entry points mechanism to advertise that functionality from your own third-party package, and then require that in the setup_requires of your package. What I recommend instead of going to all of that trouble is to use setuptools_scm, which as you can see implements the file_finders entry point for git and mercurial repositories.

I hope that helps. In the future, please create separate tickets for separate or follow-on discussion.

@bb-migration

This comment has been minimized.

bb-migration commented Jun 1, 2015

Original comment by navytux (Bitbucket: navytux, GitHub: navytux):


@jaraco, thanks for pointing to setuptools_scm. Unfortunately it does not support git submodules.

Also, what I'm generally trying to do is not to monkey-patch setuptools, but provide an entry point from my package which is not-yet-installed for usage in my setup.py, without requiring external dependencies, so building instructions for my package are self-contained in my package. That's all. But currently setuptools does not support this, so I hook the entry-point to setuptools, which is always present. Only that - I do not monkey-patch it.

Thanks again for your feedback and for clarifying about setuptools.file_finders entry point.

Kirill

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment