Unable to specify version for custom plugin.

[mrburrito]

I'm developing a custom plugin and can't get pybuilder to pull new versions when they're available. How do I specify min and/or locked version for a plugin pulled from a pypi repository?

[esc]

IIRC there is no mechanism built in to do this (yet). @mriehl to confirm

[arcivanov]

@mrburrito

I'm not sure I understand what you're asking, but here are some thing that might help :)

https://github.com/pybuilder/pybuilder/blob/master/src/main/python/pybuilder/core.py#L210
https://github.com/pybuilder/pybuilder/blob/master/src/main/python/pybuilder/pluginloader.py#L156
https://github.com/pybuilder/pybuilder/blob/master/src/unittest/python/pluginloader_tests.py#L214
https://github.com/pybuilder/pybuilder/blob/master/src/unittest/python/pluginloader_tests.py#L226

[mrburrito]

@arcivanov I'd like pybuilder to validate the plugin version each time it runs to determine if a newer version of the plugin is available (according to the version spec) and, if so, download the updated version. For -dev versions, I want it to check timestamps and/or dev[N] numbers in the repo to make sure it has the latest version. Basically, emulate maven's behavior for SNAPSHOT and release dependencies.

It doesn't seem to do any of that and I've resorted to running an rm ${venv}/lib/python2.7/site-packages/my_plugin* each time I update my plugin. This is not a scalable solution once this gets pushed out to our CI server and the rest of my dev team.

Am I missing something in my plugin definition that configures the version so pybuilder can check it? I saw the code in pluginloader._check_plugin_version, but I don't think that's getting hit. I can change my version spec in the use_plugin call to a later version than what's installed and everything continues to run without downloading the new plugin.

[arcivanov]

@mrburrito

https://github.com/pybuilder/pybuilder/blob/master/src/unittest/python/pluginloader_tests.py#L214

If you use use_plugin with version specification and set version according to https://www.python.org/dev/peps/pep-0440/#handling-of-pre-releases then pip will both pull and update prereleases for you. You can also use the VCS URL in which case plugin will always be reinstalled.

So either

use_plugin("pypi:your-plugin", "~=1.2.3.dev0")

or

use_plugin("vcs:http://github.com/...", plugin_module_name="plugin.module.name")

Please let me know if this works for you. You might have to play with the version spec a little to get exactly what you want, so please do read through PEP-0440 carefully.

[mrburrito]

None of that is working.

The plugin I'm working on is currently at version 0.3.1, with a dev version of 0.4.0.dev. When I publish/upload via the pybuilder distutils plugin, it appended the date to the end of the dev version rather than a number, so I deployed 0.4.0.dev20160310151716 to our internal PyPi repository (Artifactory).

My original include was use_plugin('pypi:my-plugin', '>=0.3.0') which, correctly, did not grab the dev dependency.

I updated it to use_plugin('pypi:my-plugin', '~=0.4.0.dev') and '~=0.4.0.dev0', neither of which worked.

I published my dev version as a release, 0.4.0 and tried again, using ~=0.3, >=0.3 and >=0.3.0 as version specs and pybuilder did not pull either my new 0.4.0 release or the 0.4.0.dev artifact from the repository.

I'm still faced with a situation where the only way I can get it to pull an updated version is by removing the cached plugin from the virtual environment.

Any ideas?

[mrburrito]

Oh, the commands I'm running to attempt to get pybuilder to download the plugin are:

• pyb -t
• pyb my_plugin_task
• pyb install_dependencies

[esc]

IIRC the "check-for-latest-version-of-plugin-and-install" never actually worked, but that was some weeks ago.

[arcivanov]

Um, weird, I'll debug and get back to you.

[esc]

I just looked at the code and judging from this:

https://github.com/pybuilder/pybuilder/blob/master/src/main/python/pybuilder/pluginloader.py#L120

it would appear that it will upgrade if a version identifier is present.

[esc]

Yeah, I can confirm that neither specifying the plugin with version or without will automatically upgrade it with a pyb install_dependencies.

[esc]

So, the ThirdPartyPluginLoader https://github.com/pybuilder/pybuilder/blob/master/src/main/python/pybuilder/pluginloader.py#L64 triggers if the plugin has already been downloaded and then simply continues without checking for updates.

[esc]

One would probably have to refactor the way in which the plugins are loaded.

You could try chaning the order in:

https://github.com/pybuilder/pybuilder/blob/master/src/main/python/pybuilder/reactor.py#L63

And put the downloading_thirdparty_plugin_loader first.

[esc]

I'll try that now locally and get back.

[esc]

So, reversing the order of the plugin loaders AND specifying the version will update the plugin.

[esc]

Not sure what kind of stuff breaks when doing this, all unittests pass though.