(Rudimentary?) Dependency Management

[sublimator]

Ideas, or at least the desire for, Sublime plugin dependency management, have been around since 2008, but still there's no facilities, official or otherwise.

Though there are lessons to be learned from the various linux package managers, homebrew, npm, etc. Sublime has the distinction of requiring python modules to run in a single process.

Possibly this speaks to an unwillingness to compromise, desiring sophisticated mechanisms for multiple versions of an entity or nothing at all.

What's the minimum that can be useful?
What are the needs out there? Others will have other cases. I'll state one.

ZenCoding was previously implemented concurrently in Python/Javascript
The author grew tired of this burden and now only the Javascript branch will be developed
It was determined that a Javascript engine as Python module could be a solution for Python consumers

This binary module is bulky and will require downloading for every trivial change to KB scale source
This binary module needs compiling for the product of platforms and architectures

This binary module woud be useful as the raw basis for many plugins

A generic Javascript API for Sublime plugins could be built on top of the bridge
This API would require versioning for people to be able to rely on it

[sergeche]

My two cents about dependency management, especially binary ones like PyV8.

Some users prefer to keep their Packages folder in some synced place like Dropbox or flash drive to work with a single set of packages and preferences, for example, at work and at home.

It means that the same packages snapshot can be used on different platforms like OSX and Windows. So the binary loaded for the first platform won’t work on the second.

To solve this issue, I keep binaries in package subfolder named by current platform. For example, osx-p3 is for ST3 on OSX, win64 is for ST2 on Win64 and so on.

Also, I’m checking PyV8 availability for current platform when host plugin loads. So if there’s no such package, it is loaded first and after that the host plugin is actually initialises.

I think such use case should be considered if you’re planning to add dependency support.

On the other hand, this can be solved by changing layout of PyV8 package: it will download all available packages and load the one for current platform. But it will require too much data transfer. For example, current PyV8 package snapshot takes about 45MB (all platform packages zipped).

[wbond]

@sergeche Package Control already has a way to resolve platform/arch-specific packages, and will automatically install missing packages when ST starts. I've been trying to in as many places as possible to encourage people to only sync their User/ folder. If they do that, and install packages via Package Control and the Add Repository command, everything will stay up-to-date and all packages will have the proper platform/arch installed.

I guess until this point I have mostly been posted re-actively about it. Perhaps with the PC 2.0 launch I will make note of it in the release notes and add instructions to the website.

[wbond]

Also as a general note, we discussed the idea of dependency management for PC 2.0 at #291 (comment). I am trying to get all of the infrastructure in place for PC 2.0, so we might delay the details of dependency management to v2.1.

[FichteFoll]

@wbond, please notify me(/us) when you are taking steps in this direction. I'd love to contribute some of my ideas (but I'm not really having a "write burst" recently).

Very interesting issues regarding this: semver/semver#205 and its origin semver/semver#200.

[Suor]

By changing imports a bit it's possible to make python use different versions of same packages in same process (npm-like). Take a look at this: https://github.com/mitsuhiko/pluginbase

[FichteFoll]

I've talked with @wbond a bit about a solution similar to that and I have to say that this looks pretty useful. If we can have the dependencies specified in a separate file and apply that to all source of a package that is using a specific context handler (which should be possible), then this is looking good.

[Suor]

That would enable building bottom up based on reusable components. Now I catch myself copy-pasting code and copying files from one sublime plugin to another.

[wbond]

I'm pretty sure what you guys are taking about would be huge hacks, if even possible since Sublime Text for all of the importing.

I'm not currently planning on trying to do package-version-based dependency resolution. The best I think we could do is give a warning about incompatible packages. We could also deal with ST-version-based dependency resolution and platform-based resolution.

[FichteFoll]

Yes, platform and ST version are a must, and a warning when updating a package (and its dependencies) would cause another package to not satisfy the requirements. Since Semver is pretty awesome in that regard, we should be fine with that.

For version selectors I currently have this draft that I intend to implement into pysemver (which Package Control uses). That should be sufficient for all needs. (Please note that the current usage of build metadata will change.)

We also have to debate where packages should specify its dependencies. It should be part of the package itself, but that would require to download the entire package just for checking if dependency conflicts would arose when updating. Specifying this in the channel is not an option right now because a new pull request would be required for each dependency change.

[thecotne]

may be simply add some functions so package can manage his Dependencies

install_packages function with one parameter array of package names or git urls
enable_packages function with one parameter array of package names or git urls
disable_packages function with one parameter array of package names or git urls

installed_packages return list of installed packages
disabled_packages return list of disabled packages
enabled_packages return list of enabled packages

there are no version control in here because it adds complexity
we need to begin some where so implementing this functions will be good start

[FichteFoll]

Implementing functionality as a subset with a different interface instead of what you want to achieve in the long run creates a situation where we have to deprecate and some day remove the old "temporary" solution which results in more work overall.

For the moment, I think it would be best if PC just provided a shorthand command to install a certain package for automation purposes. Packages could just install the packages they need and that are not available manually. Also, it should be limited to only download one package at a time.

[wbond]

This is present in 3.0.0-beta