Testing and feedback needed for newer version

[transistor1]

Please test the newer version of Shellista, based on @briarfox's ShellistaExt. I'd like to push this version to master, and push the modules to briarfox's repo.

It currently downloads the entire plugins repo when the program starts, rather than individual modules. I'd like to get feedback on how we can download individual modules instead (GitHub repos can only be downloaded as either single files or as a full repo-- you can't download directories), and on how the update system should be implemented.

Feedback on how code can be improved is also welcome. @jsbain I tried to incorporate your changes to the Git module for password caching, but haven't really tested it.

[jsbain]

Few comments. The precmd plugin probably should be setting line to the output of the plugin function. Also, we might want to pass Self to the plugin, to give it access to the class, for instance to add local storage (I'd like to attach my bash history object to Shell in the plugin, so future calls have access.

Git pull and push code should look identical, except for the command name.
I.e all the password caching stuff, and origin caching.

[jsbain]

Re versioning/updates... One thought is that the modules could be individual repos. The main repo would have a file pointing to the other ones. Update would simply be git pull.

Another thought, re updates would be to use pypi.. A little more overhead to setup the pypi entry. But this would also allow third party devs to create shellista plugins.

[transistor1]

Thanks for the quick response and feedback!

The precmd plugin probably should be setting line to the output of the plugin function.

You are right - added that, also precmd/postcmd now need to take `self`` as the first argument.

Also, we might want to pass Self to the plugin, to give it access to the class, for instance to add local storage

I agree with this also but do not have time to make that change this morning as it affects all plugins. @briarfox, any reason why this would not be a good idea?

Git pull and push code should look identical, except for the command name.

Will look, probably later tonight

Re versioning/updates... One thought is that the modules could be individual repos. The main repo would have a file pointing to the other ones. Update would simply be git pull.

I thought of individual repos, but would that be too hard to maintain if they are all in different repos? Anyone know how Git submodules work, and/or if there is any support for them in GitHub?

I am definitely not against the idea of individual repos - in fact I like it a lot. The benefit is that each dev maintains their own plugins. I would definitely like some other people weighing in before we make that change, just in case there is some other problem with that, that we're not thinking of.

Another thought, re updates would be to use pypi. A little more overhead to setup the pypi entry. But this would also allow third party devs to create shellista plugins.

Using setuptools was actually my first thought, but it didn't work in Pythonista, which is why I wrote the ModuleInstaller class. Doesn't pypi use setuptools for packaging (I know that easy_installer used it, but I thought pip just built on top of that framework)?

[transistor1]

Edited my comment - sorry my 1-year old hit the comment button for me! :)

[briarfox]

@transistor It seems to be working well. I think passing self to each module would be the correct way to go about things. I'll see if I can squeeze some time in at work and start converting over the shellistaExt modules.

I think individual repos might be the best way to go about it. Keep all the core modules in a single repo and allow other people to maintain their repos for other extensions. I agree with keeping a file that stores extension urls. Seems like the easiest way to go about it. I'd say if you want to use the ShellistaExt repo for the core, then we should reorganize it. However it might be better to move all core files into thier own repo.

As far as current extensions I'm thinking that the ssh module may need to be pulled. I re wrote it but full ssh access is seeming to cause a little bit of trouble. Commands like nano or python cause the client to hang and no data to be displayed. I'm thinking ssh might best be left to a third party app. The only command that I feel is vital to shellista from ssh tools is scp. I'm just about done with an scp module that will make it easier to get/put files from/to a server.

[transistor1]

@briarfox - excellent! Thanks for the feedback.

I think individual repos might be the best way to go about it. 

That works for me. @jsbain had a very good point about doing updates with git pull. This could make things very easy. Should Git be part of the initial download so we can manage plugin installation/updates with Git clone/pull?

Any ideas on the best way to architect the extension file plugins/update system? Do we use ConfigParser, XML, etc... do we maintain version info of the plugins ourselves, or maybe have the project store something like a version.txt file?

 I'd say if you want to use the ShellistaExt repo for the core, then we should reorganize it. However it might be better to move all core files into thier own repo.

Im open to either way. The dev version is currently pointing to a clone of your repo , which hosts my plugins. I had tried to push the branch to ShellistaExt repo in Shellista but got a "private" error. Not sure if that is specific to Dulwich or some other issue.

I'm thinking that the ssh module may need to be pulled.

That is a shame but I can certainly understand why. I can imagine things might get complex. Scp will be a great addition though. I wonder how hard it would be to make an sftp plugin.

[transistor1]

@briarfox in case you hadn't gotten around to making the self change, it is done here. I don't know what your local Git looks like -- let me know if you'd like me to create a pull request.

I just ran this on the repo in Linux to replace the main() args

find plugins | grep -v 'local-modules' | while read i; do if [[ "$i" == *.py ]]; then cat $i | sed "s/def main(/def main(self, /g" > .tmp; mv .tmp "$i" ; fi; done

[jsbain]

Regarding git and single files.... I know we've moved on, but found this while fixing gistcheck.
The github json api does allow single files.

See for example
https://api.github.com/repos/jsbain/shellista/contents
Then, if one were to follow url for the third filename, we get
https://api.github.com/repos/jsbain/shellista/contents/shellista.py?ref=master
Which has a content field, base64 encoded.

Doesn't pypi use setuptools

I'm definitely no expert, but I believe one can use distutils to package. Seuptools has some improvements, but I'm not exactly clear what the differences are. Not sure we'd want to, but in some ways pipista is a lot simpler than git, especially for the average usr who is not necessarily interested in a full git repo with history, etc, but just wants the latest version. The search capability is nice too.

[transistor1]

That is a good find -- but single files weren't the problem I meant (you can get to a single raw file on GitHub by going to the "raw" URL) What I meant was that you can't download a single directory from GitHub, which would make it hard to manage plugins from a single repo. Since we voted to use individual repos as plugins, I went ahead and coded up a quick and dirty implementation.

Latest dev-modular bootstraps (1) core modules, (2) Git, and (3) a "plugin plugin" which adds the command 'plugin' to search & install plugins via github. New repos can be added here. Anyone can edit it and it should be a simple merge.

In its current state it doesn't hot-plug plugins (you have to re-start Shellista to get it to recognize them) but I am working on it.

I have not been successful with pipista install, but maybe you have had a better experience. If you can setup an example for this we can certainly bootstrap pipista too.

[transistor1]

For new plugin repos - put your __init__.py and *_plugin.py in the root folder so the git clone works properly.

[briarfox]

@transistor1 I reworked the ssh plugin which includes scp. I added support for nano/edit to edit remote files. I still need to add a shortcut scheme for hosts but it's working. It still needs some reworking but I'd like some feedback if possible. I'll add it to the plugin list.

shellista-ssh https://github.com/briarfox/shellista-ssh

[transistor1]

Merged... did you have to fork the plugins repo to add the url? My hope was that devs could edit it directly using GitHub's built in text editor (click 'edit' in the upper right)

[transistor1]

Ssh plugin sounds great! Will definitely try it out when i get a chance.

[transistor1]

Can you also add your other non core modules like pipista?