Add the packaging metadata to build the errbot snap

[come-maiz]

This is a package for the secure installation of apps that works in most Linux distributions.
Landing it upstream will enable builds for adventurous users.

[come-maiz]

You can find more information about snapcraft here: http://snapcraft.io/

If we upload the snap to the ubuntu store we will get a great way to deliver the software, and get transactional automatic updates. I did some initial testing for the snap [1]. It needs more testing, but it seems to me good enough to publish in the edge channel, which will make it really easy to get more people interested in finding and solving potential problems before the stable release.

If you have any doubts or comments, please let me know.

[1] https://gist.github.com/elopio/45032e7a6a0d23c117a62e44b14b4042

[sijis]

I think it is a super interesting and i'm for it. 👍

[zoni]

Snaps had already been on my radar so I'm glad to see more interest. 😄 There are some concerns that I think need to be addressed but I don't have time to write up a detailed response right now, so stay tuned, I will try to get back to this soon.

[come-maiz]

I will be here ^_^
Thanks for reviewing.

[zoni]

I think snappy is promising and the fact that other Linux distros are adding support for it is a positive sign.
It's a packaging format that lends itself well to distributing apps like errbot and I think would greatly improve the robustness of installations especially for users with less knowledge about systems administration/packaging/operations experience.

That said, I have one major point of concern (which might not be a problem, but I don't know snappy just well enough to judge yet).
Errbot's highly modular/plugin-based approach means users must be able to easily install plugins which have third-party package requirements.
I wonder how this will behave within the confines of the snappy sandbox. Will pip install work at all?

If we get the technical hurdles sorted out, we also definitely need to incorporate it into the documentation.
Not only would we need some instructions on how to install errbot via snaps (though that's some simple snap install ...) but
we would definitely need to write up some instructions on what to set BOT_DATA_DIRECTORY to (probably SNAP_USER_COMMON, but since it plugs home as well, maybe users would prefer something there instead?) as well.

[zoni]

We need to add network-bind as well, so that the webserver plugin can bind to the network.

[come-maiz]

Done.

[zoni]

Will this give us python 3? I've not used snapcraft in combination with python projects so consider me ignorant and feel free to point me to any relevant documentation. 😄 (speaking of, and this isn't meant negatively, but snappy documentation is still somewhat lacking at the moment).

Does this use the python interpreter on the host machine that's building the snap, or does it download/install it as part of the build process itself?

[come-maiz]

Yes, the default is python3.

We had a sprint two weeks ago and that was one of the big items, the docs.
We will be improving them over the following weeks; but your feedback will be highly appreciated. If something is not clear, please file a bug.
For the python plugin, you can use $ snapcraft help python.

During build, it uses the python version from the host. If it's not installed, it will install it before building the snap, because they are build-packages hardcoded inside the plugin. And the python interpreter is also a stage-package that comes from the plugin, so you will be able to use the snap even if you don't have python installed.
This should be documented, btw: https://bugs.launchpad.net/snapcraft/+bug/1640867

[zoni]

I think we need to add the build-packages to stage-packages as well as they're needed by pip when installing plugins with extra dependencies. We'll probably also have to add some additional common ones (libxslt-dev is another common one which comes to mind.

We should probably add mysql (libmysqlclient-dev) and psql (not sure which package provides it) libraries as well so people can install python packages needing to link to those, such as when using our sql storage plugin.

[come-maiz]

ok, I can add them to this PR.
I personally prefer to do that as a separate PR, to better link each change with a specific requirement. This change works for a basic bot, I find it nice to keep track of which additions are needed for more complex one.

But, as I said, just let me know and I'll make the change :)

[come-maiz]

Hi @zoni o/
Thanks for your review.

There are a few options to work with plugins. You can install pip as a stage package, and make it accessible to users as an app. Then they can call $ errbot.pip install, or you could hack something to make it like $ errbot.install_plugin. The modules will end up installed in the snap's writable path. This is good for pip, but we will have to think a little more about debs like libmysqlclient-dev.

Another option is to have an env var or config value for the location of your plugin modules. I haven't fully explored this one yet though. If we want the snap to access the user's PYTHONPATH for example, I think we will need an interface. It's an interesting thing to try, and I'm playing on my free time with an errbot juju charm based on this: https://github.com/1stvamp/juju-errbot I think this is the approach I'll try, and if the python modules are installed in $HOME, we don't need anything in addition to the home plug.

One that we have been experimenting is to have the plugins packaged as snaps, and share resources between the snaps with the content interface [1]. The downside is that of course it would require work from your ecosystem.

And this is my favorite approach: an errbot remote part [2]. You can provide a standalone errbot snap for simple bots and as a demo, or for the people who want to deal with independent plugins. But you can also upload your errbot part so others can easily make fully isolated complex bots. They will just have to add in their yaml "after: [errbot]", and they can pull their database dependencies, all their python modules, and use snap install my-bot to deploy. To make that work, we need better docs for sure.

Sorry for the long reply. Let me know what do you think.

pura vida.

[1] http://mhall119.com/2016/09/sharing-is-caring-with-snaps/
[2] http://snapcraft.io/docs/build-snaps/parts

[come-maiz]

ping @zoni, would you have some time to jump into a hangout or chat? Maybe it will be easier to answer your questions interactively there :)

[sijis]

@ElOpio, we have a gitter channel. I know @zoni is always available there.

[zoni]

Yep! The aforementioned gitter chat is a good way to reach me. I also do hang out on freenode (even in #snappy 😉) so you can hit me with a private query there, same nickname (zoni). European business hours are probably the best time to reach me, though evenings work too depending on how drained I am at the end of the day. 😄

And yes, I'd actually done a bit of experimenting with snaps for errbot based on this PR and have some thoughts on it, I just haven't had time to really write them out yet. But a realtime chat about it is probably more fruitful anyway!

[come-maiz]

Hey @zoni, I finally got the time to play with my bot:
https://github.com/elopio/snappy-m-o/blob/master/snap/snapcraft.yaml

If you land this PR, I can make it simpler by adding errbot to the remote parts.

I'm now trying to add nginx and letsencrypt in a juju charm. The snap works nicely by just installing it, but I think I don't want to bundle all the other fancy things in the snap.

[flexiondotorg]

I'd certainly be interested in seeing a snap for errbot.

[gbin]

What is the status about this ? Should it be in a separate errbotio/snap-errbot repository or it needs to be in the pypi package ?

[come-maiz]

Hey @zoni, remember me?
I finally got some time to write about the bot that we are using in snapcraft, and how we deploy it through the store:
http://elopio.net/blog/errbot-snap/

[gbin]

@ElOpio this is really cool !

[come-maiz]

Thanks @gbin!

I would like to add a section to the errbot docs about deploying with snaps. Like a much more focused version of my post. What do you think?

[gbin]

It would make a lot of sense.

[gbin]

Please go ahead with the doc version, I am closing this one.