Add a new build plugin 'shell' that runs arbitrary shell commands

[Magical-Chicken]

In order to support uncommon build systems and custom written build scripts it is useful to be able to run arbitrary commands to build and install, rather than just the commands hardcoded into the other snapcraft plugins.

This plugin accepts a list of commands and runs each of them with bash variables set to represent the source dir, build dir, and dest dir for the snapcraft part._

For example, here is a snapcraft file that compiles the video player mpv, which uses both a custom bootstrap script and the build system waf:
http://paste.ubuntu.com/23050131/

If there is anything you would like added to this plugin please let me know, thanks.

[snappy-m-o]

Can one of the admins verify this patch?

[snappy-m-o]

Can one of the admins verify this patch?

[sergiusens]

This looks a lot like #664

[elopio]

Nice contribution @Magical-Chicken, thanks.

As Sergio points out, this has some similar problems than the ones we are discussion in the other PR.

This one feels better, but I'm still thinking that if you need to run a weird sequence of commands to build your project because you are not using any one of the popular build tools or you are not following their best practices, you should include a local plugin in your repository.

It would be interesting to hear your opinion about #664, and @monsterjamp's opinion about this one.

[MonJamp]

I like this PR over my own, this one is a lot cleaner 😛

Although I feel like there should be an option to run a script during the pull stage since I feel that there's only 2 main reasons to use a bash script with snapcraft. The first one being to temporarily use a bash script instead of a build tool. The second reason being to retrieve dependencies/sources via unorthodox methods. I think that by default snapcraft should support scripts for those 2 use cases, any other way of using a bash script should require the maintainer to build a custom plugin.

[sergiusens]

El 17/08/16 a las 12:53, monsterjamp escribió:

I like this PR over my own, this one is a lot cleaner 😛

Although I feel like there should be an option to run a script during
the pull stage since I feel that there's only 2 main reasons to use a
bash script with snapcraft. The first one being to temporarily use a
bash script instead of a build tool. The second reason being to retrieve
dependencies/sources via unorthodox methods. I think that by default
snapcraft should support scripts for those 2 use cases, any other way of
using a bash script should require the maintainer to build a custom plugin.

Wouldn't writing a Makefile that does all this be the better solution?

[MonJamp]

@sergiusens Using a Makefile would make sense but I don't think snapcraft should dictate what build tools a programmer should use. If snapcraft can support usage of scripts out of the box or with little hassle the better it would be for smaller projects.

[sergiusens]

@monsterjamp but this plugin still forces you to write a script that is bound to snapcraft through the use of SRCDIR, BUILDDIR and PARTNAME

The problem with scripts is that they are so disparate that it will be hard to have a generic plugin to rule them all and people will still need to adapt their scripts.

Is there more than one case where this plugin or the other one will work out of the box?

The other disadvantage about this is that we lose the declarative nature of snapcraft and make it more of an imperative thing.

In any case, to move the conversation further. What existing projects would benefit from this?

[MonJamp]

I don't know of any projects that use scripts but I would imagine small projects and people trying out snapcraft for the first time would be more inclined to use scripts.

[Magical-Chicken]

Hi,

Thanks for taking a look at this. I hadn't seen the other branch that introduces this feature through a script. I think doing it this way may be a bit cleaner than an external script, because it still keeps the snapcraft.yaml file self-contained and does nor require introducing anything else to an already existing project in order to get it snapped as adding a new script to supplement the build system being used would. Also, this makes it easy to manage paths or build and install dir through the variables it sets. I also think that since the actual commands are contained in the snapcraft.yaml file rather than in an external script it is less confusing to have a build script that is bound to snapcraft, as nothing other than snapcraft would run the script.

I think that in most cases it would definitely be cleaner for a project upstream to have a make system that fits in cleanly with one of the normal build processes. This is definitely a little bit more hackish than a makefile that handles the build cleanly.

However, I think for the process of popularizing snaps and getting more upstream developers to start maintaining their own snaps this could be beneficial. If we go to a developers on projects that have unusual build systems/custom build scripts and tell them that snaps are great and they should try it out, but they are building their software wrong and they should fix it, it might be a harder sell. If we can say to an upstream developer that snapcraft can fit into your build process with very little work on your part, it should be much easier to sell them on it.

Maybe this plugin should have a warning that using a common build system and ensuring that a project builds cleanly with it is more reliable and saner than writing out build commands by hand or using a uncommon build system. I still think that it could be useful for bringing in upstream projects like mpv, that are well established enough that it may be difficult to convince them to change their build process.

I don't know of any other projects that could benefit from this off the top of my head, but I'm sure I could find some. I think the important thing with this is that snapcraft should be as easy to use and unintrusive as possible and not have to change anything about what upstream is doing. I definitely do agree that it would be much better for most projects to just have a valid Makefile instead of relying on a custom script though.

[Magical-Chicken]

Also, I like the idea of optionally having a separate series of commands/script during the pull stage, but it might be better to look into doing that in a separate branch. I guess that adding custom pull commands also has the same issue of being imperative while snapcraft is ideally fully declarative, but I think that having the option to pull from some weird type of version control or to extract a unusual type of archive may be useful in the same way that custom build commands are.

[sergiusens]

We have finally come to meet ways with the implementation of prepare, build and install keywords which parts can use which would provide most of the functionality proposed here.

Thanks.