support running EasyBuild without supplying an easyconfig file

[boegel]

(old internal ticket 318)

Currently, EasyBuild relies on having the user provide an easyconfig file which specifies all the build parameters: software name and version, toolkit, patch files, etc.

It might be worthwhile to implement a way to run EasyBuild without having to supply an easyconfig, i.e. by just specifying the name of the software to build, and optionally a version, toolkit (version), versionprefix, etc.

EasyBuild can then try and find either a matching easyconfig file given the desired specs, pick some reasonable values for parameters that were not supplied (e.g. build the latest known software version if no version was specified), or generate an easyconfig based on the available ones (e.g., just change the version/toolkit when reasonable, and attempt to build).

This way, we could simply supply template easyconfig files in the easybuild/easyconfigs directory, and add more specific easyconfigs if needed (e.g. if certain patch files are required to build with an ictce toolkit).

Implementing this would also allow to supply an easyconfig and override some of the parameters set in that file, e.g. software version, toolkit, etc.

[boegel]

Getting this implemented was started by @nudded, and is continued by @boegel, see https://github.com/boegel/easybuild/tree/318-change-toolkit .

[fgeorgatos]

Hi,

If I understand the topic correctly, I think a simple approach to this one, would be to have a param like --amend:
--amend 'mypython string1' --amend 'mypython string2' --amend 'mypython string3'

which would basically insert the following in place of the EOF of the automatically found .eb template:
[...multiline default eb template...]
string1
string2
string3

That approach would allow generic python code to be placed in there, allowing much extensibility. Example:

--amend "toolkit={'name':'dummy','version':'dummy'}"         # would force using the stock os compiler;
--amend "versionsuffix='"`date "+-%Y%m%d-%H%M"`"'"      # would drop a timestamp as suffix
--amend 'sourceURLs=['"'http://ftp.ntua.gr/pub/gnu/%s'"' % name.lower()]' # alternative source server

Basically all parameters can be appended in there, without having to generate specific code for each one.

If you agree with this approach, then this is related: allow getting .eb info from stdinput (combining the two concepts).

[boegel]

Great suggestion @fgeorgatos! I like it.

I would like to rewrite the easyconfig file in a pretty way though (already implemented), instead of just appending to it and overwriting parameters that way. It would yield very confusing easyconfig files.

Right now, I have added several command line options like --software-version=3.14, --toolkit=ictce,4.0.10, --versionsuffix=mysuff, etc.
I think my current implementation allows to easily support --amend as well, which should probably replace some of the options I've added now.

But maybe not all though. I would try and avoid letting the user supply Python code, e.g. dictionaries or lists, but would rather use a stricter syntax which feels more like shell, if you know what I mean.
E.g. --amend=toolkit-version:4.0.10 and --amend=sourceURLs:http://ftp.ntus.gr/pub/gnu/gcc .

Although that's less flexible, I don't think that's an issue. If a user wants full flexibility, they won't mind writing an easyconfig file imho.

[boegel]

Fixed by #227.