--try-software-version leads to syntax error in Infernal EasyConfig

[SethosII]

EasyBuild generates a wrong EasyConfig when I try to use --try-software-version on the Infernal easyconfig. See this example:

$ eb --buildpath=/dev/shm --optarch=march=corei7 --prefix=/mnt/easybuild/test --try-software-version=1.0 Infernal-1.1-goolf-1.4.10.eb -dr # same problem on other values for software version (e. g. `1.1.2`) as well
== temporary log file in case of crash /tmp/eb-lmDJNO/easybuild-LW76nC.log
ERROR: SyntaxError in easyconfig pyheader easyblock = 'ConfigureMake'

name = 'Infernal'
version = "1.0"

homepage = 'http://eddylab.org/infernal/'
description = """Infernal ("INFERence of RNA ALignment") is for searching DNA sequence databases
 for RNA structure and sequence similarities."""

toolchain = {'name': 'goolf', 'version': '1.4.10'}
toolchainopts = {'optarch': True, 'pic': True}

sources = [SOURCELOWER_TAR_GZ]
source_urls = ['http://eddylab.org/%(namelower)s']

checksums = []
}

moduleclass = 'bio'
: invalid syntax (<string>, line 17)

The original EasyConfig contains this:

##
# This file is an EasyBuild reciPY as per https://github.com/hpcugent/easybuild
#
# Copyright:: Copyright 2012-2014 Uni.Lu/LCSB, NTUA
# Authors::   Cedric Laczny <cedric.laczny@uni.lu>, Fotis Georgatos <fotis@cern.ch>
# License::   MIT/GPL
# $Id$
#
# This work implements a part of the HPCBIOS project and is a component of the policy:
# http://hpcbios.readthedocs.org/en/latest/HPCBIOS_2012-94.html
##

easyblock = 'ConfigureMake'

name = 'Infernal'
version = "1.1"

homepage = 'http://eddylab.org/infernal/'
description = """Infernal ("INFERence of RNA ALignment") is for searching DNA sequence databases
 for RNA structure and sequence similarities."""

toolchain = {'name': 'foss', 'version': '2016b'}
toolchainopts = {'optarch': True, 'pic': True}

sources = [SOURCELOWER_TAR_GZ]
source_urls = ['http://eddylab.org/%(namelower)s']

checksums = ['04bc8c9693cffba371b8559f023b6548']

sanity_check_paths = {
    'files': ['bin/cm%s' % x for x in ['align', 'build', 'calibrate', 'convert', 'emit',
                                       'fetch', 'press', 'scan', 'search', 'stat']],
    'dirs': []
}

moduleclass = 'bio'

So the sanity_check_paths is somehow partially deleted? For other EasyConfigs the --try-software-version works well, so I don't know what the cause is.

EasyBuild version 3.1.1

[boegel]

Thanks for the bug report @SethosII...

My first suspicion is that it has something to do with the %s being used in sanity_check_paths in the original easyconfig...

I'll take a closer look and try to come up with a fix.

[boegel]

@SethosII Problem should be fixed with changes made in pull request #2166.

The problem was not the %s used in sanity_check_paths, and it's a pretty nasty bug (which was not easy to fix).

When using --try-software-version, the checksums parameter is reset to an empty list (since it doesn't make sense to retain it, the checksums are hopefully not the same if the software version is different).

If there is a checksums = [...] line (possible spread across multiple lines), it needs to be replaced by checksums = [].
This is done by using a regex that matches the existing checksums = [...], and replaces it via re.sub.

The problem was that the regex that was used to find a match was way too greedy; rather than matching until the closing ] of checksums = [..., it pretty much matched until the last ] that could be found in the easyconfig file (as long as one was found at the end of a line with only whitespace after it).

The enhanced regex in #2166 makes sure this doesn't happen anymore, by using a non-greedy match to avoid consuming too much, and a lookahead assertion to try and make sure enough is matched. An additional test makes sure --try-software-version now works as it should, and it probably fixes similar issues with --try-amend.

The new implementation is probably still not 100% perfect, but it certainly a lot better than what was there.

Basically, --try-software-version was broken for some easyconfigs that included a checksums line if additional lines lower down included a ] at the end of a line, as is in the case in your example with dirs: [] (which is usually dirs: [],, but not always).

[boegel]

fixed in #2166 which was merged to be included in EasyBuild v3.1.2, thanks for the bug report @SethosII!