Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make FDTD_Solutions easyblock do the install with copytree instead of rpm rebuild #1307

Merged
69 changes: 25 additions & 44 deletions easybuild/easyblocks/f/fdtd_solutions.py
Expand Up @@ -33,64 +33,45 @@
from easybuild.easyblocks.generic.packedbinary import PackedBinary
from easybuild.easyblocks.generic.rpm import rebuild_rpm
from easybuild.tools.build_log import EasyBuildError
from easybuild.tools.run import run_cmd_qa
from easybuild.tools.filetools import copy_dir
from easybuild.tools.run import run_cmd


class EB_FDTD_underscore_Solutions(PackedBinary):
"""Support for building/installing FDTD Solutions."""

def build_step(self):
"""No build step for FDTD Solutions."""
# locate RPM and rebuild it to make it relocatable
rpms = glob.glob(os.path.join(self.cfg['start_dir'], 'rpm_install_files', 'FDTD-%s*.rpm' % self.version))
def extract_step(self):
"""
After unpacking the main tar file, we need to unpack the rpm
inside it.
"""
super(EB_FDTD_underscore_Solutions, self).extract_step()

rpms = glob.glob(os.path.join(self.src[0]['finalpath'], 'rpm_install_files', 'FDTD-%s*.rpm' % self.version))
if len(rpms) != 1:
raise EasyBuildError("Incorrect number of RPMs found, was expecting exactly one: %s", rpms)
rebuilt_dir = os.path.join(self.cfg['start_dir'], 'rebuilt')
rebuild_rpm(rpms[0], rebuilt_dir)
cmd = "rpm2cpio %s | cpio -idm " % rpms[0]
run_cmd(cmd, log_all=True, simple=True)

# replace original RPM with relocatable RPM
rebuilt_rpms = glob.glob(os.path.join(rebuilt_dir, '*', '*.rpm'))
if len(rebuilt_rpms) != 1:
raise EasyBuildError("Incorrect number of rebuilt RPMs found, was expecting exactly one: %s", rebuilt_rpms)
def make_installdir(self):
"""Override installdir creation"""
self.log.warning("Not pre-creating installation directory %s" % self.installdir)
self.cfg['dontcreateinstalldir'] = True
super(EB_FDTD_underscore_Solutions, self).make_installdir()

try:
os.rename(rpms[0], '%s.bk' % rpms[0])
shutil.copy2(rebuilt_rpms[0], rpms[0])
except OSError, err:
raise EasyBuildError("Failed to replace original RPM with rebuilt RPM: %s", err)
def build_step(self):
"""No build step for FDTD Solutions."""
pass

def install_step(self):
"""Install FDTD Solutions using install.sh script."""
cmd = "./install.sh"
acceptq = ''.join([
"If you accept the terms above, type ACCEPT, otherwise press <enter> ",
"to cancel the install [REJECT]:"
])
qa = {
acceptq: "ACCEPT",
"Please select an option from the list [1]:": '1',
"Please enter the install directory [/opt/lumerical/fdtd]:": self.installdir,
}
no_qa = [
]
std_qa = {
"Press <enter> to continue.*": ''
}
run_cmd_qa(cmd, qa, no_qa=no_qa, std_qa=std_qa, log_all=True, simple=True)

def make_module_req_guess(self):
"""Adjust values for PATH, LD_LIBRARY_PATH, etc."""
guesses = super(EB_FDTD_underscore_Solutions, self).make_module_req_guess()
guesses.update({
'PATH': ['opt/lumerical/fdtd/bin'],
'LD_LIBRARY_PATH': ['opt/lumerical/fdtd/lib'],
})
return guesses
"""Install FDTD Solutions using copy tree."""
fdtd_dir = os.path.join(self.cfg['start_dir'], 'opt', 'lumerical', 'fdtd')
akesandgren marked this conversation as resolved.
Show resolved Hide resolved
copy_dir(fdtd_dir, self.installdir, symlinks=self.cfg['keepsymlinks'])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akesandgren Installation fails now because the install directory is pre-created, and copy_dir doesn't like that.

You'll need to either call remove_dir(self.installdir)s first, or override the make_installdir method to make it a no-op (just log a message like "Not pre-creating installation directory %s" % self.installdir instead)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like that then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I overlooked that make_installdir is also responsible for removing an existing installation directory in case of a forced reinstall, so you should (also) set self.cfg['dontcreateinstalldir'] = True before calling the parent:

def make_installdir(self):
    """Override installdir creation"""
    self.log.warning("Not pre-creating installation directory %s" % self.installdir)
    self.cfg['dontcreateinstalldir'] = True
    super(EB_EB_FDTD_underscore_Solutions, self).make_installdir()


def sanity_check_step(self):
"""Custom sanity check for FDTD Solutions."""
custom_paths = {
'files': [],
'dirs': ['opt/lumerical/fdtd/bin', 'opt/lumerical/fdtd/lib'],
'files': ['bin/fdtd-solutions', 'lib/fdtd-engine-mpichp4_libFNP.so'],
'dirs': ['api'],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@akesandgren There no api installed for the existing FDTD_Solutions easyconfigs (8.6.2 and 8.11.337).

And for 8.6.2, there's no lib/fdtd-engine-mpichp4_libFNP.so either (not even close).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gah.... I only have 8.16.982 to look at...
Any suggestions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just checking for bin/fdtd-solutions and a non-empty lib dir?

That's still a little bit stricter than what was there (which was basically just checking for bin & lib)...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checking for a non-empty lib? what do you mean? I can check for an existing lib dir like it was done before, but how can i check for a non-empty one without checking for a specific file?

Or did you just mean an existing lib dir?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct me if I'm wrong, but 'dirs': ['lib'] already fails if the lib directory is empty.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does? Didn't really know that, but if that's the case it's good enough for me :-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In easyblock.py :

2140         path_keys_and_check = {
2141             # files must exist and not be a directory
2142             'files': ('file', lambda fp: os.path.exists(fp) and not os.path.isdir(fp)),
2143             # directories must exist and be non-empty
2144             'dirs': ("(non-empty) directory", lambda dp: os.path.isdir(dp) and os.listdir(dp)),
2145         }

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly, the dirs entries need to exist & be non-empty. Checking for an empty dir is kind of useless. ;)

}
super(EB_FDTD_underscore_Solutions, self).sanity_check_step(custom_paths=custom_paths)