lua-luaposix: unable to install from offline system

[KineticTheory]

The recipe for lua-luaposix attempts to download http://github.com/luaposix/luaposix/archive/release-v33.4.0.zip (see luaposix-33.4.0-1.rockspec in the source tree) and fails when it cannot access the internet (e.g.: air gap system).

Possible fix: Spack might need to download this zip file as a dependency (I'm not sure how to make this work) and patch the rockspec file to point to a location on the local file system instead of a url.

I see the same problem with lua-luafilesystem. In fact, even from machines that have internet access I can't access the required files for luafilesystem because it uses a git URL (git://github.com/keplerproject/luafilesystem) and the git protocol is blocked by our corporate firewall.

My current work around is to

• Download the files with git clone git@github.com:keplerproject/luafilesystem
• Edit luafilesystem-1.6.3-1.rockspec that is located in the stage directory so that it the url entry points to the local file system location from the above step.
• Run spack install again.

[citibeth]

This is a known problem... lua-luaposix depends on the luarocks program to install --- which installs luaposix and all of its dependencies. This is problematic in many ways:

1. It writes into the lua install tree after lua has been installed, thereby violating a bunch of core Spack assumptions.

2. It won't work offline.

The solution is to stop using luarocks. That would be more work, as someone would have to make Spack packages for all the dependencies of whatever needs to be installed.

[We have the same issue in Python-land and pip. Luckily, most or all of what we need has now been built as Spack packages. Someone needs to do the same for Lua.]

[amklinv]

I am also interested in the resolution of this luarocks issue.

[KineticTheory]

@amklinv If I remember correctly (from more than a year ago), you can work around this deficiency by running spack in a mode that saves the stage directories. When the build fails, you can modify the rockspec file in the source directory to point to your local spack mirror instead of github.com and then run the install command again. It is painful, but this technique allowed me to install lmod on a stand-alone system.

[LudovicEnault]

Hi there,
I find this thread useful but still had to find the exact stuff to do.
So, for other people (and me when I would have forgotten), Here is what I have done:
(doing for lua but same spirit applies for lua-filesystem)

# spack install --keep-prefix --dirty lmod
[...]
==> Installing lua-luaposix
==> Searching for binary cache of lua-luaposix
==> Finding buildcaches in /home/shared/products/spack/spack-mirror-2019-08-23/build_cache
==> Finding buildcaches in /shared/products/spack/spack-mirror-2019-08-23/build_cache
==> No binary for lua-luaposix found: installing from source
==> Fetching file:///shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/lua-luaposix-33.4.0.tar.gz
curl: (37) Couldn't open file /shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/lua-luaposix-33.4.0.tar.gz
==> Fetching from file:///shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/lua-luaposix-33.4.0.tar.gz failed.
==> Fetching file:///home/shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/lua-luaposix-33.4.0.tar.gz
######################################################################## 100.0%
==> Staging archive: /tmp/spack-stage/enaultl/lua-luaposix-33.4.0-dc6hcemcgliozvp7tqvl5t4qb37icorc/release-v33.4.0.tar.gz
==> Created stage in /tmp/spack-stage/enaultl/lua-luaposix-33.4.0-dc6hcemcgliozvp7tqvl5t4qb37icorc
==> No patches needed for lua-luaposix
==> Building lua-luaposix [Package]
==> Executing phase: 'install'
==> Error: ProcessError: Command exited with status 1:
    '/home/shared/products/spack/opt/spack/linux-rhel7-ppc64le/gcc-4.8.5/lua-5.3.5-2e2i3yywf77k724hgd2554yiz5alawrp/bin/luarocks' '--tree=/home/shared/products/spack/opt/spack/linux-rhel7-ppc64le/gcc-4.8.5/lua-luaposix-33.4.0-dc6hcemcgliozvp7tqvl5t4qb37icorc' 'install' 'luaposix-33.4.0-1.rockspec'

1 error found in build log:
     3
  >> 4    Error: Error fetching file: Failed downloading http://github.com/luap
          osix/luaposix/archive/release-v33.4.0.zip - release-v33.4.0.zip
     5    Using luaposix-33.4.0-1.rockspec... switching to 'build' mode

See build log for details:
  /tmp/spack-stage/enaultl/lua-luaposix-33.4.0-dc6hcemcgliozvp7tqvl5t4qb37icorc/spack-build-out.txt

# (( get the file manually))
# cd ${SPACK_MIRROR}/lua-luaposix/
# tar zxf lua-luaposix-33.4.0.tar.gz
# mv lua-luaposix-33.4.0.tar.gz{,.old}
# vi luaposix-release-v33.4.0/luaposix-33.4.0-1.rockspec
-- modify url,
url = "file:///home/shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/release-v33.4.0.zip"
# tar zcf lua-luaposix-33.4.0.tar.gz luaposix-release-v33.4.0
# ls -lrt /home/shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/*.{zip,tar.gz}
 -rw-r--r-- 1 enaultl ibm 777226 Aug 26 09:53 /home/shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/release-v33.4.0.zip
-rw-rw-r-- 1 enaultl ibm 643600 Aug 26 09:57 /home/shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/lua-luaposix-33.4.0.tar.gz

# spack install --keep-prefix --dirty --no-checksum lmod

==> Continuing from partial install of lua-luaposix
==> Installing lua-luaposix
==> Searching for binary cache of lua-luaposix
==> Finding buildcaches in /home/shared/products/spack/spack-mirror-2019-08-23/build_cache
==> Finding buildcaches in /shared/products/spack/spack-mirror-2019-08-23/build_cache
==> No binary for lua-luaposix found: installing from source
==> Fetching from /home/shared/products/spack/var/spack/cache/lua-luaposix/lua-luaposix-33.4.0.tar.gz failed.
==> Fetching file:///shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/lua-luaposix-33.4.0.tar.gz
curl: (37) Couldn't open file /shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/lua-luaposix-33.4.0.tar.gz
==> Fetching from file:///shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/lua-luaposix-33.4.0.tar.gz failed.
==> Fetching file:///home/shared/products/spack/spack-mirror-2019-08-23/lua-luaposix/lua-luaposix-33.4.0.tar.gz
######################################################################## 100.0%
==> Staging archive: /tmp/spack-stage/enaultl/lua-luaposix-33.4.0-dc6hcemcgliozvp7tqvl5t4qb37icorc/release-v33.4.0.tar.gz
==> Created stage in /tmp/spack-stage/enaultl/lua-luaposix-33.4.0-dc6hcemcgliozvp7tqvl5t4qb37icorc
==> No patches needed for lua-luaposix
==> Building lua-luaposix [Package]
==> Executing phase: 'install'
==> Successfully installed lua-luaposix
  Fetch: 0.05s.  Build: 10.77s.  Total: 10.82s.
[+] /home/shared/products/spack/opt/spack/linux-rhel7-ppc64le/gcc-4.8.5/lua-luaposix-33.4.0-dc6hcemcgliozvp7tqvl5t4