Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

spack install lua-luafilesystem -> Failed cloning git repository. #7848

Closed
bengisuelis opened this issue Apr 20, 2018 · 10 comments · Fixed by #12703
Closed

spack install lua-luafilesystem -> Failed cloning git repository. #7848

bengisuelis opened this issue Apr 20, 2018 · 10 comments · Fixed by #12703
Labels
fetching

Comments

@bengisuelis
Copy link

I am trying to install lua-luafilesystem by spack and apparently there is something wrong with the repo.

Actual Result

==> libsigsegv is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libsigsegv-2.11-fypapcprssrj3nstp6njprskeyynsgaz
==> m4 is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/m4-1.4.18-r5envx3kqctwwflhd4qax4ahqtt6x43a
==> pkgconf is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkgconf-1.4.0-c5lf2hilcinnqpf3nojkif3eauay2gko
==> ncurses is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/ncurses-6.0-nxkp7kslgelgkl5orm7la6fir7b7r547
==> readline is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/readline-7.0-gys53zdfur4qv5y3e5q5s734sbziuv5q
==> gdbm is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gdbm-1.14.1-jbpld4yy6hucm7vryhrygbl7nlxei33f
==> perl is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/perl-5.24.1-ojuball5xfke3t6hfg4gaf6t3cgk7cjg
==> autoconf is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/autoconf-2.69-cpyyi6iwy7f34o4s2rardmexnmhgzujo
==> automake is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/automake-1.15.1-35ma3dfdd5ycgimfvvozubbix7frcyo4
==> zlib is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb
==> openssl is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openssl-1.0.2n-w44ec2otpz6giuwrgmzjupluxibx3kuy
==> cmake is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.11.0-dbtdkycs2lywrbbepgg3cns4ybaiqvju
==> xz is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/xz-5.2.3-htnq7wqdrqtof6uxqicdj3f7oe3xz6pw
==> libssh2 is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libssh2-1.8.0-zq2xqfvhovj2w2dztrvjizfyckddpmhh
==> curl is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/curl-7.56.0-5awsurahb7yrlzbaugtnjzhzrz3k3aoc
==> libbsd is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libbsd-0.8.6-f4qkkwmagbwroaoljw3irrpswb3uteqy
==> expat is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/expat-2.2.2-ayzd2ohzkowikiiynq7ti6vfehhidxcx
==> bzip2 is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/bzip2-1.0.6-ufczdvsqt6edesm36xiucyry7myhj7e7
==> libxml2 is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libxml2-2.9.4-sxk64lvcxhqjflzesnf3ye4wakovwi45
==> tar is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/tar-1.29-22xotf4z4ufezcxcsq3v5ptqrf4uf2iu
==> gettext is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gettext-0.19.8.1-6oy6fuc3no4tc6nmwwahrgacmci7tcms
==> libiconv is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libiconv-1.15-u2x3umvoehr7zpwwwggevpownvki6ak2
==> libtool is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libtool-2.4.6-o2pfwjf44353ajgr42xqtvzyvqsazkgu
==> pcre is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pcre-8.41-7zjt36gj3psovsvxhq5s76tq7pb7arkh
==> git is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/git-2.15.1-uycn44empprxx244qekmlw432652supe
==> lua is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/lua-5.3.4-6ayqzzib727k3oocmd63ztuikmis5ny2
==> Installing lua-luafilesystem
==> Using cached archive: /u/home/bengisu/spack/var/spack/cache/lua-luafilesystem/lua-luafilesystem-1_6_3.tar.gz
==> Staging archive: /u/home/bengisu/spack/var/spack/stage/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4/v1_6_3.tar.gz
==> Created stage in /u/home/bengisu/spack/var/spack/stage/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4
==> No patches needed for lua-luafilesystem
==> Building lua-luafilesystem [Package]
==> Executing phase: 'install'
==> Error: ProcessError: Command exited with status 1:
    '/u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/lua-5.3.4-6ayqzzib727k3oocmd63ztuikmis5ny2/bin/luarocks' '--tree=/u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4' 'install' '/u/home/bengisu/spack/var/spack/stage/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4/luafilesystem-1_6_3/rockspecs/luafilesystem-1.6.3-1.rockspec'

1 error found in build log:
     4     Cloning into 'luafilesystem'...
     5     fatal: unable to connect to github.com:
     6     github.com[0: 192.30.253.113]: errno=Connection timed out
     7     github.com[1: 192.30.253.112]: errno=Connection timed out
     8     
     9     
  >> 10    Error: Failed cloning git repository.

See build log for details:
  /u/home/bengisu/spack/var/spack/stage/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4/luafilesystem-1_6_3/spack-build.out

Steps to reproduce the issue

$ spack install lua-luafilesystem
...

Information on your system

  1. Linux i10se4 4.4.0-119-generic ncurses: update package #143-Ubuntu SMP Mon Apr 2 16:08:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
    DISTRIB_ID=Ubuntu
    DISTRIB_RELEASE=16.04
    DISTRIB_CODENAME=xenial
    DISTRIB_DESCRIPTION="Ubuntu 16.04.4 LTS"

  2. var/spack/repos/builtin/packages/lua-luafilesystem/package.py:

from spack import *


class LuaLuafilesystem(Package):
    """LuaFileSystem is a Lua library developed to complement the set of
    functions related to file systems offered by the standard Lua distribution.

    LuaFileSystem offers a portable way to access the underlying directory
    structure and file attributes.

    LuaFileSystem is free software and uses the same license as Lua 5.1
    """

    homepage = 'http://keplerproject.github.io/luafilesystem'
    url = 'https://github.com/keplerproject/luafilesystem/archive/v1_6_3.tar.gz'
    version('1_6_3', 'bed11874cfded8b4beed7dd054127b24')    

    # The version constraint here comes from this post:
    #
    # https://www.perforce.com/blog/git-beyond-basics-using-shallow-clones
    #
    # where it is claimed that full shallow clone support was added @1.9
    depends_on('git@1.9.0:', type='build')
    extends('lua')

    def install(self, spec, prefix):
        rockspec_fmt = join_path(self.stage.path,
                                 'luafilesystem-{version.underscored}',
                                 'rockspecs',
                                 'luafilesystem-{version.dotted}-1.rockspec')
        luarocks('--tree=' + prefix, 'install',
                 rockspec_fmt.format(version=self.spec.version))

After reading #2682 I also tried changing url to
url = 'https://github.com/keplerproject/luafilesystem/archive/v_1_6_3.tar.gz' but the result was the same error.

I also tried cloning both repos myself through terminal by:
git clone https://github.com/keplerproject/luafilesystem/archive/v1_6_3.tar.gz
which also gave me :

Cloning into 'v1_6_3.tar.gz'...
remote: Not Found
fatal: repository 'https://github.com/keplerproject/luafilesystem/archive/v1_6_3.tar.gz/' not found
@KineticTheory
Copy link
Contributor

KineticTheory commented Apr 20, 2018 •
edited

This may be an issue with LuaLuafilesystem or the lua-filesystem/package.py in spack.

Note:

  • The tar.gz file exists and can be downloaded.
% wget https://github.com/keplerproject/luafilesystem/archive/v1_6_3.tar.gz
100%[==============================================================>] 
2018-04-20 09:45:27 (600 KB/s) - ‘luafilesystem-1_6_3.tar.gz’ saved [30389/30389]
  • The project can be checked out from github.com:
%  git clone https://github.com/keplerproject/luafilesystem
Cloning into 'luafilesystem'...
remote: Counting objects: 1643, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 1643 (delta 0), reused 2 (delta 0), pack-reused 1638
Receiving objects: 100% (1643/1643), 328.32 KiB | 1.29 MiB/s, done.
Resolving deltas: 100% (742/742), done.

@adamjstewart
Copy link
Member

Can you try again? I was just able to fetch it:

$ spack fetch lua-luafilesystem
==> Fetching https://github.com/keplerproject/luafilesystem/archive/v1_6_3.tar.gz
######################################################################## 100.0%

@bengisuelis
Copy link
Author

Yes, I figured out luafilesystem is cloned and even extracted, although the error message tells otherwise. After spack clean and removing the spack/var/spack/cache/lua-luafilesystem and then doing spack install lua-luafilesystem I got this :

==> libsigsegv is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libsigsegv-2.11-fypapcprssrj3nstp6njprskeyynsgaz
==> m4 is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/m4-1.4.18-r5envx3kqctwwflhd4qax4ahqtt6x43a
==> pkgconf is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkgconf-1.4.0-c5lf2hilcinnqpf3nojkif3eauay2gko
==> ncurses is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/ncurses-6.0-nxkp7kslgelgkl5orm7la6fir7b7r547
==> readline is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/readline-7.0-gys53zdfur4qv5y3e5q5s734sbziuv5q
==> gdbm is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gdbm-1.14.1-jbpld4yy6hucm7vryhrygbl7nlxei33f
==> perl is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/perl-5.24.1-ojuball5xfke3t6hfg4gaf6t3cgk7cjg
==> autoconf is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/autoconf-2.69-cpyyi6iwy7f34o4s2rardmexnmhgzujo
==> automake is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/automake-1.15.1-35ma3dfdd5ycgimfvvozubbix7frcyo4
==> zlib is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb
==> openssl is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openssl-1.0.2n-w44ec2otpz6giuwrgmzjupluxibx3kuy
==> cmake is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/cmake-3.11.0-dbtdkycs2lywrbbepgg3cns4ybaiqvju
==> xz is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/xz-5.2.3-htnq7wqdrqtof6uxqicdj3f7oe3xz6pw
==> libssh2 is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libssh2-1.8.0-zq2xqfvhovj2w2dztrvjizfyckddpmhh
==> curl is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/curl-7.56.0-5awsurahb7yrlzbaugtnjzhzrz3k3aoc
==> libbsd is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libbsd-0.8.6-f4qkkwmagbwroaoljw3irrpswb3uteqy
==> expat is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/expat-2.2.2-ayzd2ohzkowikiiynq7ti6vfehhidxcx
==> bzip2 is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/bzip2-1.0.6-ufczdvsqt6edesm36xiucyry7myhj7e7
==> libxml2 is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libxml2-2.9.4-sxk64lvcxhqjflzesnf3ye4wakovwi45
==> tar is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/tar-1.29-22xotf4z4ufezcxcsq3v5ptqrf4uf2iu
==> gettext is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/gettext-0.19.8.1-6oy6fuc3no4tc6nmwwahrgacmci7tcms
==> libiconv is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libiconv-1.15-u2x3umvoehr7zpwwwggevpownvki6ak2
==> libtool is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/libtool-2.4.6-o2pfwjf44353ajgr42xqtvzyvqsazkgu
==> pcre is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pcre-8.41-7zjt36gj3psovsvxhq5s76tq7pb7arkh
==> git is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/git-2.15.1-uycn44empprxx244qekmlw432652supe
==> lua is already installed in /u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/lua-5.3.4-6ayqzzib727k3oocmd63ztuikmis5ny2
==> Installing lua-luafilesystem
==> Fetching https://github.com/keplerproject/luafilesystem/archive/v1_6_3.tar.gz
######################################################################## 100.0%
==> Staging archive: /u/home/bengisu/spack/var/spack/stage/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4/v1_6_3.tar.gz
==> Created stage in /u/home/bengisu/spack/var/spack/stage/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4
==> No patches needed for lua-luafilesystem
==> Building lua-luafilesystem [Package]
==> Executing phase: 'install'
==> Error: ProcessError: Command exited with status 1:
    '/u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/lua-5.3.4-6ayqzzib727k3oocmd63ztuikmis5ny2/bin/luarocks' '--tree=/u/home/bengisu/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4' 'install' '/u/home/bengisu/spack/var/spack/stage/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4/luafilesystem-1_6_3/rockspecs/luafilesystem-1.6.3-1.rockspec'

1 error found in build log:
     4     Cloning into 'luafilesystem'...
     5     fatal: unable to connect to github.com:
     6     github.com[0: 192.30.253.112]: errno=Connection timed out
     7     github.com[1: 192.30.253.113]: errno=Connection timed out
     8     
     9     
  >> 10    Error: Failed cloning git repository.

See build log for details:
  /u/home/bengisu/spack/var/spack/stage/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4/luafilesystem-1_6_3/spack-build.out

It seems like this is a build problem rather than clone. I am not sure If it is ok to compile the extracted file manually by make but when I do cd /spack/var/spack/stage/lua-luafilesystem-1_6_3-zzhsmvvwdjiozxshcti6l64lpr4iimh4 and make I get this :

gcc -O2 -Wall -fPIC -W -Waggregate-return -Wcast-align -Wmissing-prototypes -Wnested-externs -Wshadow -Wwrite-strings -pedantic -I/usr/include   -c -o src/lfs.o src/lfs.c
src/lfs.c:63:17: fatal error: lua.h: No such file or directory
compilation terminated.
<builtin>: recipe for target 'src/lfs.o' failed
make: *** [src/lfs.o] Error 1

@adamjstewart
Copy link
Member

I was just able to install lua-luafilesystem without any problems. Are you on an air-gapped network? If the lua-luafilesystem package is trying to fetch additional resources during installation, that could be your problem. This reminds me of #3909. I think you must be running into the same problem.

@bengisuelis
Copy link
Author

bengisuelis commented Apr 22, 2018 •
edited

Are you on an air-gapped network?

Yes

This reminds me of #3909. I think you must be running into the same problem.

Yes, that's the same problem but I could install lua-luaposix without any problem though. It's only luafilesystem that causes error.

@bengisuelis
Copy link
Author

However the work around proposed in #3909:

My current work around is to

Download the files with git clone git@github.com:keplerproject/luafilesystem
Edit luafilesystem-1.6.3-1.rockspec that is located in the stage directory so that it the url entry points to the local file system location from the above step.
Runspack installagain.

doesn't work for me !

I have noticed that with each spack install lua-luafilesystem spack extracts the cloned file from the cache directory and overwrites the one I changed in stage directory. If I change the tar file in the cache dir and do spack install lua-luafilesystem it re-fetches and overwrites the one I changed again.

Might making Spack use a tar file in a local file system location, when it needs to fetch, be a solution ? I guess, I need to change the package.py in repos/builtin/packages/lua-luafilesystem. But I am not sure how can I do that. Or is there any other way to solve this ?

@samfux84
Copy link
Contributor

samfux84 commented May 24, 2018 •
edited

@bengisuelis In my case, the proxy server that I have to use was blocking the git protocol. You can tell git to always use https, even if it gets a git url:

After running the command

git config --global url.https://github.com/.insteadOf git://github.com/

Everything worked fine.

@bengisuelis
Copy link
Author

@samfux84 Thanks.

@mforde84
Copy link

@samfux84 love you, bro! :D

@KineticTheory KineticTheory mentioned this issue Apr 6, 2019
Closed
@sethrj
Copy link
Contributor

sethrj commented May 31, 2019

I hit this too. Thanks @samfux84 for the workaround!

@rdbisme rdbisme mentioned this issue Sep 4, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
fetching
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

luarocks: Use make command instead of install
6 participants
@sethrj @samfux84 @adamjstewart @KineticTheory @mforde84 @bengisuelis