Compile grgsm_livemon and grgsm_livemon_headless with grcc #378

velichkov · 2018-02-21T23:19:56Z

Fixes #13

Remove the pre-built scripts from the repository
Install pkg-config in Debian/Ubuntu containers
It is needed to proper detect the grcc full path
Fix the parallel build.
- Override the GR_UNIQUE_TARGET function to not append a hash to the
  target name.
- Add explicit dependency between pygen_apps and _grgsm_swig
  The _grgs_swig needs to be built first because grcc evaluates and
  verify all imports
- Test the parallel build in the docker containers as well
Add Fedora 26 docker file

And remove the pre-built scripts from the repository Fixes ptrkrysik#13

It is needed to proper detect the grcc full path

- Override the GR_UNIQUE_TARGET function to not append a hash to the target name. - Add explicit dependency between pygen_apps and _grgsm_swig The _grgs_swig needs to be built first because grcc evaluates and verify all imports - Test the parallel build in the docker containers as well

On Fedora 26 when gr-gsm is installed in /usr/local/lib in order to work some paths have to be added to PYTHONPATH and LD_LIBRARY_PATH enviroment variables This fixes the following errors ImportError: No module named grgsm ImportError: No module named _grgsm_swig ImportError: libgrgsm-0.41.3.so.0.0.0: cannot open shared object file: No such file or directory

velichkov · 2018-02-22T00:48:26Z

On Fedora 26 when gr-gsm is installed in /usr/local/lib in order to work some paths have to be added to PYTHONPATH and LD_LIBRARY_PATH environment variables

export PYTHONPATH=/usr/local/lib64/python2.7/site-packages/:/usr/local/lib64/python2.7/site-packages/grgsm/:$PYTHONPATH
export LD_LIBRARY_PATH=/usr/local/lib64/:$LD_LIBRARY_PATH

And here are some logs

[root@2333638dd108 build]# cat /etc/redhat-release 
Fedora release 26 (Twenty Six)

[root@2333638dd108 build]# export
declare -x DISTTAG="f26container"
declare -x FBR="f26"
declare -x FGC="f26"
declare -x HOME="/root"
declare -x HOSTNAME="2333638dd108"
declare -x LS_COLORS="rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.m4a=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.oga=01;36:*.opus=01;36:*.spx=01;36:*.xspf=01;36:"
declare -x OLDPWD
declare -x PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
declare -x PWD="/src/build"
declare -x SHLVL="1"
declare -x TERM="xterm"

[root@2333638dd108 build]# /src/tests/scripts/decode.shTesting with:
  grgsm_decode -c tmp.cfile -s 574712 -m BCCH -t 0 -v --ppm -10

Traceback (most recent call last):
  File "/usr/local/bin/grgsm_decode", line 30, in <module>
    import grgsm
ImportError: No module named grgsm

   Result: FAILED

[root@2333638dd108 build]# PYTHONPATH=/usr/local/lib64/python2.7/site-packages/ /src/tests/scripts/decode.shTesting with:
  grgsm_decode -c tmp.cfile -s 574712 -m BCCH -t 0 -v --ppm -10

Traceback (most recent call last):
  File "/usr/local/bin/grgsm_decode", line 30, in <module>
    import grgsm
  File "/usr/local/lib64/python2.7/site-packages/grgsm/__init__.py", line 45, in <module>
    from grgsm_swig import *
  File "/usr/local/lib64/python2.7/site-packages/grgsm/grgsm_swig.py", line 17, in <module>
    _grgsm_swig = swig_import_helper()
  File "/usr/local/lib64/python2.7/site-packages/grgsm/grgsm_swig.py", line 16, in swig_import_helper
    return importlib.import_module('_grgsm_swig')
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named _grgsm_swig

   Result: FAILED

[root@2333638dd108 build]# PYTHONPATH=/usr/local/lib64/python2.7/site-packages/:/usr/local/lib64/python2.7/site-packages/grgsm/ /src/tests/scripts/decode.sh
Testing with:
  grgsm_decode -c tmp.cfile -s 574712 -m BCCH -t 0 -v --ppm -10

Traceback (most recent call last):
  File "/usr/local/bin/grgsm_decode", line 30, in <module>
    import grgsm
  File "/usr/local/lib64/python2.7/site-packages/grgsm/__init__.py", line 45, in <module>
    from grgsm_swig import *
  File "/usr/local/lib64/python2.7/site-packages/grgsm/grgsm_swig.py", line 17, in <module>
    _grgsm_swig = swig_import_helper()
  File "/usr/local/lib64/python2.7/site-packages/grgsm/grgsm_swig.py", line 16, in swig_import_helper
    return importlib.import_module('_grgsm_swig')
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: libgrgsm-0.41.3.so.0.0.0: cannot open shared object file: No such file or directory
   Result: FAILED

[root@2333638dd108 build]# PYTHONPATH=/usr/local/lib64/python2.7/site-packages/:/usr/local/lib64/python2.7/site-packages/grgsm/ LD_LIBRARY_PATH=/usr/local/lib64/ /src/tests/scripts/decode.sh
Testing with:
  grgsm_decode -c tmp.cfile -s 574712 -m BCCH -t 0 -v --ppm -10
   Result: PASSED

velichkov · 2018-03-20T15:29:18Z

Hi @pkrysik,

Have you had time to review this PR? It is closely related to #355 and a lot of users are affected.

I already reported it downstream to debian and @petterreinholdtsen prefers this to be fixed upstream first.

ptrkrysik · 2018-03-22T05:07:14Z

Hi @velichkov , I tried to add grcc compilation in the past but it didn't work well. Your changes to cmake files more look mature than mine, but I will first try to add them to a separate branch and test them in there. Many thanks for the changes.

ptrkrysik · 2018-03-22T06:54:57Z

There was some problem with grgsm_livemon rule in Ubuntu 16.04 test. Strange because there are two tests of Ubuntu 16.04 and one passes :/. I will make some dummy commit to make the travis tests run again.

ptrkrysik · 2018-03-22T07:32:05Z

It's ok now. The problem appeared because in the meantime I added a new test that didn't have your changes.

pkrysik · 2018-04-08T05:11:26Z

There are some problems with grcc compilation :/. Here it worked: https://travis-ci.org/ptrkrysik/gr-gsm/builds/360423727 and here grcc failed (error message says that due to lack of _runtime_swig.so): https://travis-ci.org/ptrkrysik/gr-gsm/builds/363104893 Both tests for the same commit to make sure that it is not fault of some change in gr-gsm. For some reason grcc compilation is not stable.

velichkov · 2018-04-08T21:13:04Z

Hi @ptrkrysik,

and here grcc failed (error message says that due to lack of _runtime_swig.so):

This seems to be a problem with the debian's gnuradio as grcc always fails with the same error even when started without arguments.

root@e9d9a0f15e75:/src/build# dpkg -l gnuradio
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                            Version              Architecture         Description
+++-===============================-====================-====================-====================================================================
ii  gnuradio                        3.7.11-8             amd64                GNU Radio Software Radio Toolkit

root@e9d9a0f15e75:/src/build# grcc
Traceback (most recent call last):
  File "/usr/bin/grcc", line 29, in <module>
    from gnuradio import gr
  File "/usr/lib/python2.7/dist-packages/gnuradio/gr/__init__.py", line 41, in <module>
    from runtime_swig import *
  File "/usr/lib/python2.7/dist-packages/gnuradio/gr/runtime_swig.py", line 17, in <module>
    _runtime_swig = swig_import_helper()
  File "/usr/lib/python2.7/dist-packages/gnuradio/gr/runtime_swig.py", line 16, in swig_import_helper
    return importlib.import_module('_runtime_swig')
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named _runtime_swig

There is a new version 3.7.12.0-1 in debian unstable where the problem is fixed so I guess we need to wait few more days until this version is migrated to testing.

root@e9d9a0f15e75:/src/build# dpkg -l gnuradio
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                            Version              Architecture         Description
+++-===============================-====================-====================-====================================================================
ii  gnuradio                        3.7.12.0-1           amd64                GNU Radio Software Radio Toolkit
root@e9d9a0f15e75:/src/build# grcc
Please specify a GRC file name to compile.

The failure on Fedora 26 is different and I think it is related with the parallel build -j $(nproc) and is not because of grcc. I will try to fix this as well

[ 0%] Built target grgsm_swig_swig_doc
Scanning dependencies of target grgsm
Scanning dependencies of target pygen_swig_72fe6
make[2]: *** No rule to make target 'swig/grgsm_swig.py', needed by 'swig/grgsm_swig.pyc'. Stop.
make[1]: *** [CMakeFiles/Makefile2:665: swig/CMakeFiles/pygen_swig_72fe6.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

velichkov · 2018-04-08T22:14:43Z

Hi @ptrkrysik,

After some more digging I found a dirty workaround for the debian problem. To make it work we need to create one symlink

ln -sf /usr/lib/x86_64-linux-gnu/libvolk.so.1.3.1 /usr/lib/x86_64-linux-gnu/libvolk.so.1.3

Here is the full log

root@46f266ad7314:/src/build# make
[ 63%] Built target grgsm
[ 63%] Built target grgsm_swig_swig_doc
[ 66%] Built target _grgsm_swig_swig_tag
[ 70%] Built target grgsm_swig_swig_2d0df
[ 75%] Built target _grgsm_swig
[ 78%] Built target pygen_swig_72fe6
[ 81%] Built target pygen_python_bd098
[ 83%] Built target pygen_python_misc_utils_d6433
[ 86%] Built target pygen_python_receiver_d6696
[ 88%] Built target pygen_python_demapping_9c707
[ 90%] Generating grgsm_livemon
Traceback (most recent call last):
  File "/usr/bin/grcc", line 29, in <module>
    from gnuradio import gr
  File "/usr/lib/python2.7/dist-packages/gnuradio/gr/__init__.py", line 41, in <module>
    from runtime_swig import *
  File "/usr/lib/python2.7/dist-packages/gnuradio/gr/runtime_swig.py", line 17, in <module>
    _runtime_swig = swig_import_helper()
  File "/usr/lib/python2.7/dist-packages/gnuradio/gr/runtime_swig.py", line 16, in swig_import_helper
    return importlib.import_module('_runtime_swig')
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named _runtime_swig
apps/CMakeFiles/pygen_apps.dir/build.make:79: recipe for target 'apps/grgsm_livemon' failed
make[2]: *** [apps/grgsm_livemon] Error 1
CMakeFiles/Makefile2:1099: recipe for target 'apps/CMakeFiles/pygen_apps.dir/all' failed
make[1]: *** [apps/CMakeFiles/pygen_apps.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2
root@46f266ad7314:/src/build# 
root@46f266ad7314:/src/build# 
root@46f266ad7314:/src/build# ln -sf /usr/lib/x86_64-linux-gnu/libvolk.so.1.3.1 /usr/lib/x86_64-linux-gnu/libvolk.so.1.3
root@46f266ad7314:/src/build# 
root@46f266ad7314:/src/build# 
root@46f266ad7314:/src/build# make
[ 63%] Built target grgsm
[ 63%] Built target grgsm_swig_swig_doc
[ 66%] Built target _grgsm_swig_swig_tag
[ 70%] Built target grgsm_swig_swig_2d0df
[ 75%] Built target _grgsm_swig
[ 78%] Built target pygen_swig_72fe6
[ 81%] Built target pygen_python_bd098
[ 83%] Built target pygen_python_misc_utils_d6433
[ 86%] Built target pygen_python_receiver_d6696
[ 88%] Built target pygen_python_demapping_9c707
[ 90%] Generating grgsm_livemon
linux; GNU C++ version 7.3.0; Boost_106200; UHD_003.010.003.000-0-unknown

[ 91%] Shebangin grgsm_livemon
[ 92%] Generating grgsm_livemon_headless
linux; GNU C++ version 7.3.0; Boost_106200; UHD_003.010.003.000-0-unknown

[ 93%] Shebangin grgsm_livemon_headless
[ 96%] Built target pygen_apps
[ 98%] Built target pygen_apps_helpers_e5341
[100%] Built target man_grgsm_livemon.rst

See ptrkrysik#378 (comment) Traceback (most recent call last): File "/usr/bin/grcc", line 29, in <module> from gnuradio import gr File "/usr/lib/python2.7/dist-packages/gnuradio/gr/__init__.py", line 41, in <module> from runtime_swig import * File "/usr/lib/python2.7/dist-packages/gnuradio/gr/runtime_swig.py", line 17, in <module> _runtime_swig = swig_import_helper() File "/usr/lib/python2.7/dist-packages/gnuradio/gr/runtime_swig.py", line 16, in swig_import_helper return importlib.import_module('_runtime_swig') File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) ImportError: No module named _runtime_swig

There is a problem with cmake 3.11.0 and the parallel build fails with See ptrkrysik#378 (comment) make[2]: *** No rule to make target 'swig/grgsm_swig.py', needed by 'swig/grgsm_swig.pyc'. Stop. make[1]: *** [CMakeFiles/Makefile2:601: swig/CMakeFiles/pygen_swig_43235.dir/all] Error 2

See #378 (comment) Traceback (most recent call last): File "/usr/bin/grcc", line 29, in <module> from gnuradio import gr File "/usr/lib/python2.7/dist-packages/gnuradio/gr/__init__.py", line 41, in <module> from runtime_swig import * File "/usr/lib/python2.7/dist-packages/gnuradio/gr/runtime_swig.py", line 17, in <module> _runtime_swig = swig_import_helper() File "/usr/lib/python2.7/dist-packages/gnuradio/gr/runtime_swig.py", line 16, in swig_import_helper return importlib.import_module('_runtime_swig') File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) ImportError: No module named _runtime_swig

There is a problem with cmake 3.11.0 and the parallel build fails with See #378 (comment) make[2]: *** No rule to make target 'swig/grgsm_swig.py', needed by 'swig/grgsm_swig.pyc'. Stop. make[1]: *** [CMakeFiles/Makefile2:601: swig/CMakeFiles/pygen_swig_43235.dir/all] Error 2

See ptrkrysik/gr-gsm#378 (comment) Traceback (most recent call last): File "/usr/bin/grcc", line 29, in <module> from gnuradio import gr File "/usr/lib/python2.7/dist-packages/gnuradio/gr/__init__.py", line 41, in <module> from runtime_swig import * File "/usr/lib/python2.7/dist-packages/gnuradio/gr/runtime_swig.py", line 17, in <module> _runtime_swig = swig_import_helper() File "/usr/lib/python2.7/dist-packages/gnuradio/gr/runtime_swig.py", line 16, in swig_import_helper return importlib.import_module('_runtime_swig') File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) ImportError: No module named _runtime_swig

There is a problem with cmake 3.11.0 and the parallel build fails with See ptrkrysik/gr-gsm#378 (comment) make[2]: *** No rule to make target 'swig/grgsm_swig.py', needed by 'swig/grgsm_swig.pyc'. Stop. make[1]: *** [CMakeFiles/Makefile2:601: swig/CMakeFiles/pygen_swig_43235.dir/all] Error 2

velichkov added 5 commits February 21, 2018 22:38

Compile the .grc files with cmake and grcc

067d42c

And remove the pre-built scripts from the repository Fixes ptrkrysik#13

Install pkg-config in Debian/Ubuntu containers

f6bcd20

It is needed to proper detect the grcc full path

Add Fedora 26 docker file

bc08bd3

ptrkrysik changed the base branch from development to grcc_compile March 22, 2018 05:02

ptrkrysik merged commit 753afeb into ptrkrysik:grcc_compile Mar 22, 2018

velichkov mentioned this pull request Mar 23, 2018

AttributeError: 'module' object has no attribute 'util' #375

Closed

velichkov deleted the grcc_compile branch April 6, 2018 15:47

This was referenced Apr 9, 2018

Grcc and cmake failures from #378 #391

Merged

got this error : AttributeError: 'module' object has no attribute 'QWidget' #355

Closed

velichkov mentioned this pull request Apr 23, 2018

Swig - no matching function for call on CentOS 7 #396

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compile grgsm_livemon and grgsm_livemon_headless with grcc #378

Compile grgsm_livemon and grgsm_livemon_headless with grcc #378

velichkov commented Feb 21, 2018

velichkov commented Feb 22, 2018

velichkov commented Mar 20, 2018

ptrkrysik commented Mar 22, 2018

ptrkrysik commented Mar 22, 2018

ptrkrysik commented Mar 22, 2018

pkrysik commented Apr 8, 2018 via email

velichkov commented Apr 8, 2018

velichkov commented Apr 8, 2018

Compile grgsm_livemon and grgsm_livemon_headless with grcc #378

Compile grgsm_livemon and grgsm_livemon_headless with grcc #378

Conversation

velichkov commented Feb 21, 2018

velichkov commented Feb 22, 2018

velichkov commented Mar 20, 2018

ptrkrysik commented Mar 22, 2018

ptrkrysik commented Mar 22, 2018

ptrkrysik commented Mar 22, 2018

pkrysik commented Apr 8, 2018 via email

velichkov commented Apr 8, 2018

velichkov commented Apr 8, 2018