Error when trying to execute bitbake -c analyse_sources_all core-image-sato

[ereshetova]

...
WARNING: Failed to fetch URL http://www.apache.org/dist/subversion/subversion-1.8.13.tar.bz2, attempting MIRRORS if available
ERROR: Error executing a python function in /home/epetmab/git/poky/meta/recipes-devtools/gcc/libgcc-initial_4.9.bb:

The stack trace of python calls that resulted in this exception/failure was:
File: 'do_analyse_sources', lineno: 64, function:
0060:
0061: return
0062:
0063:
*** 0064:do_analyse_sources(d)
0065:
File: 'do_analyse_sources', lineno: 29, function: do_analyse_sources
0025:
0026: fetch = bb.fetch2.Fetch([], d)
0027: for url in fetch.urls:
0028: workdir = d.getVar('ISAFW_WORKDIR', True)
*** 0029: fetch.unpack(workdir, (url,))
0030:
0031: recipe = isafw.ISA_package()
0032: recipe.name = d.getVar('PN', True)
0033: recipe.version = d.getVar('PV', True)
File: '/home/epetmab/git/poky/bitbake/lib/bb/fetch2/init.py', lineno: 1686, function: unpack
1682:
1683: if ud.lockfile:
1684: lf = bb.utils.lockfile(ud.lockfile)
1685:
*** 1686: ud.method.unpack(ud, root, self.d)
1687:
1688: if ud.lockfile:
1689: bb.utils.unlockfile(lf)
1690:
File: '/home/epetmab/git/poky/bitbake/lib/bb/fetch2/init.py', lineno: 1458, function: unpack
1454:
1455: os.chdir(save_cwd)
1456:
1457: if ret != 0:
*** 1458: raise UnpackError("Unpack command %s failed with return value %s" % (cmd, ret), urldata.url)
1459:
1460: if iterate is True:
1461: iterate_urldata = urldata
1462: iterate_urldata.localpath = "%s/%s" % (rootdir, iterate_file)
Exception: UnpackError: Unpack failure for URL: 'http://ftp.gnu.org/gnu/gcc/gcc-4.9.3/gcc-4.9.3.tar.bz2'. Unpack command PATH="/home/epetmab/programming/yocto/isafw_analyse_test/tmp/sysroots/x86_64-linux/usr/bin/x86_64-poky-linux.gcc-cross-initial-x86_64:/home/epetmab/git/poky/scripts:/home/epetmab/programming/yocto/isafw_analyse_test/tmp/sysroots/x86_64-linux/usr/bin/x86_64-poky-linux:/home/epetmab/programming/yocto/isafw_analyse_test/tmp/sysroots/qemux86-64/usr/bin/crossscripts:/home/epetmab/programming/yocto/isafw_analyse_test/tmp/sysroots/x86_64-linux/usr/sbin:/home/epetmab/programming/yocto/isafw_analyse_test/tmp/sysroots/x86_64-linux/usr/bin:/home/epetmab/programming/yocto/isafw_analyse_test/tmp/sysroots/x86_64-linux/sbin:/home/epetmab/programming/yocto/isafw_analyse_test/tmp/sysroots/x86_64-linux/bin:/home/epetmab/git/poky/scripts:/home/epetmab/git/poky/bitbake/bin:/home/epetmab/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/quest/bin:/opt/quest/sbin:/opt/elx/bin:/opt/elx/bin:/home/epetmab/bin:/home/epetmab/bin" bzip2 -dc /home/epetmab/programming/yocto/isafw_analyse_test/downloads/gcc-4.9.3.tar.bz2 | tar x --no-same-owner -f - failed with return value 2

ERROR: Function failed: do_analyse_sources
ERROR: Logfile of failure stored in: /home/epetmab/programming/yocto/isafw_analyse_test/tmp/work/core2-64-poky-linux/libgcc-initial/4.9.3-r0/temp/log.do_analyse_sources.824
ERROR: Task 458 (/home/epetmab/git/poky/meta/recipes-devtools/gcc/libgcc-initial_4.9.bb, do_analyse_sources) failed with exit code '1'
NOTE: Tasks Summary: Attempted 881 tasks of which 354 didn't need to be rerun and 1 failed.
Waiting for 0 running tasks to finish:

Summary: 1 task failed:

END ERROR MSG

[ereshetova]

@bluelightning, @rossburton, could you please help with figuring out what is going here?

[belzerus]

@ereshetova, @bluelightning, @rossburton, just for your information I run 'bitbake -c analyse_sources_all core-image-sato' right away without running 'bitbake core-image-sato' first and according to Elena this problem has been seen before when using the analyse task with a "clean tree".

[rossburton]

That's probably because gcc is 'special' and needs special handling when attempting the extract the sources. archiver.bbclass and the mentions of shared source might be useful to help.

[ereshetova]

What I figured out is if I do bitbake -c fetch libgcc, then gcc tar ball is NOT downloaded. And that's why my task fails to unpack the source. However, it seems that if I do bitbake -c fetch gcc-source, it gets the source. So, I am trying to solve this now by adding this dependency to my task:

do_analyse_sources[depends] += "gcc-source:do_fetch"

[ereshetova]

Which seems to solve it in my builds. Not sure it is absolutely the right way of doing it, but it works.

[ereshetova]

@belzerus, could you check that it is also fixed for you with this commit: 3da393c

[belzerus]

Nice. @ereshetova I will give it a try and let you know about the result. But the change/workaround sounds reasonable for me.

[belzerus]

@ereshetova, it solves the problem when using poky:fido. But when using poky:master I get some problems since there has been some changes in the gcc-sources structure.

$ bitbake -c analyse_sources_all core-image-sato [~BUILDDIR]
NOTE: Started PRServer with DBfile:programming/yocto/isafw_new_test/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 41711, PID: 27267
Loading cache: 100% |###################################################################################################################################################################################| ETA: 00:00:00
Loaded 1313 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies
ERROR: Nothing PROVIDES 'gcc-source'. Close matches:
gcc-source-5.2.0
gcc-source-4.9.3
gcc-source-4.8.4
ERROR: Required build target 'core-image-sato' has no buildable providers.
Missing or unbuildable dependency chain was: ['core-image-sato', 'gcc-source']

Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

[ereshetova]

Hmm.. I will take a look today. I am testing on the fido branch normally, so I didn't see this :(

[ereshetova]

What seems to be the state is master now is that they fixed the problem with libgcc that we initially had, so if you do bitbake -c fetch libgcc, it actually fetches the sources correctly. I am making a test run now, it is possible that for master we don't the fix anymore.

[ereshetova]

No, it doesn't work and there doesn't seem to be a sane way to specify the dependency in a way that correct gcc code is fetched. I will write tooe-core mailing list asking the right way to deal with this.

[ereshetova]

This is the commit that fixes the actual issue on oe-core side:
http://git.openembedded.org/openembedded-core/commit/?id=0df9d45e0be59e55e585e6d25dedbf0fc55c490c

I therefore reverted the original commit (fcf08a8) and now things work for me given that oe-core master branch is used.

@belzerus, could you confirm that it works for you also? Sorry that I took so much time, but I was really waiting on proper solution from oe-core side.

[belzerus]

@ereshetova, nice. I will try this today and let you know about the result. But the fix itself seems reasonable and I agree that it's better to solve this in oe-core then doing some kind of workaround in isafw layer.

[belzerus]

@ereshetova, now the command 'bitbake -c analyse_sources_all core-image-minimal' was executed successful. So I guess you can close this issue. One other thing I noticed was that no logs was produced when running above command (besides license_report) both in that case and when running 'bitbake core-image-minimal' cve-report was empty as well. Is that correct or do I need to do anything else to enable cve-report? Feel free to close this issue and we can discuss the other issues in a separate issue.

[ereshetova]

No, cve-report should not be empty. Do you use any proxies? It does need a network access and proxy exported normally in order to work properly. I will close the current issue.