Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libXrdCryptossl-4.so has references to build directory, how to get rid of them? #790

Closed
vp1981 opened this issue Aug 1, 2018 · 11 comments
Closed

libXrdCryptossl-4.so has references to build directory, how to get rid of them? #790

vp1981 opened this issue Aug 1, 2018 · 11 comments

Comments

@vp1981
Copy link
Contributor

vp1981 commented Aug 1, 2018

Hello,
I make (unofficial) packages for Archlinux (my Linux distribution) and makepkg (the distro package build tool) warns me that libXrdCryptossl-4.so has references to a build directory. When I run

$ strings libXrdCryptossl-4.so | grep BUILD_DIR

I see references to three files: XrdCrypto/XrdCryptosslCipher.cc, XrdCrypto/XrdCriptosslX509.cc and XrdCrypto/XrdCryptosslX509Crl.cc. My goal is either to get rid of them (see, for example, "reproducible builds") or make these paths relative by their location in source code directory. After quick glance on first file I found no reference to any special macros (for instance, ___FILE___). Does anyone known what is going on here? How to debug this?
@abh3
Copy link
Member

abh3 commented Aug 2, 2018 via email

@vp1981
Copy link
Contributor Author

vp1981 commented Aug 2, 2018

Hi abh3 (Andrew Hanushevsky),

on an RH build it works as expected (i.e. no references to these)

thanks for checking. Could you give a bit more details: do you use cmake, do you use the distribution tools for packaging (I don't know how RPM packages are created, of course, not "manually"), version of tools: gcc, cmake, glibc?

I suspect this might be due to cmake (I opened issue for xxHash program with the same question and the author replied that it is ok with make).

That aside, why is this causing a problem?

First: it is useless. Secondly, I don't understand how these paths are passed to the binaries, so I not sure what consequences it can cause.

@abh3
Copy link
Member

abh3 commented Aug 2, 2018 via email

@bbockelm
Copy link
Contributor

bbockelm commented Aug 2, 2018

Here's the info about RPATH:

https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/RPATH-handling#default-rpath-settings

Are you doing a "make install"? That should be stripping out the RPATH, although it appears the default behavior drifted through different releases.

Are you also stripping/relocating the debug symbols? I unfortunately don't have much background for Arch, but RHEL does something like this:

for f in `find "$RPM_BUILD_ROOT" -type f -a -exec file {} \; | \
        grep -v "^${RPM_BUILD_ROOT}/\?usr/lib/debug"  | \
        grep ' shared object,' | \
        sed -n -e 's/^\(.*\):[  ]*ELF.*, not stripped.*/\1/p'`; do
        $STRIP --strip-unneeded "$f"
done

If I manually run that against an install tree that refers to the build directory, I see references from the strings output disappear.

Reproducible builds are A Good Thing. Hope you can figure it out!

@vp1981
Copy link
Contributor Author

vp1981 commented Aug 3, 2018

2Andy (Andrew Hanushevsky):

We use a standard RPM script to build the RPM. You can find it in
xrootd/packaging/makesrpm.sh
it is driven off gthe particular platform in question as you will see in the enclosing directory. There is nothing magical about it. All the standad toold you mentioned are used.

I didn't think about and now looked into the packaging directory. Both debian and rhel use cmake to build packages, so at first I was confused. Then I tried to build rpm package on Fedora 28. I installed all the necessary tools and packages, then ran makesrpm.sh and after that rpmbuild --rebuild *.src.rpm. After you words I was surprised to find that xrootd-libs-20180801.df577ab9-1.fc28.x86_64.rpm has references to build directory, namely

$ strings libXrdCryptossl-4.so | grep /home
/home/vladimir/rpmbuild/BUILD/xrootd/xrootd/src/XrdCrypto/XrdCryptosslCipher.cc
/home/vladimir/rpmbuild/BUILD/xrootd/xrootd/src/XrdCrypto/XrdCryptosslX509.cc
/home/vladimir/rpmbuild/BUILD/xrootd/xrootd/src/XrdCrypto/XrdCryptosslX509Crl.cc

(I extracted this file from rpm using mc) whilst its debug version returns nothing:

$ strings libXrdCryptossl-4.so-20180801.df577ab9-1.fc28.x86_64.debug | grep /home

Be aware that when cmake creates artifacts in the "src" directory, it also sets runpath and a few other references so that you can run he executabes in he "src" directory and be sure you are not referencing anthing out side ot it.

I didn't think about that but now I think it is reasonable.

These get stripped off when you build an RPM and we strip them off manually for other kinds of builds.

Now I get puzzled. I tried to rebuild the package and checked rpm "archive" (I didn't install the package actually) and I see references to build directory, as in my case, so this is not the information that can be/should be stripped out?

he paths are forced by cmake to make sure that when you run something out of the cmake artifact directory (i.e. "src") you only reference things that are relevant to the build. When you create the distribution package, all of that stuff is stripped out so no side-effects should remain.

How this is stripped out? By strip? I tried manually do that on mention above .so file like

$ strip -s --strip-unneeded libXrdCryptossl-4.so

and still see

strings libXrdCryptossl-4.so | grep /home
/home/vladimir/rpmbuild/BUILD/xrootd/xrootd/src/XrdCrypto/XrdCryptosslCipher.cc
/home/vladimir/rpmbuild/BUILD/xrootd/xrootd/src/XrdCrypto/XrdCryptosslX509.cc
/home/vladimir/rpmbuild/BUILD/xrootd/xrootd/src/XrdCrypto/XrdCryptosslX509Crl.cc

2bbockelm (Brian Bockelman):

Here's the info about RPATH:
https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/RPATH-handling#default-rpath-settings

Thanks for the link, I'll read it later.

Are you doing a "make install"? That should be stripping out the RPATH, although it appears the default behavior drifted through different releases.

More:

make DESTDIR=${pkgdir} install

or less

DESTDIR=${pkgdir} cmake --build . --target install

Are you also stripping/relocating the debug symbols?

By default for my distribution.

I have impression that you guys know how the xrootd is build for RH, could you help me understand why I still see references to build directory for that .so? Then I could figure out what is going on my system.

@bbockelm
Copy link
Contributor

bbockelm commented Aug 3, 2018

Ah-ha!

Switching from RHEL7 to FC28, I suddenly see the build directory embedded in this shared library but not others. In fact, it's in the .rodata section, implying that it has snuck into the code itself.

However, I'm similarly stumped. I see nothing in the build logs for the Fedora build that would cause this.

@vp1981 - if you look at the object file (*.o), can you see which is producing this?

@ellert - any ideas as to why this occurs? Does Xrootd pass the Debian reproducible builds?

@vp1981
Copy link
Contributor Author

vp1981 commented Aug 3, 2018

@bbockelm,

Ok, this is what I did on my Archlinux host:

$ makepkg
...
$ cd src/build/src/CMakeFiles/XrdCryptossl-4.dir/XrdCrypto
$ strings XrdCryptosslCipher.cc.o | grep /home
/home/vladimir/pkgs/xrootd/src/xrootd-4.8.4/src/XrdCrypto/XrdCryptosslCipher.cc

Since I didn't understand why this should be in .rodata section I searched the Internet to find out how to "disassemble" the object file (I used mc to "view" object file in parsed and raw forms, in first case I didn't see that path but in the second case I see the path in literal) and found two links (wikipedia and osdev.org), so I tried that

$ objdump -s -j .rodata XrdCryptosslCipher.cc.o                                                                                                                   
                                                                                                                                                                                                                                              
XrdCryptosslCipher.cc.o:     file format elf64-x86-64

objdump: section '.rodata' mentioned in a -j option, but not found in any input file

I'm not very familiar with structure of object files but where that string lives in it?

@vp1981
Copy link
Contributor Author

vp1981 commented Aug 5, 2018

Hi,
I finished the check the various cmakes on FC 28: all three cmakes: 2.8.12.2, 3.11.4 and 3.12 give the same result. Now I suspect this is due to the GCC compiler but the same behavior I see (with cmake 3.11.4) with clang too, or may be it is the linker?

@bbockelm
Copy link
Contributor

bbockelm commented Aug 5, 2018

Yeah - I don't really think it is cmake as it would affect all Xrootd libraries equally.

The other thought I had was something with the dependencies - maybe it's picking something up from an OpenSSL header?

@vp1981
Copy link
Contributor Author

vp1981 commented Aug 6, 2018

@bbockelm ,
that was a clue! Indeed, it is openssl responsible. They define own macro OPENSSL_FILE in opensslconf.h (on FC 28 it is opensslconf-x86_64.h) but it is "protected" by macro OPENSSL_NO_FILENAMES. So it is enough to pass -DOPENSSL_NO_FILENAMES to *_FLAGS variable of cmake and after compilation one doesn't have any references in XrdCrypt directory to the source directory.

As for openssl package from CentOS7/RHEL7 it doesn't have such macros.

I suggest always pass -DOPENSSL_NO_FILENAMES to *FLAGS in CMakeLists.txt.

@vp1981 vp1981 mentioned this issue Aug 8, 2018
Open
@simonmichal
Copy link
Contributor

This has been solved in: 502907d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@vp1981 @bbockelm @abh3 @simonmichal