New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static/dynamic executable implementation #3357

Merged
merged 2 commits into from Jul 7, 2018

Conversation

Projects
None yet
2 participants
@NK-Nikunj
Contributor

NK-Nikunj commented Jun 27, 2018

Proposed Changes

  • This pr adds static/dynamic executable implementation by using linker option -Wl,-wrap.
  • Adds hpx_wrap.cpp into libhpx_init.a
  • Configured cmakelists to build and integrate hpx_wrap and add the compiler flags for Linux systems.

Any background context you want to provide?

I have used the linker option -Wl,-wrap to wrap the __libc_start_main (unlike last time where I used dlfcn to hook into it). By using the linker option the function can be understood as __wrap___libc_start_main and __real___libc_start_main. Any call to the actual function will be received by our wrapper function. It is from here that we change the entry point of the C runtime system to our own custom function (initializing_main in our case). From here we configure the HPX runtime system and initialize it with the entry point being hpx::hpx_entry. From the initialized system we call the C main function with arguments.

Using this method to generate an executable

Since this implementation requires linking to libhpx_wrap.a (build during HPX build itself), I have added a flag by the name of INCLUDE_HPX_MAIN (subject to change, if required). Addition of the flag to add_executable will take care of the linking part.

TODO

  • Add flag to all examples and tests
  • Output error if the flag is not provided but hpx_main.hpp is included
  • Output error if the header is provided but the flag is not

Failed attempt

I tried to initialize the HPX runtime system in such a manner that it would work for any global scoped object (utilizing HPX functionality) but it failed. The reason for it to fail was due to mismatch of the initialization sequence of std library, HPX runtime system and C runtime system. I tried to force initialize buffers and libraries but it still did not work as expected (It was able to run a few codes correctly while producing errors for the rest).

I am still trying to figure out ways to incorporate global objects into HPX but I can't figure out a way as of now.

Edit:

Implementation working environment

Since my implementation depends upon the Linux linker's (ld) --wrap option, this implementation becomes compiler independent (I have only tried with gcc and clang, but in theory, it should work with all compilers). This implementation is also limited to Linux operating system.

Improvised implementation

During a conversation with @hkaiser on irc he proposed to integrate the static library in libhpx_init.a and told me to make it work that way. So I have integrated it directly with libhpx_init.a and now the workings of my code is decided at runtime. It is done so with the help of a weak variable include_libhpx_wrap which has a default value of 0 and 1 when hpx_main is included. This way the runtime system is able to decide using a simple if-else construct.

P.S. Most of the tests will not pass since the flag has not been added.

if(NOT target_NOLIBS)
set(hpx_libs hpx)
set(hpx_libs hpx ${hpx_libs})
if(NOT target_STATIC_LINKING)
set(hpx_libs ${hpx_libs})

This comment has been minimized.

@hkaiser

hkaiser Jul 3, 2018

Member

Can we remove ${hpx_libs} here now?

This comment has been minimized.

@NK-Nikunj

NK-Nikunj Jul 3, 2018

Contributor

oh, I forgot to remove it from there. I'll do it now

# add linker flags for libhpx_init.a for linux environment
if(CMAKE_SYSTEM_NAME STREQUAL Linux)
set_target_properties(${target} PROPERTIES LINK_FLAGS "-Wl,-wrap=__libc_start_main")
endif()

This comment has been minimized.

@hkaiser

hkaiser Jul 3, 2018

Member

This adds the overloaded entry point unconditionally. Is that what we want? Shouldn't this be done for executables only (or at least those modules that link against hpx_init)?

This comment has been minimized.

@NK-Nikunj

NK-Nikunj Jul 3, 2018

Contributor

Yes, this should be done for executables only. I will add another parameter to the if statement to add for target type executables.

@@ -9,6 +10,18 @@
#include <hpx/config.hpp>
#include <hpx/hpx_init.hpp>
// We support different implementation depending upon the Operating
// System in use.
#if defined(__linux__)

This comment has been minimized.

@hkaiser

hkaiser Jul 3, 2018

Member

We usually check for Linux as follows:

#if defined(__linux) || defined(__linux__) || defined(linux)
...
#endif

Different flavors of Linux define different pp constants.

This comment has been minimized.

@NK-Nikunj

NK-Nikunj Jul 3, 2018

Contributor

I will update it as you have described.

// to change the program's entry point or not.
extern int include_libhpx_wrap;
int include_libhpx_wrap = 1;

This comment has been minimized.

@hkaiser

hkaiser Jul 3, 2018

Member

Could you place these global variables into a separate namespace, please? In other spots we use namespace hpx_startup {} for this.

// file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
// The following implementation is only possible for Linux systems.
#if defined(__linux__)

This comment has been minimized.

@hkaiser

hkaiser Jul 3, 2018

Member

Same as above, please use all three flavors of the Linux pp.

// HPX's implented program's entry point
extern int initialize_main(int argc, char** argv, char** envp);
// Real libc function __libc_start_main

This comment has been minimized.

@hkaiser

hkaiser Jul 3, 2018

Member

Please add a comment with a link to the source code of the original libc implementation.

// actual_main is responsible to store the pointer to the main()
// function. POSIX implementation of main requires pointer to envp
// so it is stored as well.
int (*actual_main)(int, char**, char**) = nullptr;

This comment has been minimized.

@hkaiser

hkaiser Jul 3, 2018

Member

Please move these globals into a separate namespace.

// Configuring HPX system before runtime
std::vector<std::string> const cfg = {
"hpx.commandline.allow_unknown!=1",
};

This comment has been minimized.

@hkaiser

hkaiser Jul 3, 2018

Member

Please also disable the short command line options, similarly to https://github.com/STEllAR-GROUP/hpx/blob/master/hpx/hpx_main_impl.hpp#L31-L34

if(CMAKE_SYSTEM_NAME STREQUAL Linux)
set(HPX_LINKER_FLAGS "-Wl,-wrap=__libc_start_main")
endif()

This comment has been minimized.

@hkaiser

hkaiser Jul 6, 2018

Member

I think HPX_LINKER_FLAGS should be propagated to HPXConfig.cmake as well, from where it could go into HPX_SetupTarget (see https://github.com/STEllAR-GROUP/hpx/blob/master/cmake/templates/HPXConfig.cmake.in)

This comment has been minimized.

@NK-Nikunj

NK-Nikunj Jul 6, 2018

Contributor

ok I'll add this variable to HPXConfig.cmake.

// System in use.
#if defined(__linux) || defined(__linux__) || defined(linux)
namespace hpx {

This comment has been minimized.

@hkaiser

hkaiser Jul 6, 2018

Member

Everything in namespace hpx is documented for user consumption, May I suggest to use namespace hpx_start or similar?

This comment has been minimized.

@NK-Nikunj

NK-Nikunj Jul 6, 2018

Contributor

I'll change the namespace to hpx_start.

Configured cmake and pkg-config for Linux implementation, changed nam…
…espaces to hpx_start, changed variable type to bool
@hkaiser

hkaiser approved these changes Jul 7, 2018

LGTM, many thanks!

@hkaiser hkaiser merged commit ad566a3 into STEllAR-GROUP:master Jul 7, 2018

43 of 44 checks passed

ci/circleci: install Your tests failed on CircleCI
Details
ci/circleci: checkout_code Your tests passed on CircleCI!
Details
ci/circleci: clang_tidy Your tests passed on CircleCI!
Details
ci/circleci: configure Your tests passed on CircleCI!
Details
ci/circleci: core Your tests passed on CircleCI!
Details
ci/circleci: examples Your tests passed on CircleCI!
Details
ci/circleci: inspect Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.compat Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.components Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.compute Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.config Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.include Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.lcos Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.parallel Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.performance_counters Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.plugins Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.runtime Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.traits Your tests passed on CircleCI!
Details
ci/circleci: tests.headers.util Your tests passed on CircleCI!
Details
ci/circleci: tests.performance Your tests passed on CircleCI!
Details
ci/circleci: tests.regressions Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.actions Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.agas Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.build Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.component Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.computeapi Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.diagnostics Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.lcos Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.parallel.algorithms Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.parallel.block Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.parallel.container_algorithms Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.parallel.datapar_algorithms Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.parallel.executors Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.parallel.segmented_algorithms Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.parcelset Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.performance_counter Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.resource Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.serialization Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.threads Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.traits Your tests passed on CircleCI!
Details
ci/circleci: tests.unit.util Your tests passed on CircleCI!
Details
pycicle daint-3357-clang-6.0.0-Boost-1.65.0-Release Build errors 0
Details
pycicle daint-3357-clang-6.0.0-Boost-1.65.0-Release Config errors 0
Details
pycicle daint-3357-clang-6.0.0-Boost-1.65.0-Release Test errors 0
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment