Compiling for aarch64 architecture

[sgiannoulis]

Issue Description

Trying to compile OpenR for aarch64 linux distro, The cmake hardcoded in the install scripts is a x86-64 executable. I have compiled and installed cmake 13.4.0 on my own for this platform but the install scripts still tries to use the hardcoded cmake executables which of course fail. Please propose a way to disable cmake from the install scripts and just default using the native cmake installed in a system for building. (FYI there is no aarch64 precompiled lib available from the cmake repo, so just editing the cmake manifest file to point to a aarch64 variant is not possible for 13.4.0 version of cmake). Tried also with cmake 20.0.0 that is available in aarch64 version but it fails to compile as some deps are not detected correctly (like openssl).

Environment

• tag or commit hash on which this occured
  master latest
• OS version: <e.g. ubuntu-16.04>
  Ubuntu 16.04 lts aarch64

Minimal test code / Steps to reproduce the issue

initiate openR build script

What's the actual result?

Failed to compile libevent library when changing to 20.0.0 cmake that is available in aarch64 version.

-- The C compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/local/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Git: /usr/bin/git
fatal: Not a git repository (or any of the parent directories): .git
-- Performing Test check_c_compiler_flag__Wall
-- Performing Test check_c_compiler_flag__Wall - Success
-- Performing Test check_c_compiler_flag__Wswitch
-- Performing Test check_c_compiler_flag__Wswitch - Success
-- Performing Test check_c_compiler_flag__fno_strict_aliasing
-- Performing Test check_c_compiler_flag__fno_strict_aliasing - Success
-- Looking for GNU_LIBRARY
-- Looking for GNU_LIBRARY - found
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for sys/socket.h
-- Looking for sys/socket.h - found
-- Looking for netinet/in.h
-- Looking for netinet/in.h - found
-- Looking for netinet/in6.h
-- Looking for netinet/in6.h - not found
-- Looking for unistd.h
-- Looking for unistd.h - found
-- Looking for netdb.h
-- Looking for netdb.h - found
-- Looking for dlfcn.h
-- Looking for dlfcn.h - found
-- Looking for arpa/inet.h
-- Looking for arpa/inet.h - found
-- Looking for fcntl.h
-- Looking for fcntl.h - found
-- Looking for inttypes.h
-- Looking for inttypes.h - found
-- Looking for memory.h
-- Looking for memory.h - found
-- Looking for poll.h
-- Looking for poll.h - found
-- Looking for port.h
-- Looking for port.h - not found
-- Looking for signal.h
-- Looking for signal.h - found
-- Looking for stdarg.h
-- Looking for stdarg.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stdlib.h
-- Looking for stdlib.h - found
-- Looking for strings.h
-- Looking for strings.h - found
-- Looking for string.h
-- Looking for string.h - found
-- Looking for sys/devpoll.h
-- Looking for sys/devpoll.h - not found
-- Looking for sys/epoll.h
-- Looking for sys/epoll.h - found
-- Looking for sys/eventfd.h
-- Looking for sys/eventfd.h - found
-- Looking for sys/event.h
-- Looking for sys/event.h - not found
-- Looking for sys/ioctl.h
-- Looking for sys/ioctl.h - found
-- Looking for sys/mman.h
-- Looking for sys/mman.h - found
-- Looking for sys/param.h
-- Looking for sys/param.h - found
-- Looking for sys/queue.h
-- Looking for sys/queue.h - found
-- Looking for sys/select.h
-- Looking for sys/select.h - found
-- Looking for sys/sendfile.h
-- Looking for sys/sendfile.h - found
-- Looking for sys/stat.h
-- Looking for sys/stat.h - found
-- Looking for sys/time.h
-- Looking for sys/time.h - found
-- Looking for sys/uio.h
-- Looking for sys/uio.h - found
-- Looking for include files sys/types.h, ifaddrs.h
-- Looking for include files sys/types.h, ifaddrs.h - found
-- Looking for mach/mach_time.h
-- Looking for mach/mach_time.h - not found
-- Looking for netinet/tcp.h
-- Looking for netinet/tcp.h - found
-- Looking for sys/wait.h
-- Looking for sys/wait.h - found
-- Looking for sys/resource.h
-- Looking for sys/resource.h - found
-- Looking for sys/sysctl.h
-- Looking for sys/sysctl.h - found
-- Looking for sys/timerfd.h
-- Looking for sys/timerfd.h - found
-- Looking for errno.h
-- Looking for errno.h - found
-- Looking for epoll_create
-- Looking for epoll_create - found
-- Looking for epoll_ctl
-- Looking for epoll_ctl - found
-- Looking for eventfd
-- Looking for eventfd - found
-- Looking for clock_gettime
-- Looking for clock_gettime - found
-- Looking for fcntl
-- Looking for fcntl - found
-- Looking for getaddrinfo
-- Looking for getaddrinfo - found
-- Looking for getnameinfo
-- Looking for getnameinfo - found
-- Looking for gettimeofday
-- Looking for gettimeofday - found
-- Looking for getprotobynumber
-- Looking for getprotobynumber - found
-- Looking for getservbyname
-- Looking for getservbyname - found
-- Looking for inet_ntop
-- Looking for inet_ntop - found
-- Looking for inet_pton
-- Looking for inet_pton - found
-- Looking for kqueue
-- Looking for kqueue - not found
-- Looking for mmap
-- Looking for mmap - found
-- Looking for pipe
-- Looking for pipe - found
-- Looking for pipe2
-- Looking for pipe2 - found
-- Looking for poll
-- Looking for poll - found
-- Looking for port_create
-- Looking for port_create - not found
-- Looking for sendfile
-- Looking for sendfile - found
-- Looking for sigaction
-- Looking for sigaction - found
-- Looking for signal
-- Looking for signal - found
-- Looking for splice
-- Looking for splice - found
-- Looking for strlcpy
-- Looking for strlcpy - not found
-- Looking for strsep
-- Looking for strsep - found
-- Looking for strtok_r
-- Looking for strtok_r - found
-- Looking for strtoll
-- Looking for strtoll - found
-- Looking for vasprintf
-- Looking for vasprintf - found
-- Looking for sysctl
-- Looking for sysctl - found
-- Looking for accept4
-- Looking for accept4 - found
-- Looking for arc4random
-- Looking for arc4random - not found
-- Looking for arc4random_buf
-- Looking for arc4random_buf - not found
-- Looking for epoll_create1
-- Looking for epoll_create1 - found
-- Looking for getegid
-- Looking for getegid - found
-- Looking for geteuid
-- Looking for geteuid - found
-- Looking for getifaddrs
-- Looking for getifaddrs - found
-- Looking for issetugid
-- Looking for issetugid - not found
-- Looking for mach_absolute_time
-- Looking for mach_absolute_time - not found
-- Looking for nanosleep
-- Looking for nanosleep - found
-- Looking for usleep
-- Looking for usleep - found
-- Looking for timeradd
-- Looking for timeradd - not found
-- Looking for timerclear
-- Looking for timerclear - not found
-- Looking for timercmp
-- Looking for timercmp - not found
-- Looking for timerfd_create
-- Looking for timerfd_create - found
-- Looking for timerisset
-- Looking for timerisset - not found
-- Looking for putenv
-- Looking for putenv - found
-- Looking for setenv
-- Looking for setenv - found
-- Looking for setrlimit
-- Looking for setrlimit - found
-- Looking for umask
-- Looking for umask - found
-- Looking for unsetenv
-- Looking for unsetenv - found
-- Looking for gethostbyname_r
-- Looking for gethostbyname_r - found
-- Checking prototype gethostbyname_r for EVENT__HAVE_GETHOSTBYNAME_R_3_ARG - False
-- Checking prototype gethostbyname_r for EVENT__HAVE_GETHOSTBYNAME_R_5_ARG - False
-- Checking prototype gethostbyname_r for EVENT__HAVE_GETHOSTBYNAME_R_6_ARG - True
-- Looking for select
-- Looking for select - found
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of uint8_t
-- Check size of uint8_t - done
-- Check size of uint16_t
-- Check size of uint16_t - done
-- Check size of uint32_t
-- Check size of uint32_t - done
-- Check size of uint64_t
-- Check size of uint64_t - done
-- Check size of short
-- Check size of short - done
-- Check size of int
-- Check size of int - done
-- Check size of unsigned
-- Check size of unsigned - done
-- Check size of unsigned int
-- Check size of unsigned int - done
-- Check size of long
-- Check size of long - done
-- Check size of long long
-- Check size of long long - done
-- Performing Test HAVE_INLINE
-- Performing Test HAVE_INLINE - Success
-- Looking for TAILQ_FOREACH
-- Looking for TAILQ_FOREACH - found
-- Looking for CTL_KERN
-- Looking for CTL_KERN - not found
-- Looking for KERN_ARND
-- Looking for KERN_ARND - not found
-- Looking for KERN_RANDOM
-- Looking for KERN_RANDOM - not found
-- Looking for RANDOM_UUID
-- Looking for RANDOM_UUID - not found
-- Looking for F_SETFD
-- Looking for F_SETFD - found
-- Check size of fd_mask
-- Check size of fd_mask - done
-- Check size of size_t
-- Check size of size_t - done
-- Check size of off_t
-- Check size of off_t - done
-- Check size of ssize_t
-- Check size of ssize_t - done
-- Check size of SSIZE_T
-- Check size of SSIZE_T - failed
-- Check size of socklen_t
-- Check size of socklen_t - done
-- Check size of pid_t
-- Check size of pid_t - done
-- Check size of pthread_t
-- Check size of pthread_t - done
-- Check size of uintptr_t
-- Check size of uintptr_t - done
-- Check size of void *
-- Check size of void * - done
-- Performing Test EVENT__HAVE_WAITPID_WITH_WNOWAIT
-- Performing Test EVENT__HAVE_WAITPID_WITH_WNOWAIT - Failed
-- Looking for _MINIX
-- Looking for _MINIX - not found
-- Looking for _POSIX_1_SOURCE
-- Looking for _POSIX_1_SOURCE - not found
-- Looking for _POSIX_SOURCE
-- Looking for _POSIX_SOURCE - found
-- Check size of struct addrinfo
-- Check size of struct addrinfo - done
-- Check size of struct in6_addr
-- Check size of struct in6_addr - done
-- Performing Test EVENT__HAVE_STRUCT_IN6_ADDR_S6_ADDR16
-- Performing Test EVENT__HAVE_STRUCT_IN6_ADDR_S6_ADDR16 - Success
-- Performing Test EVENT__HAVE_STRUCT_IN6_ADDR_S6_ADDR32
-- Performing Test EVENT__HAVE_STRUCT_IN6_ADDR_S6_ADDR32 - Success
-- Check size of sa_family_t
-- Check size of sa_family_t - done
-- Check size of struct sockaddr_in6
-- Check size of struct sockaddr_in6 - done
-- Performing Test EVENT__HAVE_STRUCT_SOCKADDR_IN6_SIN6_LEN
-- Performing Test EVENT__HAVE_STRUCT_SOCKADDR_IN6_SIN6_LEN - Failed
-- Performing Test EVENT__HAVE_STRUCT_SOCKADDR_IN_SIN_LEN
-- Performing Test EVENT__HAVE_STRUCT_SOCKADDR_IN_SIN_LEN - Failed
-- Check size of struct sockaddr_storage
-- Check size of struct sockaddr_storage - done
-- Performing Test EVENT__HAVE_STRUCT_SOCKADDR_STORAGE_SS_FAMILY
-- Performing Test EVENT__HAVE_STRUCT_SOCKADDR_STORAGE_SS_FAMILY - Success
-- Performing Test EVENT__HAVE_STRUCT_SOCKADDR_STORAGE___SS_FAMILY
-- Performing Test EVENT__HAVE_STRUCT_SOCKADDR_STORAGE___SS_FAMILY - Failed
CMake Error at /opt/facebook/cmake-AA6vHTxv5_KWMvgTGOWVbipzSkVjGuFLAe6ACsMRQ0E/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY
OPENSSL_INCLUDE_DIR)
Call Stack (most recent call first):
/opt/facebook/cmake-AA6vHTxv5_KWMvgTGOWVbipzSkVjGuFLAe6ACsMRQ0E/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
/opt/facebook/cmake-AA6vHTxv5_KWMvgTGOWVbipzSkVjGuFLAe6ACsMRQ0E/share/cmake-3.20/Modules/FindOpenSSL.cmake:570 (find_package_handle_standard_args)
CMakeLists.txt:683 (find_package)

-- Configuring incomplete, errors occurred!
See also "/tmp/fbcode_builder_getdeps-ZrootZmmwavesZopenrZbuildZfbcode_builder-root/build/libevent-urcq3ZGOglQ94yMJ5LETSRPR58oKUrOhHIXjzlZSloM/CMakeFiles/CMakeOutput.log".
See also "/tmp/fbcode_builder_getdeps-ZrootZmmwavesZopenrZbuildZfbcode_builder-root/build/libevent-urcq3ZGOglQ94yMJ5LETSRPR58oKUrOhHIXjzlZSloM/CMakeFiles/CMakeError.log".
Command '['/opt/facebook/cmake-AA6vHTxv5_KWMvgTGOWVbipzSkVjGuFLAe6ACsMRQ0E/bin/cmake', '/tmp/fbcode_builder_getdeps-ZrootZmmwavesZopenrZbuildZfbcode_builder-root/extracted/libevent-release-2.1.8-stable.tar.gz/libevent-release-2.1.8-stable', '-DCMAKE_INSTALL_PREFIX=/opt/facebook/libevent-urcq3ZGOglQ94yMJ5LETSRPR58oKUrOhHIXjzlZSloM', '-DBUILD_SHARED_LIBS=OFF', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DEVENT__DISABLE_TESTS=ON', '-DEVENT__DISABLE_BENCHMARK=ON', '-DEVENT__DISABLE_SAMPLES=ON', '-DEVENT__DISABLE_REGRESS=ON', '-G', 'Ninja']' returned non-zero exit status 1.
!! Failed
[ERROR]: Failed to build openr`

What's the expected result?

Should build libevent

[cooperlees]

Hi,

Would love any cmake help. Here tho, you just don't have OpenSSL libraries / headers. OPENSSL_ROOT_DIR is not set as the error says. Try install OpenSSL development packages.

If you can, it's try Ubuntu 18.04 as that is what our CI runs in. There is a known build error tho today with one dependency I have not got around to fixing. PRs welcome to make your life easy if it does not over complicate things etc. etc.

We do not test aarch64 Open/R at all or it's dependencies. So expect some potential bugs, but it should be mostly fine.

[sgiannoulis]

The real problem is i got openssl installed. But since 3.14.0 cmake is not directly available for aarch64 architectures (check https://github.com/Kitware/CMake/releases/tag/v3.14.0 ), i used 3.20.0 (edited the cmake dependency manifest file to get and build that instead, https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-aarch64.tar.gz ) which has some issues detecting correctly openssl installation. ( i have seen it in the past, newer versions of cmake need slightly different way to define how to find openssl installed libs, so the relative Cmakefiles need editing). If i use my natively compiled cmake 3.14.0 aarch64 all builds nicely. So the problem is, how can i tell your build scripts NOT to download and use the predefined cmake 3.14.00 x86 but to skip this step and use the local system installed cmake variant (13.4.0 aarch64). Is there an easy way to do that ?

[cooperlees]

I am not sure. I am not a cmake expert. Maybe we try move our x86_64 to 3.20.0 and see if that helps.

[sgiannoulis]

Its not a cmake issue, but rather if the python based facebook build architecture has a control knob to use the system cmake and not download its own, hence why i asked here if such a knob exists ( trying to avoid diving into the python based build system to figure out how it works and what the available control knobs are). In any case, i managed to make it work with downloading 3.20.00 for building the python-six dependency. For the rest i use the system cmake (3.14.0) issuing scripted build commands in order to avoid using the python based build system so i can have control over the build process.
So after solving this issue, i got openR latest RC working in aarch64 ubuntu 16.04.

[cabelitos]

Hey guys, I was trying to make this compile on my Apple M1 in order to know more about this project and I stumbled into this issue.

I realized that fbzmq does not run on os x, but fortunately I created a port for it which I submitted a pull request in here: facebookarchive/fbzmq#19

I do not know (yet) the required effort to port openr to osx on apple silicon, but I'll keep investigating. Since I'm dealing with ARM I'll also try to fix the compilation errors on linux.

I know that on real life open/r will not run on Apple hardware, but I guess this will delight me as a side project. ( But other BSD systems can benefit from it.)

Thank you.

[cooperlees]

Hi,

If the code changes are not invasive we could potentially merge this. I am keen not to introduce complexity (and thus more scope for bugs) into Open/R to make it run on BSD/Darwin as we'll never run production network devices on said operating systems.

I realized that fbzmq does not run on os x, but fortunately I created a port for it which I submitted a pull request in here: facebook/fbzmq#19

Thanks, Will get some fbzmq pros to look at this throughout the week.

I do not know (yet) the required effort to port openr to osx on apple silicon, but I'll keep investigating. Since I'm dealing with ARM I'll also try to fix the compilation errors on linux.

I know that on real life open/r will not run on Apple hardware, but I guess this will delight me as a side project. ( But other BSD systems can benefit from it.)

A "forwarding information base" FIB plugin will need to be written to talk to the BSD/Darwin routing subsystem (i.e. install the routes). In linux we do this via the netlink API. I am unsure what it is / how to drive it on BSD/Darwin.

[cabelitos]

Hi @cooperlees , thank you for the input

If the code changes are not invasive we could potentially merge this. I am keen not to introduce complexity (and thus more >scope for bugs) into Open/R to make it run on BSD/Darwin as we'll never run production network devices on said operating > >systems.

Yeah, I think that it will require some work to do this port. I don`t think it will be as small as fbzmq was.

Thanks, Will get some fbzmq pros to look at this throughout the week.

A "forwarding information base" FIB plugin will need to be written to talk to the BSD/Darwin routing subsystem (i.e. install the routes). In linux we do this via the netlink API. I am unsure what it is / how to drive it on BSD/Darwin.

I'll take a look, I sent some hours yesterday checking what kind of operations you guys do with the netlink API and if it's possible to achieve the same behavior on BSD.

Btw: I created a PR that enables to compile Open/R on ARM linux: #95

[sgiannoulis]

@cabelitos This is great, no need to manually compile and install then anymore the cmake in an arm environment and run our own scripts to make the installations with the correct aarch64 cmake version.

[cooperlees]

#95 is merged. Now just to continue making the build itself work :P