New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARM64 aka Aarch64: "posix/config.guess: unable to guess system type". Two solutions. #418

Closed
sanderjo opened this Issue Jul 25, 2017 · 20 comments

Comments

Projects
None yet
3 participants
@sanderjo
Contributor

sanderjo commented Jul 25, 2017

Trying to ./configure on my ARM64 aka Aarch64 I get

checking build system type... posix/config.guess: unable to guess system type

This script, last modified 2006-07-02, has failed to recognize
the operating system you are using. It is advised that you

Full output below. That posix/config.guess is from 2006 ... so 11 years old.

Quick solution: in the nzbget git directory, replace the files posix/config.guess and posix/config.sub with uptodate ones:

wget 'http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD' -O posix/config.guess
wget 'http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD' -O posix/config.sub

After that I can succesfully ./configure and make. Problem solved. So: update the files in posix directory? Shall I send a PR?

Or ... longer shot: my Linux system already provides a working config.guess, so maybe better to let configure use that instead of the stuff in the subdirector posix/ ?

Version already on my system:

$ /usr/share/misc/config.guess
aarch64-unknown-linux-gnu

... which has the same output as the new config.guess:

sander@nanopineo2:~/git/nzbget-better-config-for-arm64$ posix/config.guess
aarch64-unknown-linux-gnu

... so maybe it's possible to let ./configure use the system provided config.guess ... ?

Full output of non-working ./configure with the original config/config.*

sander@nanopineo2:~/git/nzbget-better-config-for-arm64$ ./configure
checking build system type... posix/config.guess: unable to guess system type

This script, last modified 2006-07-02, has failed to recognize
the operating system you are using. It is advised that you
download the most up to date version of the config scripts from

  http://savannah.gnu.org/cgi-bin/viewcvs/*checkout*/config/config/config.guess
and
  http://savannah.gnu.org/cgi-bin/viewcvs/*checkout*/config/config/config.sub

If the version you run (posix/config.guess) is already up to date, please
send the following data and any information you think might be
pertinent to <config-patches@gnu.org> in order to provide the needed
information to handle your system.

config.guess timestamp = 2006-07-02

uname -m = aarch64
uname -r = 4.11.8-sun50iw2
uname -s = Linux
uname -v = #4 SMP Sun Jul 9 19:19:37 CEST 2017

/usr/bin/uname -p =
/bin/uname -X     =

hostinfo               =
/bin/universe          =
/usr/bin/arch -k       =
/bin/arch              =
/usr/bin/oslevel       =
/usr/convex/getsysinfo =

UNAME_MACHINE = aarch64
UNAME_RELEASE = 4.11.8-sun50iw2
UNAME_SYSTEM  = Linux
UNAME_VERSION = #4 SMP Sun Jul 9 19:19:37 CEST 2017
configure: error: cannot guess build type; you must specify one

@hugbug hugbug added the improvement label Jul 25, 2017

@hugbug hugbug added this to the v19.x milestone Jul 25, 2017

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 25, 2017

Member

Or ... longer shot: my Linux system already provides a working config.guess, so maybe better to let configure use that instead of the stuff in the subdirector posix/ ?

The configure script is generated by autoconf. I don't think it has an option to use system config.guess, mainly because it has to support a wide variety of systems with different directory structures.

After that I can succesfully ./configure and make. Problem solved. So: update the files in posix directory? Shall I send a PR?

I'll gladly accept a PR. I can of course update the files myself. That's up to you to decide!

Member

hugbug commented Jul 25, 2017

Or ... longer shot: my Linux system already provides a working config.guess, so maybe better to let configure use that instead of the stuff in the subdirector posix/ ?

The configure script is generated by autoconf. I don't think it has an option to use system config.guess, mainly because it has to support a wide variety of systems with different directory structures.

After that I can succesfully ./configure and make. Problem solved. So: update the files in posix directory? Shall I send a PR?

I'll gladly accept a PR. I can of course update the files myself. That's up to you to decide!

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 25, 2017

Member

BTW, have you tried nzbget linux installer on that machine? It should install armhf version which should work.

Member

hugbug commented Jul 25, 2017

BTW, have you tried nzbget linux installer on that machine? It should install armhf version which should work.

@sanderjo

This comment has been minimized.

Show comment
Hide comment
@sanderjo

sanderjo Jul 25, 2017

Contributor

BTW, have you tried nzbget linux installer on that machine? It should install armhf version which should work.

Yes, and it worked, but 32-bit.

sander@nanopineo2:~/nzbget19.1$ /bin/sh nzbget-19.1-bin-linux.run
Installer for nzbget-19.1
Verifying package...
Checking system...
CPU-Architecture: armhf
sander@nanopineo2:~/nzbget19.1/nzbget$ file nzbget
nzbget: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped

Contributor

sanderjo commented Jul 25, 2017

BTW, have you tried nzbget linux installer on that machine? It should install armhf version which should work.

Yes, and it worked, but 32-bit.

sander@nanopineo2:~/nzbget19.1$ /bin/sh nzbget-19.1-bin-linux.run
Installer for nzbget-19.1
Verifying package...
Checking system...
CPU-Architecture: armhf
sander@nanopineo2:~/nzbget19.1/nzbget$ file nzbget
nzbget: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 25, 2017

Member

but 32-bit

It'd be great if you could make performance tests with both versions by measuring download speed and CPU usage.

I could add aarch64-binaries to universal installer if they (nzbget and unrar) perform better.

Member

hugbug commented Jul 25, 2017

but 32-bit

It'd be great if you could make performance tests with both versions by measuring download speed and CPU usage.

I could add aarch64-binaries to universal installer if they (nzbget and unrar) perform better.

@sanderjo

This comment has been minimized.

Show comment
Hide comment
@sanderjo

sanderjo Jul 25, 2017

Contributor

How about the other files in the posix directory? Should or must they be update with an update config.* ?

The depcomp in nzbget is old: 2000

$ cat posix/depcomp | grep -e 200 -e 201
# Copyright 1999, 2000 Free Software Foundation, Inc.

The one provided by Ubuntu Linux is 14 years more recent: 2014:

$ cat /usr/share/automake-1.15/depcomp | grep -e 200 -e 201
scriptversion=2013-05-30.07; # UTC
# Copyright (C) 1999-2014 Free Software Foundation, Inc.

And if it must be updated, how? https://www.gnu.org/software/automake/manual/html_node/Dependencies.html says ‘automake -a’ will install depcomp into your source tree for you. but that didn't do the trick for me (I do get a lot of warning and errors).

FWIW:

sander@nanopineo2:~/git/nzbget-try3$ aclocal
configure.ac:602: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2601: _AC_COMPILE_IFELSE is expanded from...
../../lib/autoconf/general.m4:2617: AC_COMPILE_IFELSE is expanded from...
configure.ac:602: the top level
sander@nanopineo2:~/git/nzbget-try3$ automake 
configure.ac:602: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2601: _AC_COMPILE_IFELSE is expanded from...
../../lib/autoconf/general.m4:2617: AC_COMPILE_IFELSE is expanded from...
configure.ac:602: the top level
Makefile.am:22: warning: source file 'daemon/connect/Connection.cpp' is in a subdirectory,
Makefile.am:22: but option 'subdir-objects' is disabled
automake: warning: possible forward-incompatibility.
automake: At least a source file is in a subdirectory, but the 'subdir-objects'
automake: automake option hasn't been enabled.  For now, the corresponding output
automake: object file(s) will be placed in the top-level directory.  However,
automake: this behaviour will change in future Automake versions: they will
automake: unconditionally cause object files to be placed in the same subdirectory
automake: of the corresponding sources.
automake: You are advised to start using 'subdir-objects' option throughout your
automake: project, to avoid future incompatibilities.
Makefile.am:22: warning: source file 'daemon/connect/TlsSocket.cpp' is in a subdirectory,
Makefile.am:22: but option 'subdir-objects' is disabled
Makefile.am:22: warning: source file 'daemon/connect/WebDownloader.cpp' is in a subdirectory,
Makefile.am:22: but option 'subdir-objects' is disabled
Makefile.am:22: warning: source file 'daemon/extension/FeedScript.cpp' is in a subdirectory,
Makefile.am:22: but option 'subdir-objects' is disabled
Makefile.am:22: warning: source file 'daemon/extension/CommandScript.cpp' is in a subdirectory,
Makefile.am:22: but option 'subdir-objects' is disabled
Makefile.am:22: warning: source file 'daemon/extension/NzbScript.cpp' is in a subdirectory,

Contributor

sanderjo commented Jul 25, 2017

How about the other files in the posix directory? Should or must they be update with an update config.* ?

The depcomp in nzbget is old: 2000

$ cat posix/depcomp | grep -e 200 -e 201
# Copyright 1999, 2000 Free Software Foundation, Inc.

The one provided by Ubuntu Linux is 14 years more recent: 2014:

$ cat /usr/share/automake-1.15/depcomp | grep -e 200 -e 201
scriptversion=2013-05-30.07; # UTC
# Copyright (C) 1999-2014 Free Software Foundation, Inc.

And if it must be updated, how? https://www.gnu.org/software/automake/manual/html_node/Dependencies.html says ‘automake -a’ will install depcomp into your source tree for you. but that didn't do the trick for me (I do get a lot of warning and errors).

FWIW:

sander@nanopineo2:~/git/nzbget-try3$ aclocal
configure.ac:602: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2601: _AC_COMPILE_IFELSE is expanded from...
../../lib/autoconf/general.m4:2617: AC_COMPILE_IFELSE is expanded from...
configure.ac:602: the top level
sander@nanopineo2:~/git/nzbget-try3$ automake 
configure.ac:602: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2601: _AC_COMPILE_IFELSE is expanded from...
../../lib/autoconf/general.m4:2617: AC_COMPILE_IFELSE is expanded from...
configure.ac:602: the top level
Makefile.am:22: warning: source file 'daemon/connect/Connection.cpp' is in a subdirectory,
Makefile.am:22: but option 'subdir-objects' is disabled
automake: warning: possible forward-incompatibility.
automake: At least a source file is in a subdirectory, but the 'subdir-objects'
automake: automake option hasn't been enabled.  For now, the corresponding output
automake: object file(s) will be placed in the top-level directory.  However,
automake: this behaviour will change in future Automake versions: they will
automake: unconditionally cause object files to be placed in the same subdirectory
automake: of the corresponding sources.
automake: You are advised to start using 'subdir-objects' option throughout your
automake: project, to avoid future incompatibilities.
Makefile.am:22: warning: source file 'daemon/connect/TlsSocket.cpp' is in a subdirectory,
Makefile.am:22: but option 'subdir-objects' is disabled
Makefile.am:22: warning: source file 'daemon/connect/WebDownloader.cpp' is in a subdirectory,
Makefile.am:22: but option 'subdir-objects' is disabled
Makefile.am:22: warning: source file 'daemon/extension/FeedScript.cpp' is in a subdirectory,
Makefile.am:22: but option 'subdir-objects' is disabled
Makefile.am:22: warning: source file 'daemon/extension/CommandScript.cpp' is in a subdirectory,
Makefile.am:22: but option 'subdir-objects' is disabled
Makefile.am:22: warning: source file 'daemon/extension/NzbScript.cpp' is in a subdirectory,

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 25, 2017

Member

I think the errors are caused by incompatibility between your (newer) version of autotools and the version used in nzbget. When I upgraded my build VM I had a similar issue. Instead of fighting with autotools I installed older (compatible) versions: autoconf 2.61 and automake 1.9.6 (they were in system repository too, probably a common issue).

Maybe just update depcomp manually?

Member

hugbug commented Jul 25, 2017

I think the errors are caused by incompatibility between your (newer) version of autotools and the version used in nzbget. When I upgraded my build VM I had a similar issue. Instead of fighting with autotools I installed older (compatible) versions: autoconf 2.61 and automake 1.9.6 (they were in system repository too, probably a common issue).

Maybe just update depcomp manually?

@sanderjo

This comment has been minimized.

Show comment
Hide comment
@sanderjo

sanderjo Jul 26, 2017

Contributor

Just a few notes:

  • warnings with 'subdir-objects' are gone by Adding "AUTOMAKE_OPTIONS = subdir-objects" to bottom of Makefile.am
  • depcomp will indeed be automatically installed (well: symlinked) by automake -a ... but only if depcomp is not there.
  • https://autotools.io/forwardporting/autoconf.html says something about the remaining warning
sander@nanopineo2:~/git/nzbget-better-config-for-arm64$ rm posix/depcomp
sander@nanopineo2:~/git/nzbget-better-config-for-arm64$ automake -a
configure.ac:602: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2601: _AC_COMPILE_IFELSE is expanded from...
../../lib/autoconf/general.m4:2617: AC_COMPILE_IFELSE is expanded from...
configure.ac:602: the top level
Makefile.am: installing 'posix/depcomp'
sander@nanopineo2:~/git/nzbget-better-config-for-arm64$ ll posix/depcomp
lrwxrwxrwx 1 sander sander 32 jul 26 05:54 posix/depcomp -> /usr/share/automake-1.15/depcomp*
Contributor

sanderjo commented Jul 26, 2017

Just a few notes:

  • warnings with 'subdir-objects' are gone by Adding "AUTOMAKE_OPTIONS = subdir-objects" to bottom of Makefile.am
  • depcomp will indeed be automatically installed (well: symlinked) by automake -a ... but only if depcomp is not there.
  • https://autotools.io/forwardporting/autoconf.html says something about the remaining warning
sander@nanopineo2:~/git/nzbget-better-config-for-arm64$ rm posix/depcomp
sander@nanopineo2:~/git/nzbget-better-config-for-arm64$ automake -a
configure.ac:602: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from...
../../lib/autoconf/general.m4:2601: _AC_COMPILE_IFELSE is expanded from...
../../lib/autoconf/general.m4:2617: AC_COMPILE_IFELSE is expanded from...
configure.ac:602: the top level
Makefile.am: installing 'posix/depcomp'
sander@nanopineo2:~/git/nzbget-better-config-for-arm64$ ll posix/depcomp
lrwxrwxrwx 1 sander sander 32 jul 26 05:54 posix/depcomp -> /usr/share/automake-1.15/depcomp*
@sanderjo

This comment has been minimized.

Show comment
Hide comment
@sanderjo

sanderjo Jul 26, 2017

Contributor

Update: remaing warnings warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body gone with help of https://lists.gnu.org/archive/html/bug-autoconf/2010-09/msg00091.html with the below two-line change to configure.ac.
@hugbug ... your feedback please ...

dnl 
dnl variadic macros
dnl 
AC_MSG_CHECKING(for variadic macros)
#AC_COMPILE_IFELSE([
AC_COMPILE_IFELSE([AC_LANG_SOURCE([
        #define macro(...)   macrofunc(__VA_ARGS__)
        int macrofunc(int a, int b) { return a + b; }
        int test() { return macro(1, 2); }
        ],
        AC_MSG_RESULT([yes])
        AC_DEFINE([HAVE_VARIADIC_MACROS], 1, Define to 1 if variadic macros are supported),
        AC_MSG_RESULT([no]))
])
$ diff configure.ac.ORG configure.ac
1c1
< #
---
>  #
602c602,603
< AC_COMPILE_IFELSE([
---
> #AC_COMPILE_IFELSE([
> AC_COMPILE_IFELSE([AC_LANG_SOURCE([
610c611
< 
---
> ])
Contributor

sanderjo commented Jul 26, 2017

Update: remaing warnings warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body gone with help of https://lists.gnu.org/archive/html/bug-autoconf/2010-09/msg00091.html with the below two-line change to configure.ac.
@hugbug ... your feedback please ...

dnl 
dnl variadic macros
dnl 
AC_MSG_CHECKING(for variadic macros)
#AC_COMPILE_IFELSE([
AC_COMPILE_IFELSE([AC_LANG_SOURCE([
        #define macro(...)   macrofunc(__VA_ARGS__)
        int macrofunc(int a, int b) { return a + b; }
        int test() { return macro(1, 2); }
        ],
        AC_MSG_RESULT([yes])
        AC_DEFINE([HAVE_VARIADIC_MACROS], 1, Define to 1 if variadic macros are supported),
        AC_MSG_RESULT([no]))
])
$ diff configure.ac.ORG configure.ac
1c1
< #
---
>  #
602c602,603
< AC_COMPILE_IFELSE([
---
> #AC_COMPILE_IFELSE([
> AC_COMPILE_IFELSE([AC_LANG_SOURCE([
610c611
< 
---
> ])
@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 26, 2017

Member

Thank you very much!

I did all the changes as you described and can now use the newer autotools. They are not as new as yours but still much newer than the old. I'm using Linux Mint 17 which is based on Ubuntu 14.04 LTS.

Command automake -a --force-missing --copy updates all files in posix-directory.

subdir-objects introduces one issue though. Now the source tree is populated with object files and dependency tracking files. Previously all dependency files were put into directory ".deps" on project level. Object files were put directly on project level (not the best but at least all files in one place).

Now I have to look if all these files not disturb the daily work (search in text editor etc.). One possible solution is to build in a subdirectory:

mkdir _build && cd _build
../configure && make

But in that case the binary (nzbget) is put into the build-directory and the tests will not work (nzbget --tests).
This is solvable, I just have to try what solution works best for me.


You did a great research on the topic and it would be fair if the commit comes from you. To avoid possible compatibility issues with my system I need the new files to be from my autotools version though. I could send you my files and you'll create a PR using them, OK? Please send me an email to nzbget@gmail.com.

If you don't care about your contribution history I can of course commit myself.

Member

hugbug commented Jul 26, 2017

Thank you very much!

I did all the changes as you described and can now use the newer autotools. They are not as new as yours but still much newer than the old. I'm using Linux Mint 17 which is based on Ubuntu 14.04 LTS.

Command automake -a --force-missing --copy updates all files in posix-directory.

subdir-objects introduces one issue though. Now the source tree is populated with object files and dependency tracking files. Previously all dependency files were put into directory ".deps" on project level. Object files were put directly on project level (not the best but at least all files in one place).

Now I have to look if all these files not disturb the daily work (search in text editor etc.). One possible solution is to build in a subdirectory:

mkdir _build && cd _build
../configure && make

But in that case the binary (nzbget) is put into the build-directory and the tests will not work (nzbget --tests).
This is solvable, I just have to try what solution works best for me.


You did a great research on the topic and it would be fair if the commit comes from you. To avoid possible compatibility issues with my system I need the new files to be from my autotools version though. I could send you my files and you'll create a PR using them, OK? Please send me an email to nzbget@gmail.com.

If you don't care about your contribution history I can of course commit myself.

@sanderjo

This comment has been minimized.

Show comment
Hide comment
@sanderjo

sanderjo Jul 26, 2017

Contributor

You did a great research on the topic

Just a bit of trial-and-error. ;-)

and it would be fair if the commit comes from you.

I appreciate that. But ...

I can of course commit myself.

... let's do that. Feels safer than my hacked together stuff.

Contributor

sanderjo commented Jul 26, 2017

You did a great research on the topic

Just a bit of trial-and-error. ;-)

and it would be fair if the commit comes from you.

I appreciate that. But ...

I can of course commit myself.

... let's do that. Feels safer than my hacked together stuff.

@sanderjo

This comment has been minimized.

Show comment
Hide comment
@sanderjo

sanderjo Jul 26, 2017

Contributor

A note about speed:

The ARM64 device has a SD-card, a external harddisk via USB2, and GigE.
I tested against a real, external newsserver (so not yet Nserv)

  1. With all folders on the SD-card: nzbget-32bit: Avg. 2.85 MB/s. nzbget-64bit: Avg. 2.68 MB/s. So 64-bit a bit ... slower
  2. With all folders on the external harddisk: Around 8 MB/s, with 64-bit also a bit slower.

So: the storage is the bottleneck, and 64bit is slower, which I cannot explain, because CPU-load was low.

How is the speed on x86 32bit versus 64-bit?

Update (2 oct 2017):

On my SD-card, with 64-bit, with 1 server, post of 1 day old, I now get "Avg. 10.8 MB/s".
With 32-bit, it is a bit slower: "Avg. 10.7 MB/s".
So that is quite something different than the earlier 2.8 MB/s ...
FYI: the device has a GigE-interface. my FttH linespeed is 150/150 Mbps, which is proved with another test via fast.com "Speed test [Mbps]: 150.0"

Contributor

sanderjo commented Jul 26, 2017

A note about speed:

The ARM64 device has a SD-card, a external harddisk via USB2, and GigE.
I tested against a real, external newsserver (so not yet Nserv)

  1. With all folders on the SD-card: nzbget-32bit: Avg. 2.85 MB/s. nzbget-64bit: Avg. 2.68 MB/s. So 64-bit a bit ... slower
  2. With all folders on the external harddisk: Around 8 MB/s, with 64-bit also a bit slower.

So: the storage is the bottleneck, and 64bit is slower, which I cannot explain, because CPU-load was low.

How is the speed on x86 32bit versus 64-bit?

Update (2 oct 2017):

On my SD-card, with 64-bit, with 1 server, post of 1 day old, I now get "Avg. 10.8 MB/s".
With 32-bit, it is a bit slower: "Avg. 10.7 MB/s".
So that is quite something different than the earlier 2.8 MB/s ...
FYI: the device has a GigE-interface. my FttH linespeed is 150/150 Mbps, which is proved with another test via fast.com "Speed test [Mbps]: 150.0"

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 26, 2017

Member

the storage is the bottleneck, and 64bit is slower, which I cannot explain, because CPU-load was low.

In that case you compare not only 32bit vs 64bit. The two versions were build using different compilers versions and use different standard libraries.

Can you also test the aarch64 version I've built using the same compiler (as armhf version from installer)? It's not a complete installer, you have to take the binary from archive and put it into your existing installation.

There is still a difference between aarch64 and armhf builds. For all architectures in lunux installer I use uclibc standard library. It however didn't work with aarch64 toolchain. For aarch64 I use musl standard library. When you compile directly on machine you probably have glibc. I don't know how much impact libraries have on NZBGet performance.

How is the speed on x86 32bit versus 64-bit?

I can't make such a test on Windows (only 32 bit binary) or Mac (only 64 bit binary). I have both 32bit and 64bit binaries for Linux only but I don't have a real x86 machine with Linux and making tests in a VM isn't very precise.


Even if using the same compiler and same standard library - you can never tell which one of 32bit binary or 64bit binary is faster without benchmarking.

64bit program can compute 64bit numbers faster but not every program works with 64bit numbers. NZBGet uses 64bit only for file sizes. The download code (yEnc, CRC computing) doesn't user 64bit numbers. Therefore not that much gain to expect.

On the other side 64bit code takes more space in CPU cache which is very limited and that can have a negative impact on performance.

Member

hugbug commented Jul 26, 2017

the storage is the bottleneck, and 64bit is slower, which I cannot explain, because CPU-load was low.

In that case you compare not only 32bit vs 64bit. The two versions were build using different compilers versions and use different standard libraries.

Can you also test the aarch64 version I've built using the same compiler (as armhf version from installer)? It's not a complete installer, you have to take the binary from archive and put it into your existing installation.

There is still a difference between aarch64 and armhf builds. For all architectures in lunux installer I use uclibc standard library. It however didn't work with aarch64 toolchain. For aarch64 I use musl standard library. When you compile directly on machine you probably have glibc. I don't know how much impact libraries have on NZBGet performance.

How is the speed on x86 32bit versus 64-bit?

I can't make such a test on Windows (only 32 bit binary) or Mac (only 64 bit binary). I have both 32bit and 64bit binaries for Linux only but I don't have a real x86 machine with Linux and making tests in a VM isn't very precise.


Even if using the same compiler and same standard library - you can never tell which one of 32bit binary or 64bit binary is faster without benchmarking.

64bit program can compute 64bit numbers faster but not every program works with 64bit numbers. NZBGet uses 64bit only for file sizes. The download code (yEnc, CRC computing) doesn't user 64bit numbers. Therefore not that much gain to expect.

On the other side 64bit code takes more space in CPU cache which is very limited and that can have a negative impact on performance.

hugbug added a commit that referenced this issue Jul 26, 2017

#418: updated POSIX build files to newer autotools version
- compatibility with newer autotools;
- compatibility with newer platforms such as aarch64.

@hugbug hugbug modified the milestones: v19.x, v20 Jul 26, 2017

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Jul 26, 2017

Member

Updated POSIX build files.

I've used the most actual versions of autoconf (2.69) and automake (1.15).

The nicest thing is I now can regenerate configure and makefile directly on Mac and don't need to fire up a Linux VM just to make small config change (like increasing nzbget version number or adding a new file into project).

That's so cool 👍. Thanks again.

Maybe you could test how build works for you.

Member

hugbug commented Jul 26, 2017

Updated POSIX build files.

I've used the most actual versions of autoconf (2.69) and automake (1.15).

The nicest thing is I now can regenerate configure and makefile directly on Mac and don't need to fire up a Linux VM just to make small config change (like increasing nzbget version number or adding a new file into project).

That's so cool 👍. Thanks again.

Maybe you could test how build works for you.

@sanderjo

This comment has been minimized.

Show comment
Hide comment
@sanderjo

sanderjo Jul 28, 2017

Contributor

Solved with 2124a88

Result: on ARM64 / Aarch64 I can now configure, make and run nzbget.

Thank you, @hugbug

Contributor

sanderjo commented Jul 28, 2017

Solved with 2124a88

Result: on ARM64 / Aarch64 I can now configure, make and run nzbget.

Thank you, @hugbug

@sanderjo sanderjo closed this Jul 28, 2017

hugbug added a commit that referenced this issue Jul 31, 2017

@sanderjo

This comment has been minimized.

Show comment
Hide comment
@sanderjo

sanderjo Aug 28, 2017

Contributor

Hi @hugbug, a question:

I downloaded nzbget-20.0-testing-r2075 (Release Date: 26 August 2017) and ran it on my ARM64 / Aarch64, and/but the resulting nzbget is 32-bit.

sander@nanopineo2:~/bla2$ uname -m
aarch64

and/but

sander@nanopineo2:~/bla2$ /bin/sh nzbget-20.0-testing-r2075-bin-linux.run
Installer for nzbget-20.0-testing-r2075
Verifying package...
Checking system...
CPU-Architecture: armhf

and

sander@nanopineo2:~/bla2$ file nzbget/nzbget
nzbget/nzbget: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped

I had expected the 64-bit version? Or is this a bridge too far for the moment?

Contributor

sanderjo commented Aug 28, 2017

Hi @hugbug, a question:

I downloaded nzbget-20.0-testing-r2075 (Release Date: 26 August 2017) and ran it on my ARM64 / Aarch64, and/but the resulting nzbget is 32-bit.

sander@nanopineo2:~/bla2$ uname -m
aarch64

and/but

sander@nanopineo2:~/bla2$ /bin/sh nzbget-20.0-testing-r2075-bin-linux.run
Installer for nzbget-20.0-testing-r2075
Verifying package...
Checking system...
CPU-Architecture: armhf

and

sander@nanopineo2:~/bla2$ file nzbget/nzbget
nzbget/nzbget: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped

I had expected the 64-bit version? Or is this a bridge too far for the moment?

@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Aug 28, 2017

Member

Universal installer doesn't include binaries for aarch64. My attempts to build working binaries for aarch64 failed so far; as you know the binaries don't work properly. Since armhf-binaries work just fine on aarch64 there is no pressure at the moment to spend more time on this.

Member

hugbug commented Aug 28, 2017

Universal installer doesn't include binaries for aarch64. My attempts to build working binaries for aarch64 failed so far; as you know the binaries don't work properly. Since armhf-binaries work just fine on aarch64 there is no pressure at the moment to spend more time on this.

hugbug added a commit that referenced this issue Oct 9, 2017

#418: updated POSIX build files to newer autotools version
- compatibility with newer autotools;
- compatibility with newer platforms such as aarch64.

hugbug added a commit that referenced this issue Oct 9, 2017

@ggtools

This comment has been minimized.

Show comment
Hide comment
@ggtools

ggtools Nov 22, 2017

Strange, I just installed a Rpi3 with Arch Linux in aarch 64 and the armhf-binaries are definitely not working:

$ file nzbget
nzbget: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped
$ ./nzbget
bash: ./nzbget: cannot execute binary file: Exec format error
$ uname -a
Linux rpi-02 4.13.10-1-ARCH #1 SMP Fri Oct 27 19:00:27 MDT 2017 aarch64 GNU/Linux

ggtools commented Nov 22, 2017

Strange, I just installed a Rpi3 with Arch Linux in aarch 64 and the armhf-binaries are definitely not working:

$ file nzbget
nzbget: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, stripped
$ ./nzbget
bash: ./nzbget: cannot execute binary file: Exec format error
$ uname -a
Linux rpi-02 4.13.10-1-ARCH #1 SMP Fri Oct 27 19:00:27 MDT 2017 aarch64 GNU/Linux
@sanderjo

This comment has been minimized.

Show comment
Hide comment
@sanderjo

sanderjo Nov 22, 2017

Contributor

(Case was closed, but anyway)

Which nzbget version?
Where did you get it?
What is the md5 of it?

For nzbget-20.0-testing-r2159-bin-linux.run I get

$ md5sum nzbget
45364cf91ee7099ef286acc9651f6c56  nzbget

$ file nzbget
nzbget: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 2.6.32, stripped

Contributor

sanderjo commented Nov 22, 2017

(Case was closed, but anyway)

Which nzbget version?
Where did you get it?
What is the md5 of it?

For nzbget-20.0-testing-r2159-bin-linux.run I get

$ md5sum nzbget
45364cf91ee7099ef286acc9651f6c56  nzbget

$ file nzbget
nzbget: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 2.6.32, stripped

@ggtools

This comment has been minimized.

Show comment
Hide comment
@ggtools

ggtools Nov 22, 2017

Yes case is closed just giving my two cents

  • nzbget version: 19.1
  • get it through curl -O -L https://github.com/nzbget/nzbget/releases/download/v19.1/nzbget-19.1-bin-linux.run about an hour ago
  • md5:
    • nzbget: 6eeb462e9125d41c9bad17619c883b0b
    • nzbget-19.1-bin-linux.run: 768e9ddf3e5ac83a150f23b9f251c5db

I just tried with the testing version:

$ md5sum nzbget
45364cf91ee7099ef286acc9651f6c56  nzbget
$ file nzbget
nzbget: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 2.6.32, stripped
$ ./nzbget
bash: ./nzbget: cannot execute binary file: Exec format error

ggtools commented Nov 22, 2017

Yes case is closed just giving my two cents

  • nzbget version: 19.1
  • get it through curl -O -L https://github.com/nzbget/nzbget/releases/download/v19.1/nzbget-19.1-bin-linux.run about an hour ago
  • md5:
    • nzbget: 6eeb462e9125d41c9bad17619c883b0b
    • nzbget-19.1-bin-linux.run: 768e9ddf3e5ac83a150f23b9f251c5db

I just tried with the testing version:

$ md5sum nzbget
45364cf91ee7099ef286acc9651f6c56  nzbget
$ file nzbget
nzbget: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 2.6.32, stripped
$ ./nzbget
bash: ./nzbget: cannot execute binary file: Exec format error
@hugbug

This comment has been minimized.

Show comment
Hide comment
@hugbug

hugbug Nov 22, 2017

Member

@ggtools, the original issue was about compiling. Since you use installer that has nothing to do with the original issue. I would split the topic but unfortunately GitHub doesn't provide such function. Please create a separate issue for easier tracking.

Member

hugbug commented Nov 22, 2017

@ggtools, the original issue was about compiling. Since you use installer that has nothing to do with the original issue. I would split the topic but unfortunately GitHub doesn't provide such function. Please create a separate issue for easier tracking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment