32-bit compatibility (64-bit gid on 32-bit architecture) #1065

gtrensch · 2018-10-30T12:26:43Z

This pull request resolves #1031.

This pull request fixes an issue with compiling and running NEST on a 32-bit platform. It has been tested on a 32-bit ARM Raspberry Pi.

The source code changes do the following:

Let CMake identify when compiling for a 32-bit architecture and then include the appropriate headers. Even on 32-bit platforms there is usually an uint64_t available in stdint.h.
Define the gidexplicitly as 64-bit. Both, unsigned long and unsigend long long are not appropriate for this. They are platform-dependent data types!
The 64-bit gid on 32-bit required a few more changes down to sli-types, i.e., overloading constructors and ensuring that this does not conflict with data types on 64-bit platforms, i.e., uint64_t might be equivalent to unsigned long preventing overloading.

I found that it is not needed for the moment to change the types of index and thread as they are compiled from a subset of the bits still fitting 32-bit. Making this uint64_t breaks sli-types and will require more changes. Since we decided to follow the C++11 standard, I also introduced constexpr for the constant-definitions to ensure their evaluation at compile-time.

- include stdint.h on 32-bit platforms - use uint64_t instead of unsigned long and size_t to ensure 64-bit data types also on 32-bit platforms

apeyser

Very nice!

nestkernel/nest_types.h

nestkernel/source.h

nestkernel/target.h

sli/token.cc

gtrensch · 2018-11-09T13:23:58Z

@apeyser please review the refactored version of the code. For the refactored experimental boost cpptests I suggest @jakobj as a reviewer.

The 64-bit target-id layout can now be specified in a more generic way. Bit-masks and max-values are generated by const-expressions. I also removed some of the raw numbers to get rid of dangerous logical dependencies!

apeyser

Only one comment requires any response: whether cmake should just fail for 32-bit but non-ILP32 data model machines (the unlikely but technically possible case). Currently, it should compile properly rather than failing before fairly cryptically with a template error --- but maybe it should fail explicitly if someone where to try that (and maybe no one ever will).

cmake/CheckIncludesSymbols.cmake

nestkernel/target.h

gtrensch · 2018-11-12T13:34:59Z

@apeyser thanks for your comments! I have included your suggestions in the code.

sanjayankur31

The code bits look mostly OK to me. Has this been tested by CI on the different architectures? I'm testing it on the different arches Fedora supports at the moment. I'll have more to say once those builds finish.

sanjayankur31 · 2018-11-15T10:01:31Z

cmake/CheckIncludesSymbols.cmake

+    set( CMAKE_EXTRA_INCLUDE_FILES "stdint.h" )
+    check_type_size( "long" LONG_SIZE )
+    if( NOT LONG_SIZE EQUAL 4 )
+        message( FATAL_ERROR "Platform not supported: None-ILP32 data model for 32 bit architecture." )


Suggested change

message( FATAL_ERROR "Platform not supported: None-ILP32 data model for 32 bit architecture." )

message( FATAL_ERROR "Platform not supported: Non-ILP32 data model for 32 bit architecture." )

sanjayankur31 · 2018-11-15T10:02:23Z

cmake/CheckIncludesSymbols.cmake

@@ -53,6 +53,30 @@ if ( UINT16_T_SIZE GREATER 0 )
  set( HAVE_UINT16_T ON )
 endif ()

+set( CMAKE_EXTRA_INCLUDE_FILES "sys/types.h" )


I'm wondering of this some comments should be added here just to make it easier for devs in the future to understand why this is here?

This macro needs to be set up before check_type_size() and loses scope after execution of check_type_size(). This is CMake magic. I think it is clear what this part of the CMake script does. The rest is somehow CMake-technical and I tend not to begin documenting it.

sanjayankur31 · 2018-11-15T11:20:21Z

Still getting build failures on the 2 32 bit architectures supported in Fedora: i686 and armv7hl:

I'm also uploading the build logs. These are "scratch" builds and will be deleted in few hours.

armv7hl-build-log.txt
i686-build-log.txt

jakobj

thanks a lot, new code looks good. test cases look solid 👍 i just have one small question

jakobj · 2018-11-16T12:51:18Z

nestkernel/exceptions.h

+ * Exception to be thrown if an internal error occures.
+ * @ingroup KernelExceptions
+*/
+class InternalError : public KernelException


why do we need this new exception? wouldn't a KernelException by sufficient?

Are not all exceptions derived from the base class KernelException? The enumeration that I introduced in targets.h as well as the InternalError exception should make the code more robust and easier to extend - just in case that there is more than a single status bit needed in future.

sanjayankur31 · 2018-11-16T12:59:52Z

I'm afraid all the travis tests machines seem to be amd64 machines, so the tests don't really check these code changes. The code doesn't build in i686 and armv7hl for me (as commented above). Can someone please look at the attached logs to see if I've missed something there?

gtrensch · 2018-11-18T22:31:14Z

@sanjayankur31 Ich checked the logs. The "constructor ambiguity" issue was one of the problems I also went into when testing on the 32-bit ARM Raspberry. It should be resolved. One of the warnings shows:

warning: unsigned conversion from 'long long int' to 'size_t' {aka 'unsigned int'} changes value from '4611686018427387903' to '4294967295' [-Woverflow]
   static const size_t disabled_marker_ = 4611686018427387904 - 1; // 2 ** 62 - 1

I removed that raw number from the code. Can you please check whether you ran your tests on the correct branch?

sanjayankur31 · 2018-11-19T11:16:32Z

@gtrensch ah, I think I may have run the build before those commits. I'll re-run with the latest one in your branch and report back.

sanjayankur31 · 2018-11-19T13:02:19Z

The latest builds for i686 and armvhl7 completed successfully:

heplesser · 2018-11-20T08:00:16Z

@gtrensch Could you resolve the conflicts that have occurred (I assume #1062 may be at fault), then this one should be ready to be merged.

…ensch/nest-simulator into issue1031_64bitOn32bitSystem

gtrensch · 2018-11-23T12:02:12Z

The conflict is resolved.

gtrensch added 3 commits October 26, 2018 17:36

- CMake check for 32-bit platform

b39b879

- include stdint.h on 32-bit platforms - use uint64_t instead of unsigned long and size_t to ensure 64-bit data types also on 32-bit platforms

use 64-bit data type

3d5d27b

code formatting

a28b8e0

gtrensch mentioned this pull request Oct 30, 2018

'success' in 'struct nest::StaticAssert...' does not name a type #1031

Closed

apeyser suggested changes Oct 31, 2018

View reviewed changes

nestkernel/nest_types.h Outdated Show resolved Hide resolved

nestkernel/source.h Outdated Show resolved Hide resolved

nestkernel/target.h Outdated Show resolved Hide resolved

nestkernel/target.h Show resolved Hide resolved

sli/token.cc Show resolved Hide resolved

heplesser requested a review from sanjayankur31 November 3, 2018 14:39

gtrensch added 6 commits November 8, 2018 14:17

refactor Target class

ec75ab5

clang-format

b1b0408

Merge branch 'master' into issue1031_64bitOn32bitSystem

10cc0bb

fix dependecies on experimental boost cpptests

da58062

refactor experimental boost cpptests

2c79576

remove raw numbers

390b0f1

apeyser reviewed Nov 12, 2018

View reviewed changes

cmake/CheckIncludesSymbols.cmake Outdated Show resolved Hide resolved

nestkernel/target.h Show resolved Hide resolved

nestkernel/target.h Outdated Show resolved Hide resolved

terminate CMake if none-ILP32 data model

33ff50f

apeyser approved these changes Nov 13, 2018

View reviewed changes

sanjayankur31 suggested changes Nov 15, 2018

View reviewed changes

jakobj approved these changes Nov 16, 2018

View reviewed changes

sanjayankur31 approved these changes Nov 19, 2018

View reviewed changes

gtrensch added 2 commits November 22, 2018 17:55

resolve conflicts

e807bb1

Merge branch 'issue1031_64bitOn32bitSystem' of https://github.com/gtr…

9c41e52

…ensch/nest-simulator into issue1031_64bitOn32bitSystem

heplesser merged commit 6b26ff5 into nest:master Nov 26, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

32-bit compatibility (64-bit gid on 32-bit architecture) #1065

32-bit compatibility (64-bit gid on 32-bit architecture) #1065

gtrensch commented Oct 30, 2018

apeyser left a comment

gtrensch commented Nov 9, 2018

apeyser left a comment

gtrensch commented Nov 12, 2018

sanjayankur31 left a comment

sanjayankur31 Nov 15, 2018

sanjayankur31 Nov 15, 2018

gtrensch Nov 23, 2018

sanjayankur31 commented Nov 15, 2018 •

edited

jakobj left a comment

jakobj Nov 16, 2018

gtrensch Nov 23, 2018

sanjayankur31 commented Nov 16, 2018 •

edited

gtrensch commented Nov 18, 2018

sanjayankur31 commented Nov 19, 2018

sanjayankur31 commented Nov 19, 2018

heplesser commented Nov 20, 2018

gtrensch commented Nov 23, 2018

	message( FATAL_ERROR "Platform not supported: None-ILP32 data model for 32 bit architecture." )
	message( FATAL_ERROR "Platform not supported: Non-ILP32 data model for 32 bit architecture." )

32-bit compatibility (64-bit gid on 32-bit architecture) #1065

32-bit compatibility (64-bit gid on 32-bit architecture) #1065

Conversation

gtrensch commented Oct 30, 2018

apeyser left a comment

Choose a reason for hiding this comment

gtrensch commented Nov 9, 2018

apeyser left a comment

Choose a reason for hiding this comment

gtrensch commented Nov 12, 2018

sanjayankur31 left a comment

Choose a reason for hiding this comment

sanjayankur31 Nov 15, 2018

Choose a reason for hiding this comment

sanjayankur31 Nov 15, 2018

Choose a reason for hiding this comment

gtrensch Nov 23, 2018

Choose a reason for hiding this comment

sanjayankur31 commented Nov 15, 2018 • edited

jakobj left a comment

Choose a reason for hiding this comment

jakobj Nov 16, 2018

Choose a reason for hiding this comment

gtrensch Nov 23, 2018

Choose a reason for hiding this comment

sanjayankur31 commented Nov 16, 2018 • edited

gtrensch commented Nov 18, 2018

sanjayankur31 commented Nov 19, 2018

sanjayankur31 commented Nov 19, 2018

heplesser commented Nov 20, 2018

gtrensch commented Nov 23, 2018

sanjayankur31 commented Nov 15, 2018 •

edited

sanjayankur31 commented Nov 16, 2018 •

edited