FB4 build fails on big endian

[AlexPeshkoff]

This is related to #6852

For big endian ttmath library should not be used

[real-dam]

I wonder if libabsl shouldn't be used on all platforms instead of big-endian only? That would simplify the build process and guarantee equivalent behavior on little- and big-endian platforms.
(And libabsl is already available in Debian. I could provide the patch to add --with-system-absl to configure)

[AlexPeshkoff]

Pay attention to comments in mentioned pull request - it's much slower than ttmath. And that's not only due to mentioned asm optimization - even when it turned off ttmath is much faster.

[real-dam]

Yes, I just read those. Sorry for missing them earlier. Half as fast is a deal breaker indeed.

[AlexPeshkoff]

What about mentioned switch - it will not work on HW where gcc supports native __int128 due to alignment issues in FB. I've added slightly modified abseil's code.

[real-dam]

What about mentioned switch - it will not work on HW where gcc supports native __int128 due to alignment issues in FB. I've added slightly modified abseil's code.

I want to report a relative success on big-endian hardware (powerpc64) with libabsl20200923 that is available in Debian. FB4 client/isql on amd64 (little endian) talks to a server running on powerpc64 (big endian):

$ isql-fb -z
ISQL Version: LI-V4.0.0.2496 Firebird 4.0
SQL> connect to 'ppc64:test.fdb';
Server version:
LI-V4.0.0.2496 Firebird 4.0
LI-V4.0.0.2496 Firebird 4.0/tcp (ppc64)/P16:C
LI-V4.0.0.2496 Firebird 4.0/tcp (debian)/P16:C
SQL> select cast('123456789012345678901234567890123456789' as int128), cast('1234567890123456789012345678901234' as decfloat) from rdb$database;

                                         CAST                                       CAST 
============================================= ========================================== 
      123456789012345678901234567890123456789         1234567890123456789012345678901234 

This is 4.0.0 + a bunch of patches from the v4.0-release branch, including fixes for #6978 and PR#6852. Excluding extern/int128 and using libabsl20200923 from the Debian package. This is a private build for evaluating the situation and not in any way official or final.

The build went fine on s390x too, but I can't test the resulting server due to the limited access I have.

Is the above test enough to ensure compatibility with the official FB client/server?

[AlexPeshkoff]

On 10/3/21 7:09 PM, real-dam wrote: What about mentioned switch - it will not work on HW where gcc supports native __int128 due to alignment issues in FB. I've added slightly modified abseil's code. I want to report a relative success on big-endian hardware (powerpc64) with libabsl20200923 that is available in Debian. FB4 client/isql on amd64 (little endian) talks to a server running on powerpc64 (big endian): |$ isql-fb -z ISQL Version: LI-V4.0.0.2496 Firebird 4.0 SQL> connect to 'ppc64:test.fdb'; Server version: LI-V4.0.0.2496 Firebird 4.0 LI-V4.0.0.2496 Firebird 4.0/tcp (ppc64)/P16:C LI-V4.0.0.2496 Firebird 4.0/tcp (debian)/P16:C SQL> select cast('123456789012345678901234567890123456789' as int128), cast('1234567890123456789012345678901234' as decfloat) from rdb$database; CAST CAST ============================================= ========================================== 123456789012345678901234567890123456789 1234567890123456789012345678901234 | This is 4.0.0 + a bunch of patches from the v4.0-release branch, including fixes for #6978 <#6978> and PR#6852. /Excluding/ |extern/int128| and using libabsl20200923 from the Debian package. This is a private build for evaluating the situation and not in any way official or final. The build went fine on s390x too, but I can't test the resulting server due to the limited access I have. Is the above test enough to ensure compatibility with the official FB client/server?

That's enough to be sure that mapper between internal and network representation works correctly. And yes, compatibility with the official FB client/server is verified. Very interesting - does mentioned PPC have HW support for decfloats? We use IBM's emulation library and on PPC with such support is should use HW instead emulator. Byt there is one more issue - data alignment. To check it do something like this:     create table tbl1(fld1 int128);     insert into tbl1 values (12345);     select * from tbl1; In a case of alignment problems you get segfault when doing select.

[real-dam]

That's enough to be sure that mapper between internal and network representation works correctly. And yes, compatibility with the official FB client/server is verified.

Great!

Very interesting - does mentioned PPC have HW support for decfloats? We use IBM's emulation library and on PPC with such support is should use HW instead emulator.

I don't really know. How do I check? I can compile/run things if that would help.

The following program compiles and prints Size of __int128: 16 when run:

#include <stdio.h>

int main(int argc, char** argv) {
	__int128 value;
	printf("Size of __int128: %ld\n", sizeof(value));
	return 0;
}

But there is one more issue - data alignment. To check it do something like this:
create table tbl1(fld1 int128);
insert into tbl1 values (12345);
select * from tbl1;
In a case of alignment problems you get segfault when doing select.

Seems to execute normally on the build hosts (both ppc64 and s390x; real hardware; in embedded mode):

create database 'test.fdb';
create table t1(f1 int128);
insert into t1(f1) values(12345);
select * from t1;

                                           F1 
============================================= 
                                        12345 

also works when the client is amd64 (little endian) and the server is ppc64 (big endian; qemu)

[AlexPeshkoff]

On 10/6/21 8:13 AM, real-dam wrote: Very interesting - does mentioned PPC have HW support for decfloats? We use IBM's emulation library and on PPC with such support is should use HW instead emulator. I don't really know. How do I check? I can compile/run things if that would help.

Looks I had too good mind about IBM's library and C++ complers. HW support (should be present since Power6 family) is actually not used by both of them :-(

The following program compiles and prints |Size of __int128: 16| when run: |#include <stdio.h> int main(int argc, char** argv) { __int128 value; printf("Size of __int128: %ld\n", sizeof(value)); return 0; } | But there is one more issue - data alignment. To check it do something like this: create table tbl1(fld1 int128); insert into tbl1 values (12345); select * from tbl1; In a case of alignment problems you get segfault when doing select. Seems to execute normally on the build hosts (both ppc64 and s390x; real hardware; in embedded mode): |create database 'test.fdb'; create table t1(f1 int128); insert into t1(f1) values(12345); select * from t1; F1 ============================================= 12345 | also works when the client is amd64 (little endian) and the server is ppc64 (big endian; qemu)

For many years we used to have alignment issues when proting to non-intel HW. Looks like now x64 has alignment issues missing on PPC/S390. The reason is use of XMM command (http://www.club155.ru/x86cmdsimd/MOVAPS) to manipulate __int128 variables on x64 (yes, arithmeticsis emulated, but native command for moving data). Well, looks like on non-intel (at least PPC and S390) one can use intrinsic __int128. And that's definitely faster!