Add CI for s390x and fix big-endian arch

[wismill]

Fix for 32bits words on big-endian arch.

Fixes #97

[harendra-kumar]

We need to fix the following files as well for the same issue:

• unicode-data-scripts/test/Unicode/Char/General/ScriptsSpec.hs
• unicode-data-scripts/lib/Unicode/Char/General/Scripts.hs

We may have problems in future if we forget to include MachDeps.h , this is a nasty thing that is not caught by any tool and can be easily missed. We can do two things to mitigate this:

• define a CPP macro for reading Word32 with endianness handling and then use that macro everywhere. That way we need to include MachDeps only in the file defining that macro. Both byteswap and narrowing can be included in that macro so we do not miss it anywhere.
• Add a CI for s390x. We can copy it from unicode-transforms repo.

[harendra-kumar]

I assume byteSwap32# changes the byte ordering. The doc (https://hackage.haskell.org/package/base-4.17.0.0/docs/GHC-Exts.html#v:byteSwap32-35-) says:

Swap bytes in the lower 32 bits of a word. The higher bytes are undefined. 

I do not understand what "swap bytes" means for 4 bytes, the term "swap" is applicable only to 2 bytes.

Is there a documentation somewhere about what narrow32Word# does? Does it zero the higher 32 bits? The doc of byteSwap32# above says the "higher bytes are undefined" so perhaps that is what the bug was?

[wismill]

I did not find much information. I found some hints in cryptonite package.

My understanding is that byte swapping adapt the endianness and is not limited to 2 bytes. Here byteSwap32# is for 4 bytes, and byteSwap16# would be for 2, etc.

[wismill]

We need to fix the following files as well for the same issue:

* unicode-data-scripts/test/Unicode/Char/General/ScriptsSpec.hs

* unicode-data-scripts/lib/Unicode/Char/General/Scripts.hs

These are already fixed.

We may have problems in future if we forget to include MachDeps.h , this is a nasty thing that is not caught by any tool and can be easily missed. We can do two things to mitigate this:

* define a CPP macro for reading Word32 with endianness handling and then use that macro everywhere. That way we need to include MachDeps only in the file defining that macro. Both byteswap and narrowing can be included in that macro so we do not miss it anywhere.

* Add a CI for s390x. We can copy it from unicode-transforms repo.

I think the latter is better. I am giving a try.

[wismill]

@harendra-kumar there is an issue with the CI for GHC 8.0.2 and 8.2.2. It seems they are not supported anymore.

[harendra-kumar]

We need to fix the following files as well for the same issue:

* unicode-data-scripts/test/Unicode/Char/General/ScriptsSpec.hs

* unicode-data-scripts/lib/Unicode/Char/General/Scripts.hs

These are already fixed.

It see this in unicode-data-scripts/lib/Unicode/Char/General/Scripts.hs:

#ifdef WORDS_BIGENDIAN
#if MIN_VERSION_base(4,16,0)
            byteSwap32# (word32ToWord# (indexWord32OffAddr# addr# k#));
#else
            byteSwap32# (indexWord32OffAddr# addr# k#);
#endif

Don't we need a call to narrowWord here?

[harendra-kumar]

@harendra-kumar there is an issue with the CI for GHC 8.0.2 and 8.2.2. It seems they are not supported anymore.

The installation seems to be failing, not sure what's wrong. We can investigate if someone asks for it, otherwise let's ignore it for now.

[wismill]

@harendra-kumar After the experiment in #99, I think this is ready to merge if the relevant tests pass (let’s discard GHC ≤ 8.2). I reorganized and squashed some commits.

Keep in mind that we can fully test only against GHC 9.6. And GHC support for Unicode < 12.1 is not reliable, because there are some discrepancies (I checked manually some errors).

[harendra-kumar]

It seems you missed this #98 (comment) . Can you please respond to this.

[wismill]

It seems you missed this #98 (comment) . Can you please respond to this.

I think I did fix it. Are you using the latest version?

[wismill]

@harendra-kumar kind reminder

[harendra-kumar]

I think I did fix it. Are you using the latest version?

Looks like I missed it, maybe I did not pull it again or ignored the pull error.