Initial powerpc and arm support, Gentoo crossdev support #70

grepwood · 2019-03-27T22:20:48Z

CMake/crossdev contains CMake toolchain files that work with Gentoo crossdev
some platforms lack rotl and rotr, so there are hints for the compiler to either improvise or figure out a good solution (powerpc has rlwinm which could do the task)
unk_volume from SourceX/sound.cpp has to be changed to signed char because just char is unsigned on x86 but signed on powerpc

AJenbo · 2019-03-27T22:40:57Z

Nice PR, how well are the new platform running with these changes? Is this powerpc LE or BE?

I believe char is normal signed on x86 maybe you meant that it is normally unsigned on powerpc (and arm)?

Are there any decisions in the compile process (and dependencies) when building for these targets (maybe it's worth updating the Readme)?

grepwood · 2019-03-27T23:14:39Z

Nice PR, how well are the new platform running with these changes?

Not tested just yet. More on this later.

Is this powerpc LE or BE?

All PPCs do not have a hardware based rotr like x86 does. However, the crossdev toolchain file I committed is specifically for 64-bit Big-Endian PowerPC, such as Cell Broadband Engine, Xenon or G5. Little-Endian PowerPC is often denoted as ppc64le. Being LE isn't the only difference.

I believe char is normal signed on x86 maybe you meant that it is normally unsigned on powerpc

Snap, you're totally right! Sorry, it's midnight and I'm tired from putting together stuff from Ikea all day long after work.
The array has to be defined as signed because powerpc's char (not sure about arm!) is implicitly unsigned and therefore will always fail in situations relying on the negative nature of some values in that array.

Are there any decisions in the compile process (and dependencies) when building for these targets (maybe it's worth updating the Readme)?

None. These targets run as such:

mkdir build-ppc64
pushd build-ppc64
cmake -DCMAKE_TOOLCHAIN_FILE=../CMake/crossdev/powerpc64-unknown-linux-gnu.cmake ..
make
popd
mkdir build-armv6j
pushd build-armv6j
cmake -DCMAKE_TOOLCHAIN_FILE=../CMake/crossdev/armv6j-unknown-linux-gnueabihf.cmake ..
make
popd

Provided that your crossdev toolchains have world-installed the dependencies:

media-libs/sdl2-ttf
media-libs/sdl2-mixer
dev-libs/libsodium

and their toolchain's compiler has USE="sanitize", then EVERYTHING WILL BUILD FINE :)
sanitize is required for asan support. However you may want to also have USE="threads" because =media-libs/libsdl2-2.0.9 won't build without it.

Here's the cherry on the cake:
These changes are just paving the way for my testing of the engine on Sony PlayStation 3 and Raspberry Pi 1B+, both running Gentoo. In due time I will have results.

AJenbo · 2019-03-28T20:35:31Z

The rotate functions have been a pain since the start of the project, however I do think we have gotten it down to the few that where actually in the original C code. They may eventually be removed from devilutionX, but probably not before further cleanup has happened upstream.

@pbekesky has already done some initial work on getting the game running on big endian PowerPc (running AmigaOS). The menu is working (after fixing pcx loading), but there are some issues in regards to handling the save game creation /loading.
Once that is solved I expect there will be issues with loading image graphics. The current render is very coupled with the file formats, making biendianess harder to implement, but we plan to update the render in the near future so that it is abstracted to the level of file loading.
See issue #21 for a bit more on this. (I see you have already been there :))

Arm should be in a must better state with mostly only being affected by the unsigned char issue, we even had a prototype that was able load in to town. However the code has improved dramatically since then so you might even have a playable game without much trouble.

As you might now powerpc and playstation support is highly sought-after by the project, in order to work on the modern versions of the original supported platforms. Still with the PR being untested I would like for that to happen before we think about merging the PR, but let's keep it open to track the progress for now.

grepwood · 2019-03-29T10:19:01Z

@AJenbo last night I was trying to form a CMake toolchain file for the free software PS3 SDK available here https://github.com/ps3dev/ps3toolchain
It gave me lots of headaches and this will require more work.
Even if it compiles with the SDK, we're not guaranteed it will work. GameOS works in ways that appear to me to be mysterious.

AJenbo · 2019-03-29T11:11:22Z

Ok, that's fine we can always dive in to it at later point after we get some more of the other related platforms working :)

AJenbo · 2019-05-11T11:28:12Z

The code parts are not implemented in master. The game is also fully working on Raspberry Pi 3B+. I didn't setup cross compiling but instead compiled directly on the Pi. Any chance you could test if this now works with your cross compiling setup for Pi 1B+? I think most of all I'm missing a guide for how to set up the cross compiler.

I'm also working on bringing 32bit PowerPC to a working state, so fare I have the menu mostly working and will probably start to merge in some of the needed changes for it.

AJenbo · 2019-05-12T11:44:33Z

The menu should now work on PowerPC and other Big-Endian systems.

glebm · 2019-10-03T23:11:19Z

SourceS/miniwin.h

+#	ifdef __x86__
+#		include <x86intrin.h>
+#	else
+#		define _rotr(a,b)	((a >> b) ^ (a << (32-b)))


I've removed the last usages of _rotl and _rotr in 2cc0c2c

x86intrin shouldn't be needed anymore

AJenbo · 2020-01-31T19:25:18Z

Closing this PR as the given targets are untested and I don't have a way to test them (my PowerPC is 32bit). Feel free to open the PR if you get a fully running port going.

runlevel5 · 2020-02-06T14:56:51Z

@AJenbo I would like to help out with the port in any way. I do own a RaptorCS Blackbird system running Linux ppc64le.

AJenbo · 2020-02-06T16:21:19Z

@runlevel5 excelent, my expectations is that at this point it should just build (tm), so the biggest issue at this point is finding a machine to test on. Do you need any help in doing so?

runlevel5 · 2020-02-07T06:56:31Z

@AJenbo Travis CI does offer ppc64le build option however they aren't useful for dev.

What do you think about using my box for development instead? I could set up SSH with X11 forwarding for you.

AJenbo · 2020-02-07T07:00:59Z

@runlevel5 I have previously attempted running the game via X11 forwarding but apparently this doesn't work. But I'm willing to give it ago. However, if you can just test the game an confirm that it works that would also be good enough :)

runlevel5 · 2020-02-08T00:56:28Z

@AJenbo I will get onto it and let you know later.

Feel free to drop me your ssh public key if you want me to setup you an account on my box

AJenbo · 2020-02-08T11:01:33Z

You can find users public keys here on GitHub :)
https://api.github.com/users/AJenbo/keys

runlevel5 · 2020-02-08T12:51:20Z

@AJenbo

Ran into issue with CMake failed to pick up libsodium, reported that to upstream libsodium as you suggested and they came back to me saying that file has been removed.

The compilation run fine (if disabling the multiplayer network feature) without any hiccup.

The only issue I could pick up is that Show Credits shows no text, there was an error in the console saying the font is missing

AJenbo · 2020-02-08T13:01:07Z

The reply from libsodium basically amounts to "won't fix" and then deleting it to not get any more reports of issues, so no help from them I guess. We have used the one from Facebook previously but that didn't work for some Mac users, so I'm hesitant to just switch to it.

You probably meant "Show Credits", you need to copy CharisSILB.ttf font from Packaging/resources to wherever you put the application.

Glad to see that it worked other then that, feel free to report if you find any issues while playing.

glebm · 2020-02-08T13:24:33Z

@runlevel5 Can you list all the package files? Per your error message, it finds the header and is able to extract version information, but fails to find libsodium.a

kraileth · 2020-02-08T15:54:55Z

We have used the one from Facebook previously but that didn't work for some Mac users, so I'm hesitant to just switch to it.

@AJenbo Which library are you referring to here as an alternative to libsodium? Just curious.

AJenbo · 2020-02-08T16:00:08Z

@kraileth It's just the findlibsodium.cmake helper script we are talking about. It's unclear where to find the upstream or maintained version.

MBeijer · 2020-02-08T16:36:43Z

The findlibsodium is broken on Amiga as well, but it’s easily fixed by modifying the file. It’s not part of our PR though since we got into other issues regarding the net code.

…

On Sat, 8 Feb 2020 at 17:00, Anders Jenbo ***@***.***> wrote: @kraileth <https://github.com/kraileth> It's just the findlibsodium.cmake helper script we are talking about. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#70?email_source=notifications&email_token=ACC5VBPICDRAIE2NAI6GGFDRB3JIZA5CNFSM4HB3SL72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELFU7JQ#issuecomment-583749542>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACC5VBIQD5BN5CWPS4QFFBTRB3JIZANCNFSM4HB3SL7Q> .

kraileth · 2020-02-08T16:42:17Z

@AJenbo Ah, I see. That kind of stuff is pretty annoying...

AJenbo · 2020-02-08T16:47:33Z

I have offered to maintain the script in libsodiums official repo, where can I find the needed changes for Amiga support?

MBeijer · 2020-02-08T16:54:28Z

if(UNIX OR CMAKE_SYSTEM_NAME STREQUAL "Generic") -> if(UNIX OR CMAKE_SYSTEM_NAME STREQUAL "Generic" OR CMAKE_SYSTEM_NAME STREQUAL “AmigaOS”) Previously this was only if (UNIX) which obviously wouldn’t work. I’ve added the os-type AmigaOS to the Amiga docker container and cmake toolchain.

…

On Sat, 8 Feb 2020 at 17:47, Anders Jenbo ***@***.***> wrote: I have offered to maintain the script in libsodiums official repo, where can I find the needed changes for Amiga support? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70?email_source=notifications&email_token=ACC5VBK4D2MZSJSODRERXSTRB3O2NA5CNFSM4HB3SL72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELFWEAA#issuecomment-583754240>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACC5VBJQWVBYUGNOAEWXCMDRB3O2NANCNFSM4HB3SL7Q> .

MBeijer · 2020-02-08T16:55:40Z

OR AMIGA also works and covers all Amiga based platforms!

…

On Sat, 8 Feb 2020 at 17:54, Marlon Beijer ***@***.***> wrote: if(UNIX OR CMAKE_SYSTEM_NAME STREQUAL "Generic") -> if(UNIX OR CMAKE_SYSTEM_NAME STREQUAL "Generic" OR CMAKE_SYSTEM_NAME STREQUAL “AmigaOS”) Previously this was only if (UNIX) which obviously wouldn’t work. I’ve added the os-type AmigaOS to the Amiga docker container and cmake toolchain. On Sat, 8 Feb 2020 at 17:47, Anders Jenbo ***@***.***> wrote: > I have offered to maintain the script in libsodiums official repo, where > can I find the needed changes for Amiga support? > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#70?email_source=notifications&email_token=ACC5VBK4D2MZSJSODRERXSTRB3O2NA5CNFSM4HB3SL72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELFWEAA#issuecomment-583754240>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ACC5VBJQWVBYUGNOAEWXCMDRB3O2NANCNFSM4HB3SL7Q> > . >

AJenbo · 2020-02-08T17:09:54Z

Thanks I think we will go with AMIGA

MBeijer · 2020-02-08T17:12:08Z

Yeah, that’ll probably be best! All Amiga toolchains are nix based, so that’s why this works. :)

…

On Sat, 8 Feb 2020 at 18:09, Anders Jenbo ***@***.***> wrote: Thanks I think we will go with AMIGA — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70?email_source=notifications&email_token=ACC5VBLZJQSWTX7FI2HFJRTRB3ROHA5CNFSM4HB3SL72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELFWU7Y#issuecomment-583756415>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACC5VBP2HIGN56NNMDMOUZDRB3ROHANCNFSM4HB3SL7Q> .

AJenbo · 2020-02-08T17:14:09Z

Just to clarify is it if(UNIX OR CMAKE_SYSTEM_NAME STREQUAL "Generic" OR CMAKE_SYSTEM_NAME STREQUAL “AMIGA”), or just feel free to throw a PR for this one :)

MBeijer · 2020-02-08T18:09:15Z

if(UNIX OR CMAKE_SYSTEM_NAME STREQUAL "Generic" OR AMIGA)

…

On Sat, 8 Feb 2020 at 18:14, Anders Jenbo ***@***.***> wrote: Just to clarify is it if(UNIX OR CMAKE_SYSTEM_NAME STREQUAL "Generic" OR CMAKE_SYSTEM_NAME STREQUAL “AMIGA”), or just feel free to throw a PR for this one :) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70?email_source=notifications&email_token=ACC5VBLGTUM74JNB6YDPDITRB3R6FA5CNFSM4HB3SL72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELFWYAI#issuecomment-583756801>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACC5VBLHWNSNOJQ76KANV53RB3R6FANCNFSM4HB3SL7Q> .

runlevel5 · 2020-02-09T06:21:20Z

@AJenbo I've setup an account for you on my box

ssh aj@100.68.27.243 -p 2200

Let me know if you could gain access

AJenbo · 2020-02-09T11:32:58Z

ssh: connect to host 100.68.27.243 port 2200: Network is unreachable

runlevel5 · 2020-02-10T11:30:06Z

@AJenbo Sorry I give you the wrong IP address, please try:

ssh aj@180.150.48.55 -p 2200

runlevel5 · 2020-03-09T05:51:59Z

@AJenbo a quick update from me, the v1.0.1 release is successfully compiled with libsodium for Linux ppc64le.

I could verify that LAN game works as expected

Wondering if the project should offer Linux ppc64le binary for v1.0.1 release too?

AJenbo · 2020-03-09T10:25:36Z

Oh that's good news. We still appear to have a few problems for ppc64be:
#633

And ppc32be:
#653

Could you test multiplayer with an x86 machine?

SSH also appears to be working now.

runlevel5 · 2020-03-09T13:23:28Z

@AJenbo

And ppc32be #653

Does that issue apply to ppc64 too? I could see what I could do to test if ppc64 (Big Endian) could talk to ppc64le or not.

Issue with Out of Hash Space #633

I am not a FreeBSD man so unable to help here. But if you like, I could set up FreeBSD on my box for you to poke into

Could you test multiplayer with an x86 machine?

Yes. I confirm that Linux ppc64le client could successfully connect to x86_64 host (in my case, I am using macOS build). The ppc64le can also successfully host game for x86_64 clients.

SSH also appears to be working now.

Feel free to experiment.

AJenbo · 2020-03-09T14:01:18Z

Excelent, in that case I would say that ppc64le is supported, but work still needs to be done for ppc64be.

runlevel5 · 2020-03-10T13:28:05Z

@AJenbo if you are looking into CI that supports ppc64le, please have a look at TravisCI https://blog.travis-ci.com/2019-11-12-multi-cpu-architecture-ibm-power-ibm-z

AJenbo · 2020-03-10T16:33:38Z

We already use TravisCI for Mac builds, could you make a PR that sets it up to also build for ppc64le?

runlevel5 · 2020-03-11T07:40:00Z

@AJenbo Wondering if you could point me to which command that package the build? Cheers

FYI #658

grepwood@sucs.org added 2 commits March 27, 2019 23:15

initial powerpc and arm support

f9a7d36

Gentoo crossdev toolchain files

bca9e4e

Predelnik mentioned this pull request Jul 9, 2019

64bit Compatible Saves #162

Merged

AJenbo added the build system label Sep 24, 2019

AJenbo force-pushed the master branch from 552730f to ea4358f Compare October 3, 2019 21:11

glebm reviewed Oct 3, 2019

View reviewed changes

AJenbo closed this Jan 31, 2020

Initial powerpc and arm support, Gentoo crossdev support #70

Initial powerpc and arm support, Gentoo crossdev support #70

Conversation

grepwood commented Mar 27, 2019 • edited

AJenbo commented Mar 27, 2019

grepwood commented Mar 27, 2019 • edited

AJenbo commented Mar 28, 2019 • edited

grepwood commented Mar 29, 2019

AJenbo commented Mar 29, 2019

AJenbo commented May 11, 2019

AJenbo commented May 12, 2019

glebm Oct 3, 2019 • edited

Choose a reason for hiding this comment

AJenbo commented Jan 31, 2020

runlevel5 commented Feb 6, 2020

AJenbo commented Feb 6, 2020

runlevel5 commented Feb 7, 2020

AJenbo commented Feb 7, 2020

runlevel5 commented Feb 8, 2020 • edited

AJenbo commented Feb 8, 2020

runlevel5 commented Feb 8, 2020 • edited

AJenbo commented Feb 8, 2020

glebm commented Feb 8, 2020

kraileth commented Feb 8, 2020

AJenbo commented Feb 8, 2020 • edited

MBeijer commented Feb 8, 2020 via email

kraileth commented Feb 8, 2020

AJenbo commented Feb 8, 2020

MBeijer commented Feb 8, 2020 via email

MBeijer commented Feb 8, 2020 via email

AJenbo commented Feb 8, 2020

MBeijer commented Feb 8, 2020 via email

AJenbo commented Feb 8, 2020

MBeijer commented Feb 8, 2020 via email

runlevel5 commented Feb 9, 2020 • edited

AJenbo commented Feb 9, 2020

runlevel5 commented Feb 10, 2020

runlevel5 commented Mar 9, 2020 • edited

AJenbo commented Mar 9, 2020 • edited

runlevel5 commented Mar 9, 2020 • edited

AJenbo commented Mar 9, 2020

runlevel5 commented Mar 10, 2020

AJenbo commented Mar 10, 2020

runlevel5 commented Mar 11, 2020 • edited

grepwood commented Mar 27, 2019 •

edited

grepwood commented Mar 27, 2019 •

edited

AJenbo commented Mar 28, 2019 •

edited

glebm Oct 3, 2019 •

edited

runlevel5 commented Feb 8, 2020 •

edited

runlevel5 commented Feb 8, 2020 •

edited

AJenbo commented Feb 8, 2020 •

edited

runlevel5 commented Feb 9, 2020 •

edited

runlevel5 commented Mar 9, 2020 •

edited

AJenbo commented Mar 9, 2020 •

edited

runlevel5 commented Mar 9, 2020 •

edited

runlevel5 commented Mar 11, 2020 •

edited