Nintendo switch support #8069

jyapayne · 2018-06-19T08:32:49Z

The long-awaited Nintendo Switch support! This allows compilation of Nintendo Switch architecture compatible elf files using the --os:nintendoswitch flag.

Environment vars required: DEVKITPRO which is the path to the root of your devkit pro installation. There are some directories expected to exist in a specific structure for now until I figure out a better way to specify them. They are:

DEVKITPRO/portlibs/switch/lib
DEVKITPRO/libnx/lib
DEVKITPRO/portlibs/switch/include
DEVKITPRO/libnx/include

Other environment variables that are not required: SWITCH_LIBS for any extra libraries required by your application (-lLIBNAME or -LLIBPATH), and SWITCH_INCLUDES for any extra include files (-IINCLUDE_PATH)

dom96 · 2018-06-19T09:38:48Z

Well, now I just have to buy a Nintendo Switch :P

Nice work!

dom96

I'd love to have this in the compiler. Not sure what @Araq's thoughts about it are, but this is awesome :)

dom96 · 2018-06-19T09:40:50Z

changelog.md

@@ -170,4 +170,15 @@
 - ``experimental`` is now a pragma / command line switch that can enable specific
  language extensions, it is not an all-or-nothing switch anymore.

+- Nintendo Switch was added as a new platform target. Simply add --os:nintendoswitch
+  to your usual ``nim c`` or ``nim cpp`` command. DevkitPro setup must be the same as


This is great! But I think a better place for it is in the nimc docs: https://github.com/nim-lang/Nim/blob/devel/doc/nimc.rst

You can link to them from the changelog :)

Cool, sounds good. I'll make that change.

dom96 · 2018-06-19T09:42:44Z

compiler/extccomp.nim

@@ -83,6 +83,31 @@ compiler gcc:
    props: {hasSwitchRange, hasComputedGoto, hasCpp, hasGcGuard, hasGnuAsm,
            hasAttribute})

+# GNU C and C++ Compiler
+compiler aarch64NoneElfGCC:


Nitpick but I would name this switchGCC. We might have a generic aarch64-none-elf-gcc in the future (I guess?)

dom96 · 2018-06-19T09:44:28Z

compiler/installer.ini

@@ -15,6 +15,7 @@ Platforms: """
  dragonfly: i386;amd64
  haiku: i386;amd64
  android: i386;arm;arm64
+  nintendoswitch: armv8a57


I think this is only necessary if you are going to be compiling Nim on the Nintendo Switch. I'm guessing that won't happen, will it?

Well, you never know. Linux can run on the switch already :)

dom96 · 2018-06-19T09:45:18Z

compiler/options.nim

@@ -124,7 +124,7 @@ type
    disabledSf, writeOnlySf, readOnlySf, v2Sf

  TSystemCC* = enum
-    ccNone, ccGcc, ccLLVM_Gcc, ccCLang, ccLcc, ccBcc, ccDmc, ccWcc, ccVcc,
+    ccNone, ccGcc, ccaarch64NoneElfGcc, ccLLVM_Gcc, ccCLang, ccLcc, ccBcc, ccDmc, ccWcc, ccVcc,


Again I'd probably call this ccSwitch or ccNintendoSwitch.

Alright, that makes more sense. I wasn't sure what to call it, so ccNintendoSwitch shall be it's name!

dom96 · 2018-06-19T09:47:18Z

compiler/platform.nim

@@ -208,7 +213,8 @@ const
    (name: "sparc64", intSize: 64, endian: bigEndian, floatSize: 64, bit: 64),
    (name: "mips64", intSize: 64, endian: bigEndian, floatSize: 64, bit: 64),
    (name: "mips64el", intSize: 64, endian: littleEndian, floatSize: 64, bit: 64),
-    (name: "riscv64", intSize: 64, endian: littleEndian, floatSize: 64, bit: 64)]
+    (name: "riscv64", intSize: 64, endian: littleEndian, floatSize: 64, bit: 64),
+    (name: "armv8a57", intSize: 64, endian: bigEndian, floatSize: 64, bit: 64)]


Can't arm64 be reused? Is there a specific reason you need bigEndian?

Yep, you're right. Checked the switch's cpu mode and it can be either.

dom96 · 2018-06-19T09:50:23Z

lib/system/threads.nim

+      Pthread_attr {.importc: "pthread_attr_t",
+                    header: "<sys/types.h>".} = object
+      ThreadVarSlot {.importc: "pthread_key_t",
+                    header: "<sys/types.h>".} = distinct cuint


This code is very similar to the code above it, can the above code be reused for this branch?

Yeah, you're right. I just wasn't sure about the abi: array[56 div sizeof(clong), clong], but it looks like it's fine.

jyapayne · 2018-06-19T10:38:58Z

@dom96 Cool to see you're on board. It's super cool to have a program running Nim on the switch!

yglukhov · 2018-06-19T11:33:56Z

lib/pure/selectors.nim

@@ -312,6 +312,8 @@ else:
    include ioselects/ioselectors_poll # need to replace it with event ports
  elif defined(genode):
    include ioselects/ioselectors_select # TODO: use the native VFS layer
+  elif defined(nintendoswitch):
+    include ioselects/ioselectors_select


doesn't the os have something better than select? kqueue, epoll?

Unfortunately, not that I can find. If you can find it in the devkitPro libs, I would be up for changing it.

jyapayne · 2018-06-21T02:13:53Z

@Araq, what are your thoughts on this pull request? Any feedback?
@dom96 I've made the requested changes. If there's anything else let me know!

Araq · 2018-06-21T15:25:55Z

I like it very much but travis is not happy:

�[0m�[1mlib/pure/times.nim(794, 29) �[0m�[31mError: �[0mundeclared identifier: 'CTime'
�[0mError: call to nim compiler failed
�[31mError: �[0mexecution of an external program failed: './tools/niminst/niminst --var:version=0.18.1 --var:mingw=none csource --main:compiler/nim.nim compiler/installer.ini'

jyapayne · 2018-06-21T21:11:08Z

@Araq, is that due to my code? I didn't mess with the times module. I'll check it out.

jyapayne · 2018-06-21T22:27:09Z

Yep, it's due to my stuff. Looks like I have more stuff to implement!

jyapayne · 2018-06-22T01:56:20Z

lib/posix/posix_nintendoswitch.nim

+           final, pure.} = object
+    abi: array[((64+(sizeof(clong) * 8)-1) div (sizeof(clong) * 8)), clong]
+
+proc si_pid*(info: SigInfo): Pid =


So, this is to get around the issue that Switch's header struct for siginfo doesn't contain si_pid, which means waiting for a child process to finish doesn't work. I don't think that's too big of a deal right now, but what are your thoughts on this? Is there some other way to correctly identify where the Signal is coming from?

I don't know, sorry.

jyapayne · 2018-06-22T23:13:26Z

Well I'm okay with the way this is now if you are. I can add a note in the docs about how waiting on subprocesses isn't supported yet and then I think it's done if it's okay with you guys.

jyapayne · 2018-06-22T23:20:57Z

And if you have no comments on my most recent commits of course :)

GULPF · 2018-06-23T06:29:56Z

I don't know anything about this, but if nintendo switch is supported by posix.nim, shouldn't defined(nintendoswitch) imply defined(posix)? Otherwise defined(posix) or defined(nintendoswitch) must be used everywhere, which is a bit annoying.

jyapayne · 2018-06-23T07:01:50Z

@GULPF Yeah, you're right. I should have realized that. Nice find!

jyapayne · 2018-06-23T22:37:38Z

@dom96 looks like the build failed, but I don't think it was me this time. It looks like yarn wasn't able to be found or something? Does this happen sometimes?

jyapayne · 2018-06-24T05:39:09Z

Nevermind, I think it had to do with some old code. I updated from latest devel and ran the tests locally and it all looks good! Hopefully travis agrees :)

jyapayne · 2018-06-26T06:34:34Z

Okay, I think I'm happy with this now. If there are any other outstanding issues, please let me know!

Araq · 2018-06-26T16:06:40Z

compiler/extccomp.nim

@@ -556,14 +582,20 @@ proc getCompileCFileCmd*(conf: ConfigRef; cfile: Cfile): string =
    else:
      cfile.obj

+  # D files are required by nintendo switch libs for
+  # compilation. They are basically a list of all includes.
+  let dfile = objfile.replace(".o", ".d").quoteShell()


use changeFileExt instead.

Araq

posix_nintendoswitch.nim seems copy&pasted code and I dunno why that is.

Araq · 2018-06-26T16:07:20Z

compiler/extccomp.nim

+    # Map files are required by Nintendo Switch compilation. They are a list
+    # of all function calls in the library and where they come from.
+    var mapfile = getNimcacheDir(conf) / splitFile(projectFile).name & ".map"
+    mapfile = quoteShell(mapfile)


Combine these two statements.

Araq · 2018-06-26T16:08:59Z

doc/nimc.rst

+Simply add --os:nintendoswitch
+to your usual ``nim c`` or ``nim cpp`` command. DevkitPro setup must be the same as
+what is the default with their new installer
+[here for Mac/Linux](https://github.com/devkitPro/pacman/releases) or


That's not how RST links look like.

Oops, copy pasted from MD file. I'll update to be in line with rst.

Araq · 2018-06-26T16:09:05Z

doc/nimc.rst

+to your usual ``nim c`` or ``nim cpp`` command. DevkitPro setup must be the same as
+what is the default with their new installer
+[here for Mac/Linux](https://github.com/devkitPro/pacman/releases) or
+[here for Windows](https://github.com/devkitPro/installer/releases).


Araq · 2018-06-26T16:09:44Z

doc/nimc.rst

+
+This will generate a file called ``switchhomebrew.elf`` which can then be turned into
+an nro file with the ``elf2nro`` tool in the DevkitPro release. Examples can be found at
+[the nim-libnx github repo](https://github.com/jyapayne/nim-libnx.git).


Araq · 2018-06-26T16:10:28Z

doc/nimc.rst

+There are a few things that don't work because the DevkitPro libraries don't support them.
+They are:
+
+    1. Waiting for a subprocess to finish. A subprocess can be started, but right


Don't indent this list.

Run nim rst2html doc/nimc.rst and look at the produced HTML.

Thanks for the tip! I didn't know about this command.

Araq · 2018-06-26T16:18:39Z

lib/pure/dynlib.nim

+  #
+
+  proc dlclose(lib: LibHandle) =
+    raise newException(OSError, "dlclose not implemented on Nintendo Switch!")


It would be better if these would cause a compile-time error but I can imagine why that's rather hard to do...

Is there a way that you usually do this in Nim?

proc dlclose(lib: LibHandle) {.error: "not implemented".} but it's a compilerProc so it needs a patch in cgen.nim.

Araq · 2018-06-26T16:19:57Z

lib/posix/posix_nintendoswitch.nim

+#
+#
+#            Nim's Runtime Library
+#        (c) Copyright 2018 Andreas Rumpf


Use 'Nim contributors' as the name or your own name.

Araq · 2018-06-26T16:20:15Z

lib/posix/posix_nintendoswitch.nim

+
+# To be included from posix.nim!
+
+{.deadCodeElim: on.}  # dce option deprecated


Remove this line.

Araq · 2018-06-26T16:21:36Z

lib/posix/posix_nintendoswitch.nim

+    aio_offset*: Off          ## File offset.
+    reserved: array[32, uint8]
+
+when hasSpawnH:


Since this is nintendo specific code, you know whether this hasSpawnH is true or not.

Araq · 2018-06-26T16:22:20Z

lib/posix/posix_nintendoswitch.nim

+  timezone* {.importc: "_timezone", header: "<time.h>".}: clong
+
+# Regenerate using detect.nim!
+include posix_linux_amd64_consts


Really? You detected these and put them into posix_linux_amd64_consts?

Oops, I missed that one. I'll update to the generated code from the switch.

jyapayne · 2018-06-27T01:09:29Z

posix_nintendoswitch.nim seems copy&pasted code and I dunno why that is.

I did copy & paste from amd64 because the file was largely the same but the fields in each object were slightly different. I thought about doing when(nintendoswitch) statements, but there are so many of them I thought it would be better to group them in one file.

Araq · 2018-06-27T10:29:41Z

I thought about doing when(nintendoswitch) statements, but there are so many of them I thought it would be better to group them in one file.

Alright then.

kobi2187 · 2018-07-02T19:22:22Z

@jyapayne Your patience is admirable!

jyapayne · 2018-07-02T23:50:26Z

@kobi2187 OSS contributions require a lot of patience most of the time :) Thank you for the kind words!

jyapayne added 6 commits June 19, 2018 16:58

Add config section for Nintendo Switch

4874be9

Add compiler configuration for Nintendo Switch and it's CPU

c40ecec

Add specific lib code for Nintendo Switch

8a0bff9

Add GC support for Nintendo Switch

c349083

Update changelog for Nintendo Switch

c8e0c5a

Update changelog with more info about fixed paths

e4dff3a

dom96 reviewed Jun 19, 2018

View reviewed changes

jyapayne added 6 commits June 19, 2018 19:07

Cleaned up GC memory management a bit

14ebe3e

Relocate docs for Switch

ed18079

Rename aarch64NoneElfGcc to nintendoSwitchGCC

0cf2e68

Remove armv8a57

1935c16

Fix installer.ini

f7ec343

Reuse code in linux and amd64

3afe344

yglukhov reviewed Jun 19, 2018

View reviewed changes

jyapayne mentioned this pull request Jun 21, 2018

Add configurable compiler to global section genotrance/nimgen#4

Merged

jyapayne added 3 commits June 22, 2018 10:44

Add posix defs for nintendo switch

9b300a5

Add more defined sections for nintendo switch

d3cf79e

Remove old comment

de8eb3d

jyapayne commented Jun 22, 2018

View reviewed changes

Add what's not supported for Nintendo Switch docs

06ba653

Make nintendoswitch == posix

cd42369

Merge branch 'devel' into nintendo_switch_support

c137097

Varriount assigned Araq Jun 24, 2018

Remove DEVKITPRO references from nim.cfg

3561f83

Araq reviewed Jun 26, 2018

View reviewed changes

Araq requested changes Jun 26, 2018

View reviewed changes

jyapayne added 3 commits June 27, 2018 10:02

Make PR extccomp changes

25ae23a

Remove Result type alias

b525bcc

Add separate switch consts file

66c0ed9

jyapayne added 2 commits June 27, 2018 10:30

Update docs for nintendo switch

bec2c96

Fix travis errors with undefined consts and add correct wait.h procs

b233ce0

Araq approved these changes Jun 27, 2018

View reviewed changes

Araq merged commit 559a761 into nim-lang:devel Jun 27, 2018

alaviss mentioned this pull request Jan 7, 2019

dynlib: remove redundant nintendoswitch "implementation" #10223

Closed


		# To be included from posix.nim!

		{.deadCodeElim: on.} # dce option deprecated

Nintendo switch support #8069

Nintendo switch support #8069

Conversation

jyapayne commented Jun 19, 2018

dom96 commented Jun 19, 2018

dom96 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jyapayne commented Jun 19, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jyapayne commented Jun 21, 2018

Araq commented Jun 21, 2018

jyapayne commented Jun 21, 2018

jyapayne commented Jun 21, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jyapayne commented Jun 22, 2018

jyapayne commented Jun 22, 2018

GULPF commented Jun 23, 2018

jyapayne commented Jun 23, 2018

jyapayne commented Jun 23, 2018

jyapayne commented Jun 24, 2018

jyapayne commented Jun 26, 2018

Choose a reason for hiding this comment

Araq left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jyapayne commented Jun 27, 2018 • edited Loading

Araq commented Jun 27, 2018

kobi2187 commented Jul 2, 2018

jyapayne commented Jul 2, 2018

jyapayne commented Jun 27, 2018 •

edited

Loading