SlimCS

Overview

SlimCS is a Demiurge Studios built C# “lite” to Lua transpiler used in Seoul Engine.

What?

SlimCS is a transpiler. It converts C# .cs files, containing a “lite” version of C#, into equivalent Lua code in .lua files. The game’s scripting backend is then a LuaJIT virtual machine.

“Lite” here is a deliberately and specifically chosen C# subset that facilitates clean, “simple”, and straightforward transpilation to and from C# and Lua.

For example, our C# subset does not include structs, because these can have surprising qualities as a Lua equivalent (all objects in Lua are by-reference tables, so structs would be more expensive in cases where they would be less expensive in C#).

An important point - subset and “lite” here do not mean “broken”. Our C# transpiler is built using Roslyn. It was our intention that, within explicitly stated constraints, all valid C# code will transpile to valid Lua code of equivalent behavior. Think of working in SlimCS as akin to working in a C++ codebase that does not use the std:: namespace and disables some languages features like RTTI and exceptions.

Where?

That are 3 distinct components that make up the entirety of our SlimCS toolset:

• SlimCS - this is the command-line application that wraps the transpiler.
• SlimCSLib - this is the actual C# to Lua transpiler (built against Roslyn, written in C#). SlimCSLib also includes an analyzer that can be added to Visual Studio scripts project to enable additional SlimCS checks on code at edit time.
• SlimCSVS - integration into Visual Studio. This is our debugger integration (to debug Seoul Engine applications running SlimCS scripts).
• ScriptDebuggerClient - the client software to that connects to the debugger server implemented in SlimCSVS. The Script project contains the entirety of our LuaJIT integration.

Why?

SlimCS exists to fulfill the following goals:

• Static typing and first class tooling for editing (Visual Studio).
• First class runtime interpreter to support rapid iteration and to reduce download size and binary executable bloat.
• Sandboxed and relatively simple to support and maintain script runtime.

Given the above guidelines, we arrived at a compromise solution that uses C# for authoring (great tooling, static typing, lots of industry experience, samples, and documentation) and LuaJIT for the runtime (very fast dynamic JIT and interpreter, highly vetted, relatively easy to integrate and understand).

How?

Q: How do I get started?

Open Scripts.sln. Note that you will need to install the SlimCSVS plugin first to open this solution. Also note that the corresponding project file (Scripts.csproj) uses a wildcard pattern to include files. In order to refresh the solution to see new files, you will need to unload and then reload the project.

IMPORTANT: Do not manually add .cs files to the Scripts project, or this will break wildcard inclusion. Once broken, the .csproj will stop being a proper view of the script files that will actually be included in the SlimCS build.

Q: How do I debug the debugger?

The Visual Studio debugger plugin is a host. The client in Seoul Engine connects via a TCP socket. If you open the SlimCSVS solution and start a debugging session targeting the SlimCSVS.Package project, this will launch a Visual Studio devenv in “experimental” mode, which allows you to debug the server code.

You can likewise launch the game application in a separate Visual Studio session to debug the client code.

Both server and client code also support verbose logging, which log each send and receive on each end. Search for m_bVerbose in either codebase.

Q: A SlimCS is always running? Why?

That is the SlimCS daemon. It keeps the SlimCS binary "warm" and reduces compile times.

Q: How do I debug a SlimCS generation bug?

All .lua files generated for a Windows Developer build will be in GeneratedPC/ScriptsDebug. The quickest way to catch a compiler bug is to diff the input .cs file located in Authored/Scripts against its generated output .lua file.

Locally, generated .lua files will be located in GeneratedLocal/ScriptsDebug.

Q: Cooker unit tests failed with "there appears to be a bug in SlimCS". How do I fix it?

This type of failure can be one of the following (test only) failures:

1. SlimCS successfully compiled one or more files that fail to compile with the standard Visual C# compiler.
2. SlimCS generated .lua code that fails to compile.

Failure 1 is likely missing validation of the C# AST. This code is located in Compiler.cs.

Failure 2 is likely an incorrect AST to Lua source transformation. This is likely due to a bug in some part of the Generator.

Q: I changed SlimCS generation, and tests are failing because File A is X bytes, but file B is Y bytes. How do I fix it?

1. Reproduce the failure locally with RunCookerChecks.bat.
2. Use a folder diff (e.g. Beyond Compare) to sync UnitTests/SlimCS/Scripts against the files leftover by the interrupted unit test in %TEMP%\SeoulTmp\SeoulUnitTestOutLua.
3. Check-in or commit the resulting updated unit test files.

Q: A [Pure] marked method is not behaving as expected. How do I fix it?

The [Pure] attribute enforcement implementation is in EnforcePure() in Constraints.cs.

Q: SlimCS errors are not generated by Intellisense, only when building?

Currently, SlimCS errors require you to enable "full solution analysis" to see SlimCS specific errors when editing in Visual Studio:

This is disabled by default because it can slow down the performance of Visual Studio, so disable it again if your Visual Studio editing becomes unstable when editing C# projects and code.

Q: Scripts.csproj is complaining about a missing .cs file?

Known issue with wildcard inclusion of C# files. Close and re-open the project in Visual Studio. If this does not resolve the issue, close Visual Studio, delete the .vs folder next to Scripts.csproj, and re-open Visual Studio.

Limitations

SlimCS was specifically designed to be a subset of the C# language, in order to generate as little "surprising" Lua code as possible (in both structure and speed).

Completely Unsupported Features

None of the .NET library is available in SlimCS. A minimal amount of "glue" types may appear as they are required by the C# language (e.g. System.Exception).

The following C# features are entirely unsupported in SlimCS:

• The async keyword.
• The address of (&) operator.
• The pointer dereference (*) operator.
• The arrow dereference (->) operator.
• Class destructors (e.g. ~MyClass() {}).
• Variables with the fixed modifier.
• LINQ expressions.
• The lock keyword.
• The ref and out keywords (recommend that you return multiple values with an anonymous tuple instead).
• The struct keyword (only class types can be defined - all user defined types in SlimCS are by-reference).
• Reflection attributes (e.g. [Flags]) can be specified but no runtime API exists to query them and they are not actually constructed at runtime. Therefore, there is only utility in attributes that have compile-time impacts (e.g. [Conditional]).
• Values of type dynamic and the entire .NET dynamic runtime are not supported.
• The yield keyword is not yet supported. Used the coroutine API for similar functionality.
• Any .NET functionality not implemented or exposed by SlimCSCorlib (the SlimCS variation of the .NET framework). Either browse the corlib metadata via the Scripts.csproj or browse the source code in SeoulTools\Code\SlimCSCorlib\ for supported functionality. Notable exclusions include:
  • MulticastDelegate, which backs the event keyword in C#.
  • The decimal type.
  • The Console type.

Arrays

Arrays of type object are not allowed. This is to ensure that an array is always known (at compile time) to contain entirely either by-value (cannot be null) or by-reference (can be null) values.

SlimCS does not support multi-dimensional arrays.

Assignments

The assignment operator and all of its flavors (e.g. *=, +=, etc.) are statements in SlimCS, not expressions. As a result, the following is valid C# code but invalid SlimCS code:

int i = 0;
int j = ++i;

Built-in Types

SlimCS supports a limited subset of C#’s built-in types. These includes:

• bool
• double
• int
• object
• string

uint can also be used in very limited circumstances. In particular, to cast the left hand operand of a right shift (e.g. ((uint)i) >> 1) in order to specify a right shift vs. an arithmetic right shift.

Generics

Generic support is incomplete or inconsistent in SlimCS. Recommended to treat Generics as entirely unsupported, but limited use is possible (e.g. MovieClip.GetChildByName<T>). In general, Generics support is considered bugged in the current implementation of SlimCS.

Integer Operations

Ops on type int (e.g. +, /, %, *, - and bit operations) are supported. Handling of invalid integers ops can differ from stock C#. In particular:

• a / b where b == 0:
  • In C#, this will raise a DivideByZeroException.
  • In SlimCS, this will produce:
    • On x86/x64: -2,147,483,648 (Int32.MinValue).
    • On arm/arm64:
      • 2,147,483,647 when a > 0
      • 0 when a == 0
      • -2,147,483,648 when a < 0
• a / b where a == -2,147,483,648 and b == -1:
  • In C#, this will raise an OverflowException.
  • In SlimCS, this will produce:
    • On x86/x64: -2,147,483,648 (Int32.MinValue).
    • On arm/arm64: 2,147,483,647 (Int32.MaxValue).
• a % b where b == 0:
  • In C#, this will raise a DivideByZeroException.
  • In SlimCS, this will produce 0.
• a % b where a == -2,147,483,648 and b == -1:
  • In C#, this will raise an OverflowException.
  • In SlimCS, this will produce 0.

Tuples

Anonymous tuples (e.g. public (int, int) MyMethod()) are partially supported in SlimCS (and encouraged as a replacement for the unsupported ref and out keywords). Limitations of anonymous tuples:

• Tuple variables are not supported (e.g. (int, int) a = (1, 2); is not valid SlimCS code).
• Tuples cannot be function parameters (e.g. public void MyMethod((int, int) a) is not valid SlimCS code).

In short, tuples are limited to fulfill the role of a placeholder for multiple return values in Lua. They can be returned from methods and decomposed (e.g. (var a, var b) = (1, 2)) but not used freely as their own variable type.

Known Issues

The following are known bugs in SlimCS. This section includes features that will be implemented but have not yet been implemented, as well as poor or broken behavior of the SlimCS compiler itself:

• Variadic arguments (params int[]) may produce inefficient or incorrect Lua output.
• Variadic arguments will be handled as arrays (e.g. vararg[1] will be emitted as vararg[1+1], assuming the array is 0-based) but this is incorrect.
• Operator overloading (e.g. operator++) is not consistently/completed supported.
• Local methods will produce incorrect code if they are called before their declaration site.
• Inline static member initialization can produce a runtime error if the initialization value is a complex type or expression (e.g. static A s_a = new A(); ).
• ToString() behavior of a value of an enum type is incorrect. An integer value is printed instead of a string value.
• Exception messages generated from invalid casts do not match the stock C# runtime.
• Method hiding and interfaces can generate incorrect code. For example, if A : B, B defines a method Foo, and A hides that method Foo with a new method Foo using the new keyword, the new method in A will not be invoked if A also implements an interface IA that is fulfilled by the new method in A.
• A partial class which splits its instance fields or properties across multiple files will generate incorrect code at runtime (multiple default constructors will be generated but not all of those constructors will be executed).
• <string>.Length will return the wrong value for strings with Unicode characters (C# string is UTF16, SlimCS string is UTF8).
• Casts between complex types (e.g. var a = new A(); var b = (B)a;) will not always type assert. This can result in a cast success when one is not actually valid.
• The stock C# analyzers in Visual Studio will sometimes do the wrong thing. For example, a conditional expression on a Delegate invocation (e.g. if (null != a.b) { a.b(); }) will be auto corrected to a.b?.Invoke();, which is invalid in SlimCS.
• Variables cannot be defined in case labels of switch statements unless they are wrapped in a block. e.g. switch { case 1: var a = 1; } will result in a runtime error while switch { case 1: { var a = 1; } } is ok.
• static methods that access static private members can fail if the static private members are not defined (textually) first.

Debugger Integration

Installation

The current debugger integration version is 0.1.5.

To open Scripts.sln, install the latest version of the SlimCS debugger extension.

Run the .vsix installer and it will add SlimCSVS to Visual Studio.

To debug, hit F5 in Visual Studio. Unlike most debugging flows, this will enable debug hosting but not actually launch a debuggable application. Once the debuggable application is launched, it will automatically connect to an active debugger host.

Tips

To enable code editing while the debugger is active, navigate to Tools -> Options -> Debugging and uncheck “Enable Edit and Continue”. This is unintuitive, but the reason code editing is locked when “Edit and Continue” is enabled is to allow the Debug Engine to intercept the edit events.

Known Issues

• Currently, the debugging session in Visual Studio will end whenever the debugged application terminates. The desired behavior is to leave the debugging session active until it is manually stopped.
• Call stacks do not include the enclosing class or any file information.
• Conditional breakpoints (of various types, including expression and breakpoint count) are not supported.
• The protocol used to communicate values has various robustness bugs. This mostly manifests as a failure to display large arrays or arrays of complex types.