Separating compilation units

A Clay project can be separated into compilation units using external functions to interface between modules. Whereas normal Clay functions have internal linkage (like static functions in C) and use an unspecified call-by-reference based calling convention, external functions use the platform's standard C calling convention (including call-by-value semantics) and can be invoked from other compilation units written in C, C++, Clay, or other C-compatible programming languages.

=== Defining external functions

External functions are defined by placing the external keyword in front of a function definition:

  external distance(x:Double, y:Double) = sqrt(x*x + y*y);

Unlike a normal Clay function, an external function cannot be generic and all of its input types must be fully specified. The function may return no more than one value. All input and output types must be POD; that is, they cannot have custom copy constructor, move, destroy, or assign overloads. The function cannot throw exceptions; if a Clay exception is raised and left uncaught in an external function, the function will call abort. The name of the function must be unique and cannot be overloaded or reused by another module within the same compilation unit, since symbol name for the function will be decorated like a C function name in the resulting object file.

=== Compiling code with external functions

To build a compilation unit without a main function, use clay -shared -asm to generate a .s file that can be assembled by gcc:

  clay -shared -asm -o foo.s foo

Despite the name "shared", the generated assembler file can be statically linked with other object files with no problems. Without the -asm flag, -shared will build a standalone dynamically linked library:

  clay -shared -o foo.so foo

=== Using external functions

An external function in another compilation unit can be linked using an external declaration without a function body:

   external distance(x:Double, y:Double) Double

The output type, if any, must be specified. If no return type is specified, then the function is declared as returning no values (like a void returning function in C). Since it uses the C calling convention, the referenced function may be a Clay external function, a C function, an extern "C" C++ function, or any C-compatible function.

=== Organizing compilation units within a Clay project

=== Working around limitations of external functions

Because they use the C calling convention, external functions are limited compared to normal Clay functions: they can't directly accept values of non-POD Clay types as inputs or return them as outputs, and they can't throw exceptions. However, pointers to non-POD types can be passed to external functions:

  external extern_doubleVector(in:Pointer[Vector[Int]], out:Pointer[Vector[Int]]) {
      out^ <-- map(x => x*2, in^);
  }

On the client end, the external function can then be wrapped in a Clay function to provide a normal Clay interface to the function:

  external extern_doubleVector(in:Pointer[Vector[Int]], out:Pointer[Vector[Int]]);

  doubleVector(in:Vector[Int]) out:Vector[Int] {
      extern_doubleVector(&in, &out);
  }

To pass exceptions across compilation units, an external function must catch all unhandled exceptions and pass them to the caller to be rethrown:

  // Library compilation unit
  external extern_doubleVector(maybeEx:Pointer[Maybe[Exception]], in:Pointer[Vector[Int]], out:Pointer[Vector[Int]]) {
      try {
          out^ <-- map(x => x*2, in^);
      } catch (ex) {
          maybeEx = Maybe(ex);
      }
  } 

  // Client compilation unit
  external extern_doubleVector(maybeEx:Pointer[Maybe[Exception]], in:Pointer[Vector[Int]], out:Pointer[Vector[Int]]);
  
  doubleVector(in:Vector[Int]) out:Vector[Int] {
      var maybeEx = nothing(Exception);
      extern_doubleVector(&maybeEx, &in, &out);
      maybe(maybeEx, ex => { throw ex; });
  }

=== Using the externals module

The above workarounds are tedious and difficult to get right. The externals module provides two helper functions, wrapAsExternal and callExternal, which abstract away the workaround details indicated above. They allow both the library implementation and Clay code to use native Clay interfaces with the expected reference semantics. Here's an example:

  // Library compilation unit
  private doubleVector(in:Vector[Int]) = map(x => x*2, in);
  
  external extern_doubleVector(ex:ExternalException, in:Pointer[Vector[Int]], out:Pointer[Vector[Int]]) {
      wrapAsExternal(doubleVector, static 1, ex, in, out);
  }

  // Client compilation unit
  external extern_doubleVector(ex:ExternalException, in:Pointer[Vector[Int]], out:Pointer[Vector[Int]]);
  
  doubleVector(in:Vector[Int]) = callExternal(extern_doubleVector, in);

wrapAsExternal takes a Clay function name, a static integer indicating the number of input arguments to the named function, an ExternalException value to receive any caught exceptions, and then a list of pointers to input values followed by output values. The exception, in, and out pointers should all be received as arguments to the external function. wrapAsExternal dereferences all of the pointers and calls the Clay function on the referenced values. It also catches any exception and writes it to the ExternalException parameter if caught. On the other side, callExternal takes an external function name and a list of input arguments and calls the external function, using the output pointers to fill its return values. If the external function writes an exception to its ExternalException argument, callExternal will rethrow it. wrapAsExternal and callExternal thus work together to provide a channel with full Clay semantics across the C calling convention.