External Callbacks

This document discusses various design issues related to callbacks that cross the Rebol-C border.

Callbacks have been released in A103. See Extension Callbacks for details. User comments can be added below.

Table of Contents User Comments Callback Directions Types of C Callbacks Synchronous Callback Asynchronous callback Special Issues Allocation and GC Threading Struct Passing

User Comments

Comments here.

Callback Directions

WARNING: The documentation below is older and obsolete.

There are two fundamental types of cross-domain callbacks:

• Callbacks that pass from the Rebol domain to the C domain.
• Callbacks that pass from the C domain to the Rebol domain.

The first is not difficult. It is done by way of a COMMAND in an extension module.

The second is more difficult and is the subject of this document.

Types of C Callbacks

There are two types of callbacks from C to Rebol:

1. Synchronous callback - a direct call
2. Asynchronous callback - an event driven call

Synchronous Callback

A synchronous callback passes control back through the Rebol Extension API, which generates the necessary stack frame, and re-enters the interpreter's function evaluator. It's like calling a function via APPLY.

The arguments to the function are passed in a standard Rebol extension format, and the result is returned in that same format. Simple immediate values and series values with offsets can be passed and returned. More complex datatypes cannot be passed, and when required, must be handled with context references (for example by passing a context-relative word or block relative index to locate the more complex datatype.)

The disadvantages of this method are:

1. The C stack now holds state from both the Rebol and C domains.
2. An exception (such as an ERROR or THROW) within the Rebol may throw the C stack completely past the C domain, losing the ability to conclude any state related to that domain (e.g. allocation, open files, etc.)
3. GC can occur during the Rebol code, invalidating any Rebol series that may have been created exclusively in the C domain and not otherwise referenced from Rebol.

It should be noted that good coding practices can prevent most of these problems. In addition, the APPLY function can be told via an option to catch exceptions and return them directly, rather than throwing the stack past the C domain.

Asynchronous callback

An event can be used to trigger a callback. This happens in an asynchronous fashion. The C code calls the extension API which builds an event to hold the callback's function and argument list. The event is queued to the system port's event list, and a special signal bit is set, and the interpreter will interrupt evaluation to process the callback function.

This method allows, for example, C code to register specific C-based callbacks with an operating system service that executes those callbacks in a fashion similar to an interrupt.

Rebol devices use this method as the primary mechanism for all asynchronous I/O operations.

The main disadvantage of this method is that a programmer must be careful not to evaluate code that may conflict with the callback function. In addition, an event driven callback has no return value (it will be ignored) so any results must be stored in a contextual variable (object or module).

Special Issues

Allocation and GC

Rebol uses automatic memory management, so Rebol and C code cannot be directly merged or referenced.

In addition, if the C code allocates Rebol series such as a string, image, or block, that memory is "volatile" because without proper referencing, it can be targeted for garbage collection.

Solutions to this problem are:

• A mode where GC is intentionally disabled within the callback, to prevent series GC.
• A protection flag set on extension-created Rebol series which prevents these from being collected. Such series would need to be tracked and freed by the C code at a later time.

Threading

Callbacks, as with extensions in general, should be designed and compiled with usual thread-safe best-practices. This includes things like:

• Only linking with (or loading of) multi-thread safe system libraries. (E.g. on Windows VisualC, see the library linkage options to select the correct library.)
• Avoiding the use of shared memory spaces (e.g. shared, static function vars, non-thread-local globals or non-constant arrays or structs).
• If a target OS does not support thread-local variables or separate data-segment virtual memory allocation on per-thread library loading, then your code must use local (stacked) variables only. (Hopefully, there are no such OSes out there.)

Note: If we missed any, please add them.

Struct Passing

C structs and Rebol are not directly compatible. Therefore, it is the responsibility of the C extension code to deal with the "marshaling" of values between the two domains.

For example, a callback may need to return multiple values to Rebol. This can be done with a Rebol block for simple structs or with an object for more complex structs. The allocation of the block or object can be passed in from Rebol, done in advance by the C command, or done dynamically within the callback, assuming that it has access to the R3 Extension Lib API.