natfeats proposal
Original proposal document by --- 
Laurent Vogel 2001/12/08
This document describes a general technique to allow interaction between software running on a m68k processor inside an emulator, and native software.
In this document, the emulator is some software running on a native platform, to run software in a 68k platform. This document specifies a general technique by which 68k software can interact with special native software to achieve special effects not allowed directly on the 68k platform.
To this end two interfaces are specified:
- the 68k interface, describing special illegal opcodes by which the 68k code can call the native feature services.
- the implementation interface, describing how native features can be implemented in a partable way independently of the specific emulator.
This is illustrated in the following diagram.
native side 68k side
______________________/\______________________ ____/\____
/ \ / \
implementation 68k
interface interface
__________ (nf.h) _________________________ __________
| | | | g | | | |
| feature |<------->| native | l | CPU and |<------->| 68k code |
| implemen-| | feature | u | memory | |__________|
| tation | | services | e | core |
|__________| |__________|___|__________|
\______ ______/
\/
specific to
each particular
emulator
The Native feature proposal comprises of mandatory specifications, optional specifications, and optional implementations.
The mandatory specifications is the 68k interface, i.e. the behaviour that can be observed from 68k side regardless of how it is implemented in the native side.
An optional set of specification is a basic set of native features, that can be supplied or not supplied by the emulator.
Another optional specification is the implementation interface, i.e. the services provided by the emulator to implementations of particular native features. This specification is optional because it is perfectly valid to implement native features without using this interface (for example for native features very tied to particular emulators).
Other optional specifications may describe Operating System-related services on the 68k side to encapsulate the raw 68k interface. A suitable additional specification for TOS is provided in annex E.
Optional implementations are
- a supplied implementation of the generic native feature services;
- supplied implementation of some particular native features;
- supplied glue to particular emulators These are optional, because it is perfectly valid to provide the same functionality using a different implementation.
The 68k platform here is a virtual machine based on a motorola 680xx processor, where xx is one of 00, 08, 10, 20, 30, 40, 60, or any other processor with the same programming model and instruction set.
Each native 'feature' is a group of 'functions', each taking arguments and returning a value.
here is a short list of terms that have a special meaning in this document:
id - non-null feature identifier with bits 0-19 set to zero
sub-id - 20-bit function identifier integer
feature - a groupe of functions
function - a native function
68k side - a virtual machine based on a supported processor
supported processor - 68008, 680[012346]0, and compatibles
special opcodes - 68k opcodes used by the native feature framework
emulator - the container of both the 68k side and the native side
basic set - the mandatory set of features
Other 68k processors like the ColdFire, whose instruction set is not compatible to the instruction set of the supported 68k processors, are not taken into account by this document. In the event where some 68k code were to be designed to support both the current specification (when running on a supported 68k processor) and unsupported processors, that code should not use the special opcodes if it can be determined that the processor is unsupported, either by first checking the processor type, or by determining in an operating-system-dependant way that native features are not available.
This document does not support emulators emulating a multi-processor 68k machine (several 68k processors running in parallel).
The implementation interface, and the proposed implementation of native feature services do not fully support multi-thread native environments in which the 68k memory can be modified during the execution of one native feature call.
All 68k memory accessed during the execution of a native function, either directly (the stack), or indirectly (following pointers) must reside in physical memory before the native function is called. As it is an error to call a native function with non-accessible memory, the implementation of native functions may ignore the possibility that bus errors occur when they fetch 68k memory. (so, they can let the 68k glue raise a bus error automatically, without testing beforehand that the memory is valid and without having to take steps to ensure that no resource lossage occurs due to such automatic bus errors being raised).
Some of these limitations might be solved by changing the implementation of native feature services.
Each native feature has a name uniquely identifying this feature. 68k code inquires whether the feature is available by asking for the 'ID' corresponding to the name:
long id = nf_get_id(name);
If the returned ID is zero, then the feature isn't available. Else, functions provided by this feature may be called given this ID, a sub-ID identifying the function among those provided by the feature, and the requested function arguments:
long result_value = nf_call(id+sub_id, the, requested, arguments);
IDs have the following format:
- bits 20-31 hold a meaningless number internally managed by the emulator;
- bits 0-19 are set to zero, and serve as a place-holder to contain the sub-ID, a number that is given to the native implementation to allow efficient handling of multiple actual functions all regrouped under the same feature name.
See Annex G for a full example of how to call the basic set.
This document describes
- a small number of special opcodes (m68k illegal instructions) used to manage native features;
- guidance for defining native features;
- a proposed basic set of native features that may be implemented by emulators.
Any emulator has the choice of providing the full interface described in this document, or nothing.
If the emulator does not provide the interface, then all illegal opcodes defined below must provoke the normal m68k behaviour (i.e. raise an illegal exception).
If the emulator provides one illegal opcode defined below, then it must also provide the complete set of opcodes, with the behaviour defined in this document. Such an emulator is called a compliant emulator. In the rest of the document, 'emulator' means 'compliant emulator'.
Native functions do the job of C functions, using the C calling convention described here:
- values are represented in memory as big endian (the byte with lowest address is the most significant byte);
- 'char' are 8bit values, either signed or unsigned (unspecified);
- 'short' are 16bit values;
- 'int', 'long' and pointers are 32 bit values;
- the return type of a function is either void (no return type), or a 32bit value; functions cannot return 8bit or 16bit values, and functions cannot return structs;
- parameters are passed on the stack; the last parameter is pushed first, then any parameters until the first parameter which is pushed last; when pushing parameters, if the size of the parameter is less than 32 bit, the parameter is converted to a 32bit value by padding to the left with unspecified bytes;
- struct members are set in memory in growing addresses, with padding using the smallest number of unspecified bytes needed so that any member not of type char, signed char, unsigned char, or an array of these three types, has an even address.
String encoding is not specified in this document. Generally speaking native functions should document this. When a string is to be displayed to the user (as opposed as written in a file) it can be assumed that chars in the range 0x20 to 0x7e are ASCII (ISO-646).
Native functions should not alter any registers other than d0, unless properly justified and documented.
Feature names beginning with 'NF_' or 'nf_' are reserved.
Special opcodes are taken in the range 0x73nn (illegal moveq instructions). The following opcodes are defined:
mnemonic nf_getid (in ARAnyM debugger it's NATFEAT_ID)
value 0x7300
prototype unsigned long nf_get_id(const char * feature_name);
Ignore the first long word on the stack; let feature_name be the second long word on the stack; if the null-ended string pointed to by feature_name is the name of an implemented native feature, then return its ID in register D0, else return zero.
mnemonic nf_call (in ARAnyM debugger it's NATFEAT_CALL)
value 0x7301
prototype long nf_call(unsigned long feature_id, ...);
Ignore the first long word on the stack; let feature_id be the second long word on the stack; if feature_id is not a valid native function ID then the result is unspecified; else, call the function defined by the ID (optionnally containing a non-null sub-ID), passing the rest of the stack arguments (i.e. starting at address stack+8) as parameters. Returns in register D0 the 32bit value returned by the fonction. If the function is documented as not returning any meaningful value, (i.e. declared as void function(...)) then the value in D0 is unspecified.
Note: both these special opcodes ignore the first long word on the stack. This is done so that these special opcodes can be encapsulated into small assembler routines that are declared with the same prototype as indicated above for each special opcode. For instance, here is an actual declaration and code for nf_get_id:
extern long nf_get_id(const char * feature_name);
.global _nf_get_id:
_nf_get_id:
dc.w 0x7300
rts
and here is another implementation of the same:
long nf_get_id_opcodes = 0x73004e75;
#define nf_get_id ((long (*)(const char *))&nf_get_id_opcodes)
These opcodes are defined both for supervisor mode (Interrupt or Master modes on 680x0) and normal user mode. Each NatFeat has a bool flag "isSupervisorModeRequired" and if set to true then calling such opcode from user mode will result in a priviledge exception.
The stack used for passing arguments will be the current stack (SSP/ISP for Supervisor/Interrupt mode, MSP for Master mode).
Feature documentation shall provide the following information:
- the feature name, a case-insensitive string of printable ASCII characters.
- the features that are required for this one to work
- the list of functions provided by this feature, and for each function:
- its sub-id
- a description of the way the feature works, including a function prototype and a description of return values and error cases
Optionally the feature text can include an implementation, that is, source code running inside the emulator. In this case it is recommended that it be given with a licence allowing it to be included in most emulators, including closed-source ones (licence like LGPL); also it is best if such sample implementation is written in one of the proposed implementation interfaces.
sub-ID 0 - user allowed
unsigned long getName(char *buffer, unsigned long size)
Fills 'buffer' with a null-terminated string representing the emulator name. At most 'size' bytes will be written in the buffer (which means that the string will be truncated if it contains more than size - 1 characters). The string should contain only printable ASCII characters (ASCII 32 to 126 inclusive). Returns the length of the string before truncation.
sub-ID 1 - user allowed
unsigned long getFullName(char *buffer, unsigned long size)
Similar to getName, but the string is the emulator name plus its version (and anything else that might be printed together with the emulator name) Returns the length of the string before truncation.
sub-ID 0 - user allowed
long getVersion(void);
Returns the version of the native feature framework, hex encoded (upper word = major number, lower word = minor number). This document describes native features version 1.0 (hex encoding 0x00010000).
sub-ID 0 - user allowed
unsigned long output(const char *string);
Emit a null-terminated string of printable ASCII chars on a particular output stream, intended for use to display debug messages. Returns the number of chars printed. Characters not among the set of ASCII chars 9 (horizontal tabulation '\t'), 10, 13, (both meaning '\n' newline), 32-126 will have implementation defined representations.
sub-id 0 - supervisor mode only
void shutdown(void);
Immediately terminate the execution of the emulator, with no user interaction.
Features should be developed in a way that allow them to be called from multitasking host systems. To achieve this, there should not be a single 'errno' feature collecting the status of multiple distinct features, but rather each feature or feature group should provide an independent way of reporting errors.
In any case calling a feature with incorrect parameters should not crash the emulator.
On emulators implementing MMU and where physical addresses differ from logical addresses, the memory that will be accessed by native features uses the physical addresses (so the NF caller needs to make sure to pass in the translated memory address). The reason for this is to be able to implement the native features on the host side without the help of emulated CPU, working very much like a DMA (Direct Memory Access) to gain maximum possible speed.
If the feature was asked to access memory that does not exist, or whose access is not allowed by the current MMU policy, or to write to ROM, then the feature should directly provoke a bus error. The exact address used in the bus error stack frame can be any address within the range of offending addresses.
TODO: remove the need for reporting the offending address?
DISCUSSION: almost no emulator keeps track of the offending address, and even less 68k software cares for it. relaxing the need for reporting an offending address would simplify native feature implementations
for instance, if a bzero native feature is asked to clear from memory 0x2000 to 0x3fff, and only memory 0 to 0x2fff is writeable, then
- there should be a bus error,
- the value of memory from 0x2000 to 0x2fff is 'unspecified'
- the address reported in the bus error can be any address between 0x3000 and 0x3fff
for features running on a 68000-only emulator, asking for unaligned memory accesses will NOT provoke address errors. However, such aligned memory accesses may be much slower.
DISCUSSION: We don't want to impose the emulator to implement unaligned memory access; however we don't want to impose to the native function implementation to check whether they are running on 68000-only emulators.
There is no provision in this document to allow native features to call 68k subroutines. Therefore the control flow must reside entirely on the 68k side. Feature designers should decompose the native functions into small functions that can be interspersed, on the 68k side, by calls to the required subroutines.
-
REMOVE the NF cookie stuff which is now obsolete ant NOT TO BE USED!
-
Adjust the document to match the supervisor only rule (not only for FreeMiNT) implied by our philosophy to support m68k development
-
add a paragraph encouraging feature designers to think!
-
add a paragraph encouraging native features using interrupts to be as independent from other features as possible, to avoid deadlock
-
rework appendix B completely.
- Convert all examples to use the freemint_cvsroot\sys\mint\arch\nf_ops.h
see file nf.h (to be rewritten following experience gained)
(to be rewritten following experience gained)
Thanks to all who did contribute to this document, including:
Milan Jurik
Petr Stehlik
Martin Döring
Johan Klockars
...
TODO: complete this list.
THIS IS OBSOLETE! Go to the Native Features Best Practice to find out how to use NF!
TOS-compatible operating systems may define a cookie named ''_ _NF (0x5f5f4e46L)'', pointing to the following data:
struct cookie__NF_data {
long magic; /* equals 0x20021021 */
long (*getid)(const char *name);
long (*nfcall)(long id, ...);
};
this cookie serves two purposes:
- report that native features are available
- provide pointers to functions encapsulating the illegal opcodes.
It is intended to be used as follows:
- 68k code that do not intend to ever run on unsupported processors is not obliged to check for this cookie, and may instead use the illegal opcodes directly; However it may check for the cookie, and if the cookie is not present it may still try the illegal opcodes. If the cookie is present it may use either the supplied function pointers or the illegal opcodes.
- 68k code that must be able to run on a unsupported processor must check for this cookie. In any case the illegal opcodes should NOT be used directly. If the cookie is not present, then no native features are available. If the cookie is present, then native features are available, throught the use of the provided function pointers in the cookie.
see file ../atari_example.c ???
2001-12-08
- 'native feature proposal' posted to aranym mailing list.
2001-12-09
- erratum about an interrupt manager (not integrated in this document)
2001-12
- discussions on the aranym mailing list; moving from 16 to 32bit ints, question about string charsets, replaced 'host side' by '68k side'.
2002-03-02
- rewriting of the document, put some things in attic.txt,
- rewrote the beginning of the document, added an overall diagram.
- put the implementation interface in a separate nf.h
2002-08-11 (joy)
- corrected the actual nf calls according to what is implemented in ARAnyM.
- NatFeats are usermode callable now.
- Value-only features removed.
- Basic set of features replaced with list of current basic NF implemented in ARAnyM.
2002-10-22
- enumerated supported processors, natfeat usage on unsupported processor need an OS-specific way (i.e. __NF cookie)
- removed direct call; opcodes better explained (ignored return address)
- told about sub-id, distinguished between 'features' and 'functions'
- reworded a little the basic set (TODO, check with actual aranym implementation (joy?))
2002-10-28
- NF_STDERR added, NF_NAME corrected (returns the number of chars before truncating).
2002-10-29
- a note about valid m68k physical memory added
2003-06-03
- NF_SHUTDOWN added
2006-01-25
- Added a few TODO entries (needs head2heal revision, a lot of stuff is obsolete)
- Converted from the CVS/src/natfeat/doc/native.txt to DokuWiki format