Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 323 lines (216 sloc) 11.705 kB
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
1
2 =encoding utf8
3
04840a3 [Spec] treat all authors equally
lwall authored
4 =head1 TITLE
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
5
6 DRAFT: Synopsis 21: Calling Foreign Code
7
04840a3 [Spec] treat all authors equally
lwall authored
8 =head1 VERSION
9
10 Created: 27 Feb 2009
11
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
12 Last Modified: 23 Nov 2012
13 Version: 2
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
14
ecda04f @cognominal S21: removing now obsolete mentions of zavolaj
cognominal authored
15 The document is a draft.
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
16
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
17 =head1 SYNOPSIS
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
18
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
19 use NativeCall;
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
20
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
21 sub native_function(int arg) is native('libsomething') { * }
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
22 sub short_name() is native('libsomething') is symbol('long_and_complicated_name') { * }
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
23
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
24 native_function(42);
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
25
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
26 =head1 DESCRIPTION
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
27
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
28 Perl 6 has a standard foreign function interface, NativeCall. The only
29 libraries NativeCall is able to interface with are those written in C.
30 Languages like Fortran and C++ require name mangling, which is
31 compiler-specific and thus falls well beyond the scope of this specification.
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
32
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
33 Hypotheticals:
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
34
35 =over
36
37 =item
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
38 This is likely not an exhaustive list of showstoppers for C++/Fortran compat;
39 also, some platforms may be tricky simply in terms of C interop as well
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
40
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
41 =back
42
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
43 =head2 Calling foreign code
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
44
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
45 A sub is marked as a native routine with the C<is native> trait. A native sub
46 must have an attached signature, which is used to specify the native-level
47 argument structure of the function. If the return type of the function is
48 C<Mu> the native function returns no value, any other return type must be
49 compatible with the types specified in the next section.
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
50
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
51 =head3 The C<is native> trait
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
52
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
53 sub trait_mod:<is>(Routine $r, :$native!) is export(:DEFAULT, :traits) { ... }
5b9746a Wrote this a few days ago, but forgot to svn add.
wayland authored
54
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
55 The C<is native> trait is the main gateway used to access C libraries. A
56 routine with this trait applied will not be a normal Perl 6 callable, but will
57 call into the function with the same name in the specified library.
58
59 The library name passed to C<is native> is passed unmodified to
60 L<man:dlopen(3)> or the platform's equivalent and the symbol is the looked for
61 in the handle returned from the call to C<dlopen>. If the library name is an
62 undefined value or the empty string, the symbol will be searched for in the
63 currently loaded libraries of the process; that is, behaviour consistent with
64 C<dlsym(RTLD_DEFAULT, symbol)> in C.
65
66 Hypotheticals:
61f7da8 @paultcochrane Adding required empty line before `=for`
paultcochrane authored
67
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
68 =for item
69 Perl 6 allows a greater range of characters in identifiers than C. Should we
70 look for cases where the identifier isn't legal in C?
71
72 =for item
73 This is rather UNIX-centric. Other platforms may very well complicate things.
74
75 =head3 The C<is symbol> trait
76
77 sub trait_mod:<is>(Routine $r, :$symbol!) is export(:DEFAULT, :traits) { ... }
78
79 Since all symbols in a C library share a single namespace with all other
80 libraries, it is common practice to prefix externally visible symbols with a
81 library prefix so as not to interfere with other libraries. In Perl 6 this may
82 be a nuisance, and the C<is symbol> trait lets a user specify a different
83 symbol name to search for than the name of the sub.
84
85 A native sub also adorned with C<is symbol> will search for the symbol
86 specified in the symbol trait, rather than the name of the subroutine itself.
87
88 =head3 The C<is nativeconv> trait
89
90 sub trait_mod:<is>(Routine $r, :nativeconv!) is export(:DEFAULT, :traits) { ... }
91
92 Native code typically supports several different calling conventions. If a
93 convention different than the default one is needed, it is specified with C<is
94 nativeconv($convention)>. The conventions supported are platform-specific.
95
96 =head3 The C<is encoded> trait
97
98 sub trait_mod:<is>(Routine $r, :encoded!) is export(:DEFAULT, :traits) { ... }
99 sub trait_mod:<is>(Parameter $p, :encoded!) is export(:DEFAULT, :traits) { ... }
100
101 Input arguments and return values that are strings may be returned in any of a
102 multitude of encodings. If the value is encoded differently from UTF-8, it
103 must be stated explicitly.
104
105 =head3 Global variables
106
ecda04f @cognominal S21: removing now obsolete mentions of zavolaj
cognominal authored
107 Caveat emptor: This whole section is conjectural.
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
108
109 Just like functions exported by a library, global variables are accessed with
110 the C<is native> trait; after all, all exported symbols are the same from the
111 point of view of the linker: a pointer to something. The C<is symbol> and
112 C<is encoding> (for strings) traits also apply to variables.
113
114 =head2 Marshalling and demarshalling of Perl 6 data
115
116 The raw internal representation of most Perl 6 objects can't be expected to
117 work sensibly with native code. To specify how to marshal and demarshal
118 complex Perl 6 objects, representation polymorphism is most frequently used,
119 but some classes are provided for frequent use cases.
120
121 For pointer types, the type object associated with the Perl 6 class represents
122 the null pointer.
123
124 =head3 Numeric types
125
126 Numeric types, both native types and not, have obvious marshalling semantics
127 (as long as they are not arbitrary-precision types). A NativeCall
bfb393d @Util Fix typos in S02, S03, S21, S26, S29, S99, and S32/Exception.
Util authored
128 implementation should support the following types:
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
129
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
130 =over
131
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
132 =item C<int8>, C<uint8> signed and unsigned byte
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
133
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
134 =item C<int16>, C<uint16> signed and unsigned two-byte integer
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
135
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
136 =item C<int32>, C<uint32> signed and unsigned four-byte integer
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
137
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
138 =item C<int64>, C<uint64> signed and unsigned eight-byte integer
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
139
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
140 =item C<int>, C<uint> signed and unsigned machine word
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
141
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
142 =item C<Int> largest available integer type
143
144 =item C<num32> four-byte floating point number
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
145
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
146 =item C<num>, C<num64> eight-byte floating point number
147
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
148 =back
149
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
150 Hypotheticals:
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
151
152 =over
153
154 =item
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
155 This is a wider range of native types than what S02 mandates. We'll either
156 want to expand that list of natives, or find some other way of specifying
157 sizes.
158
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
159 =item
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
160 There is no obvious mirror of C<Int> for largest available I<unsigned> type.
161
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
162 =item
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
163 Should C<Num> be a synonym for C<num>/C<num64>?
164
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
165 =item
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
166 If the Int or Num type object is passed, should it be silently converted to a
167 zero value, or cause an exception?
168
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
169 =item
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
170 How should overflows be handled?
171
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
172 =back
173
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
174 =head3 Strings
175
176 multi explicitly-manage(Str $x is rw, :$encoding = 'utf8') is export(:DEFAULT, :utils) { ... }
177
178 By default, a string passed to a native sub wil be marshalled to a C<char *>
179 appropriately encoded as specified with the C<is encoded> trait. The memory
180 allocated to the C string is freed when the function returns. If a C<Str>
181 object should have a persistent C<char *> associated with it, this can be
bfb393d @Util Fix typos in S02, S03, S21, S26, S29, S99, and S32/Exception.
Util authored
182 signalled by calling C<explicitly-manage($str, $encoding)>. The buffer
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
183 allocated will never be freed.
184
185 A string-valued native sub's return value will be unmarshalled according to
186 the C<is encoded> trait, and the C pointer is not freed as deciding whether
187 the caller or callee owns the data can't be decided automatically, and freeing
188 by default risks causing later code to access freed memory.
189
190 Hypotheticals:
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
191
192 =over
193
194 =item
fa517be @Util Fix typos in S02, S21, S99, S32/IO, and S32/Exception
Util authored
195 We need better facilities for signaling when it's appropriate to free data.
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
196 The current facilities have the benefit that it won't cause memory-related
197 errors later on, but on the flip side, it will leak memory over time.
198
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
199 =back
200
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
201 =head3 The C<OpaquePointer> class
202
203 class OpaquePointer is repr('CPointer') { }
204
205 The C<OpaquePointer> type is the simplest possible way to interface with C
206 pointers, and can be seen as similar to the C<void *> type in C. An
207 C<OpaquePointer> offers no way to inspect the pointer or manipulate it; it can
208 only be passed around in the program and back to C.
209
210 =head3 The C<CPointer> REPR
211
212 typedef struct _magic magic;
213 magic *magic_new(void);
214 void magic_perform(magic *m);
215
216 class Magic is repr('CPointer') {
217 my Magic sub magic_new() is native('libmagic') { * }
218 my sub magic_perform(Magic $m) is native('libmagic') { * }
219
220 method new() { magic_new(); }
221 method perform() { magic_perform(self); }
222 }
223
224 The C<CPointer> REPR enables types that are similar to C<OpaquePointer> in
225 that they cannot be introspected or mutated, but different in that they can
226 have methods. This makes it easy to interface with "object-oriented" C code
227 that returns an opaque pointer handle that encapsulate the resources used by
228 the library and lets us implement this naturally using Perl 6 OO.
229
230 A C<CPointer> object can not have attributes.
231
232 =head3 The C<CArray> class
233
234 class CArray[::Type] does Positional[Type] is export(:DEFAULT, :types) { ... }
235
236 General Perl 6 arrays support features such as laziness, which means that they
237 can not easily be marshalled into a C representation. Thus, NativeCall
238 provides the CArray type which supports a set of array features compatible
239 with marshalling to and from C. The C<Type> parameter is, of course, mandatory
240 as the exact layout of the array in memory depends on the type of the elements.
241
242 A C<Carray> that has been marshalled from a value returned from C cannot,
243 given how arrays work in C, know the bounds of the array. Thus, it is the
244 I<user's> responsibility to ensure that all accesses are within the bounds of
245 the array. NativeCall will make no attempt to figure this out, and requests
246 for array elements outside of the array is likely to result in death by
247 segmentation fault.
248
249 If the C<CArray> has been created in Perl 6, the bounds of the array are
250 known, and operations can be bounds-checked and the array grown appropriately.
251 Note, however, that growing an array may result in its C representation being
252 moved to a different memory location. Thus, if a piece of C code has stored
253 the location of an array and it is later on moved due to operations on the
254 Perl side, strange bugs and segfaults are likely to ensue.
255
256 =head3 The C<CStruct> REPR
257
258 class StructObject is repr('CStruct') { ... }
259
260 Structs are an important part of most non-trivial C APIs; using the C<CStruct>
3c1f3f7 @paultcochrane Purge trailing whitespace in S21 -- S99
paultcochrane authored
261 REPR, arbitrary structs can be accessed just like ordinary Perl 6 classes.
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
262
263 =head3 Callable objects
264
265 Callback arguments are, in essence, no different from normal data. They are
266 declared as callables (typically with the C<&> sigil) and also have an
267 attached signature. The signature is important as the callback handling code
268 needs this information to get the function's arguments off the stack.
269
270 Callbacks returned from C are specified identically, but as return values
ecda04f @cognominal S21: removing now obsolete mentions of zavolaj
cognominal authored
271 rather than parameters (note: callbacks returned from C NYI).
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
272
273 =head3 Complex data value types
274
275 Caveat emptor: This section, like the one on global variables, is all
ecda04f @cognominal S21: removing now obsolete mentions of zavolaj
cognominal authored
276 conjecture. Nothing is implemented.
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
277
278 In Perl 6 the distinction between value type and reference is intrinsic to the
279 type. In C, on the other hand, any type can be used both as a value and
280 reference type, depending on how it's used. Thus, NativeCall needs some
281 mechanism to duplicate this. One possible source of inspiration for this is
282 C#. C# distinguishes between value and reference types similarly to Perl 6 and
283 also has a well-supported foreign function interface.
284
285 =head3 Varargs
286 To be determined. This section is hypothetical.
287
288 One option is an API similar to the C99 C<stdarg.h> macros and explicitly get
289 arguments off an opaque object. For example C<my $arg = va_arg($args, Type)>.
290
291 =head2 Miscellaneous helper functions
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
292
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
293 =head3 Refreshing outdated objects
294
295 multi refresh($obj) is export(:DEFAULT, :utils) { { ... }
296
297 To avoid unmarshalling data from the C representation whenever data is
298 accessed, an efficient implementation is going to want to cache unmarshalled
299 data. Whenever a complex object is passed to a native subroutine, the
300 implementation should make sure the cache data isn't out of date. However, if
301 the C code saves a pointer passed to it and a later invocation mutates the
302 data pointed to, NativeCall can't magically detect this. In cases like this,
303 the user will have to use C<refresh> to invalidate any outdated objects in the
304 cache.
305
306 Hypotheticals:
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
307
308 =over
309
310 =item
6fca0a0 @arnsholt Rewrite S21 so that Zavolaj/NativeCall is spec foreign function inter…
arnsholt authored
311 Sometimes it will be necessary to reinterpret a pointer-valued object as a
312 different kind of pointer. One way to provide this would be a function a la:
313 C<my $val = reinterpret($ptr, Type)>.
f6636e2 @lue [S21] Fixed some POD formatting errors.
lue authored
314
315 =back
34a30e7 @lizmat Make sure we expand TABs to spaces
lizmat authored
316
3d64f1a @lucasbuchala Move AUTHORS sections to end of file
lucasbuchala authored
317 =head1 AUTHORS
318
319 Arne Skjærholt <arnsholt@gmail.com>
320 Jonathan Worthington <jnthn@jnthn.net>
321
34a30e7 @lizmat Make sure we expand TABs to spaces
lizmat authored
322 =for vim:set expandtab sw=4:
Something went wrong with that request. Please try again.