/
nativecall.pod6
1145 lines (872 loc) · 37.6 KB
/
nativecall.pod6
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
=begin pod :tag<perl6>
=TITLE Native calling interface
=SUBTITLE Call into dynamic libraries that follow the C calling convention
=head1 Getting started
X<|nativecall>
The simplest imaginable use of C<NativeCall> would look something like this:
use NativeCall;
sub some_argless_function() is native('something') { * }
some_argless_function();
The first line imports various traits and types. The next line looks like
a relatively ordinary Perl 6 sub declaration—with a twist. We use the
"native" trait in order to specify that the sub is actually defined in a
native library. The platform-specific extension (e.g., C<.so> or C<.dll>),
as well as any customary prefixes (e.g., 'lib') will be added for you.
The first time you call "some_argless_function", the "libsomething" will be
loaded and the "some_argless_function" will be located in it. A call will then
be made. Subsequent calls will be faster, since the symbol handle is retained.
Of course, most functions take arguments or return values—but everything else
that you can do is just adding to this simple pattern of declaring a Perl 6 sub,
naming it after the symbol you want to call and marking it with the "native"
trait.
You will also need to declare and use native types. Please check L<the native types page|/language/nativetypes> for more information.
=head1 Changing names
Sometimes you want the name of your Perl subroutine to be different from the
name used in the library you're loading. Maybe the name is long or has
different casing or is otherwise cumbersome within the context of the module you
are trying to create.
NativeCall provides a C<symbol> trait for you to specify the name of the native
routine in your library that may be different from your Perl subroutine name.
=begin code :solo
unit module Foo;
use NativeCall;
our sub init() is native('foo') is symbol('FOO_INIT') { * }
=end code
Inside of C<libfoo> there is a routine called C<FOO_INIT> but, since we're
creating a module called Foo and we'd rather call the routine as C<Foo::init>,
we use the C<symbol> trait to specify the name of the symbol in C<libfoo>
and call the subroutine whatever we want ("init" in this case).
=head1 Passing and returning values
Normal Perl 6 signatures and the C<returns> trait are used in order to convey
the type of arguments a native function expects and what it returns. Here is
an example.
use NativeCall;
sub add(int32, int32) returns int32 is native("calculator") { * }
Here, we have declared that the function takes two 32-bit integers and returns a
32-bit integer. You can find the other types that you may pass in the
L<native types|/language/nativetypes> page. Note that the lack of a C<returns>
trait is used to indicate C<void> return type. Do I<not> use the C<void> type
anywhere except in the Pointer parameterization.
For strings, there is an additional C<encoded> trait to give some extra hints on
how to do the marshaling.
use NativeCall;
sub message_box(Str is encoded('utf8')) is native('gui') { * }
To specify how to marshal string return types, just apply this trait to the
routine itself.
use NativeCall;
sub input_box() returns Str is encoded('utf8') is native('gui') { * }
Note that a NULL string pointer can be passed by passing the Str type
object; a NULL return will also be represented by the type object.
If the C function requires the lifetime of a string to exceed the function
call, the argument must be manually encoded and passed as C<CArray[uint8]>:
use NativeCall;
# C prototype is void set_foo(const char *)
sub set_foo(CArray[uint8]) is native('foo') { * }
# C prototype is void use_foo(void)
sub use_foo() is native('foo') { * } # will use pointer stored by set_foo()
my $string = "FOO";
# The lifetime of this variable must be equal to the required lifetime of
# the data passed to the C function.
my $array = CArray[uint8].new($string.encode.list);
set_foo($array);
# ...
use_foo();
# It's fine if $array goes out of scope starting from here.
X<|repr>X<|is repr>X<|CStruct (native representation)>
=head1 Specifying the native representation
When working with native functions, sometimes you need to specify what kind of
native data structure is going to be used. C<is repr> is the term employed for
that.
=begin code
use NativeCall;
class timespec is repr('CStruct') {
has uint32 $.tv_sec;
has long $.tv_nanosecs;
}
sub clock_gettime(uint32 $clock-id, timespec $tspec --> uint32) is native { * };
my timespec $this-time .=new;
my $result = clock_gettime( 0, $this-time);
say "$result, $this-time"; # OUTPUT: «0, timespec<65385480>»
=end code
The original function we are calling,
L<clock_gettime|https://linux.die.net/man/3/clock_gettime>, uses a pointer to
the C<timespec> struct as second argument. We declare it as a
L<class|/syntax/class> here, but specify its representation as C<is
repr('CStruct')>, to indicate it corresponds to a C data structure. When we
create an object of that class, we are creating exactly the kind of pointer
C<clock_gettime> expects. This way, data can be transferred seamlessly to and
from the native interface.
=head1 Basic use of pointers
When the signature of your native function needs a pointer to some native type
(C<int32>, C<uint32>, etc.) all you need to do is declare the argument C<is rw> :
use NativeCall;
# C prototype is void my_version(int *major, int *minor)
sub my_version(int32 is rw, int32 is rw) is native('foo') { * }
my_version(my int32 $major, my int32 $minor); # Pass a pointer to
Sometimes you need to get a pointer (for example, a library handle) back from a
C library. You don't care about what it points to - you just need to keep hold
of it. The Pointer type provides for this.
use NativeCall;
sub Foo_init() returns Pointer is native("foo") { * }
sub Foo_free(Pointer) is native("foo") { * }
This works out OK, but you may fancy working with a type named something better
than Pointer. It turns out that any class with the representation "CPointer"
can serve this role. This means you can expose libraries that work on handles
by writing a class like this:
=begin code
use NativeCall;
class FooHandle is repr('CPointer') {
# Here are the actual NativeCall functions.
sub Foo_init() returns FooHandle is native("foo") { * }
sub Foo_free(FooHandle) is native("foo") { * }
sub Foo_query(FooHandle, Str) returns int8 is native("foo") { * }
sub Foo_close(FooHandle) returns int8 is native("foo") { * }
# Here are the methods we use to expose it to the outside world.
method new {
Foo_init();
}
method query(Str $stmt) {
Foo_query(self, $stmt);
}
method close {
Foo_close(self);
}
# Free data when the object is garbage collected.
submethod DESTROY {
Foo_free(self);
}
}
=end code
Note that the CPointer representation can do nothing more than hold a C pointer.
This means that your class cannot have extra attributes. However, for simple
libraries this may be a neat way to expose an object oriented interface to it.
Of course, you can always have an empty class:
class DoorHandle is repr('CPointer') { }
And just use the class as you would use Pointer, but with potential for
better type safety and more readable code.
Once again, type objects are used to represent NULL pointers.
=head1 Function pointers
C libraries can expose pointers to C functions as return values of functions and as
members of Structures like, e.g., structs and unions.
Example of invoking a function pointer "$fptr" returned by a function "f", using a
signature defining the desired function parameters and return value:
=begin code :skip-test
sub f() returns Pointer is native('mylib') { * }
my $fptr = f();
my &newfunc = nativecast(:(Str, size_t --> int32), $fptr);
say newfunc("test", 4);
=end code
=head1 Arrays
NativeCall has some support for arrays. It is constrained to work with machine-size
integers, doubles and strings, sized numeric types, arrays of pointers, arrays of
structs, and arrays of arrays.
Perl 6 arrays, which support amongst other things laziness, are laid out in memory
in a radically different way to C arrays. Therefore, the NativeCall library offers
a much more primitive CArray type, which you must use if working with C arrays.
Here is an example of passing a C array.
=begin code :skip-test
sub RenderBarChart(Str, int32, CArray[Str], CArray[num64]) is native("chart") { * }
my @titles := CArray[Str].new;
@titles[0] = 'Me';
@titles[1] = 'You';
@titles[2] = 'Hagrid';
my @values := CArray[num64].new;
@values[0] = 59.5e0;
@values[1] = 61.2e0;
@values[2] = 180.7e0;
RenderBarChart('Weights (kg)', 3, @titles, @values);
=end code
Note that binding was used to C<@titles>, I<not> assignment! If you assign, you
are putting the values into a Perl 6 array, and it will not work out. If this
all freaks you out, forget you ever knew anything about the C<@> sigil and just
use C<$> all the way when using NativeCall.
use NativeCall;
my $titles = CArray[Str].new;
$titles[0] = 'Me';
$titles[1] = 'You';
$titles[2] = 'Hagrid';
Getting return values for arrays works out just the same.
Some library APIs may take an array as a buffer that will be populated by the
C function and, for instance, return the actual number of items populated:
use NativeCall;
sub get_n_ints(CArray[int32], int32) returns int32 is native('ints') { * }
In these cases it is important that the CArray has at least the number of
elements that are going to be populated before passing it to the native
subroutine, otherwise the C function may stomp all over Perl's memory
leading to possibly unpredictable behaviour:
=begin code :skip-test
my $number_of_ints = 10;
my $ints = CArray[int32].allocate($number_of_ints); # instantiates an array with 10 elements
my $n = get_n_ints($ints, $number_of_ints);
=end code
I<Note>: C<allocate> was introduced in Rakudo 2018.05. Before that, you had to
use this mechanism to extend an array to a number of elements:
=begin code :skip-test
my $ints = CArray[int32].new;
my $number_of_ints = 10;
$ints[$number_of_ints - 1] = 0; # extend the array to 10 items
=end code
The memory management of arrays is important to understand. When you create an
array yourself, then you can add elements to it as you wish and it will be
expanded for you as required. However, this may result in it being moved in
memory (assignments to existing elements will never cause this, however). This
means you'd best know what you're doing if you twiddle with an array after
passing it to a C library.
By contrast, when a C library returns an array to you, then the memory can not
be managed by NativeCall, and it doesn't know where the array ends. Presumably,
something in the library API tells you this (for example, you know that when
you see a null element, you should read no further). Note that NativeCall can
offer you no protection whatsoever here - do the wrong thing, and you will get a
segfault or cause memory corruption. This isn't a shortcoming of NativeCall, it's
the way the big bad native world works. Scared? Here, have a hug. Good luck!
X<|CArray>
X<|CArray methods>
=head2 CArray methods
Besides the usual methods available on every Perl 6 instance,
C<CArray> provides the following methods that can be used to interact with the it
from the Perl 6 point of view:
=item C<elems> provides the number of elements within the array;
=item C<AT-POS> provides a specific element at the given position (starting from zero);
=item C<list> provides the L<List|/type/List> of elements within the array building
it from the native array iterator.
As an example, consider the following simple piece of code:
=begin code
use NativeCall;
my $native-array = CArray[int32].new( 1, 2, 3, 4, 5 );
say 'Number of elements: ' ~ $native-array.elems;
# walk the array
for $native-array.list -> $elem {
say "Current element is: $elem";
}
# get every element by its index-based position
for 0..$native-array.elems - 1 -> $position {
say "Element at position $position is "
~ $native-array.AT-POS( $position );
}
=end code
that produces the following output
=for code :lang<output>
Number of elements: 5
Current element is: 1
Current element is: 2
Current element is: 3
Current element is: 4
Current element is: 5
Element at position 0 is 1
Element at position 1 is 2
Element at position 2 is 3
Element at position 3 is 4
Element at position 4 is 5
=head1 Structs
Thanks to representation polymorphism, it's possible to declare a normal looking
Perl 6 class that, under the hood, stores its attributes in the same way a C
compiler would lay them out in a similar struct definition. All it takes is a
quick use of the "repr" trait:
class Point is repr('CStruct') {
has num64 $.x;
has num64 $.y;
}
The attributes can only be of the types that NativeCall knows how to marshal into
struct fields. Currently, structs can contain machine-sized integers, doubles,
strings, and other NativeCall objects (CArrays, and those using the CPointer and
CStruct reprs). Other than that, you can do the usual set of things you would with
a class; you could even have some of the attributes come from roles or have them
inherited from another class. Of course, methods are completely fine too. Go wild!
CStruct objects are passed to native functions by reference and native functions
must also return CStruct objects by reference. The memory management
rules for these references are very much like the rules for arrays, though simpler
since a struct is never resized. When you create a struct, the memory is managed for
you and when the variable(s) pointing to the instance of a CStruct go away, the memory
will be freed when the GC gets to it. When a CStruct-based type is used as the return
type of a native function, the memory is not managed for you by the GC.
NativeCall currently doesn't put object members in containers, so assigning new values
to them (with =) doesn't work. Instead, you have to bind new values to the private
members:
=begin code :skip-test
class MyStruct is repr('CStruct') {
has CArray[num64] $!arr;
has Str $!str;
has Point $!point; # Point is a user-defined class
submethod TWEAK {
my $arr := CArray[num64].new;
$arr[0] = 0.9e0;
$arr[1] = 0.2e0;
$!arr := $arr;
$!str := 'Perl 6 is fun';
$!point := Point.new;
}
}
=end code
As you may have predicted by now, a NULL pointer is represented by the
type object of the struct type.
=head2 CUnions
Likewise, it is possible to declare a Perl 6 class that stores its
attributes the same way a C compiler would lay them out in a similar
C<union> definition; using the C<CUnion> representation:
=begin code
use NativeCall;
class MyUnion is repr('CUnion') {
has int32 $.flags32;
has int64 $.flags64;
}
say nativesizeof(MyUnion.new); # 8, ie. max(sizeof(MyUnion.flags32), sizeof(MyUnion.flags64))
=end code
=head2 Embedding CStructs and CUnions
CStructs and CUnions can be in turn referenced by—or embedded into—a
surrounding CStruct and CUnion. To say the former we use C<has>
as usual, and to do the latter we use the C<HAS> declarator
instead:
=begin code :skip-test
class MyStruct is repr('CStruct') {
has Point $.point; # referenced
has int32 $.flags;
}
say nativesizeof(MyStruct.new); # 16, ie. sizeof(struct Point *) + sizeof(int32_t)
class MyStruct2 is repr('CStruct') {
HAS Point $.point; # embedded
has int32 $.flags;
}
say nativesizeof(MyStruct2.new); # 24, ie. sizeof(struct Point) + sizeof(int32_t)
=end code
=head2 Notes on memory management
When allocating a struct for use as a struct, make sure that you allocate your
own memory in your C functions. If you're passing a struct into a C function
which needs a C<Str>/C<char*> allocated ahead of time, be sure to assign a
container for a variable of type C<Str> prior to passing your struct into the
function.
=head3 In your Perl 6 code...
=begin code :skip-test
class AStringAndAnInt is repr("CStruct") {
has Str $.a_string;
has int32 $.an_int32;
sub init_struct(AStringAndAnInt is rw, Str, int32) is native('simple-struct') { * }
submethod BUILD(:$a_string, :$an_int) {
init_struct(self, $a_string, $an_int);
}
}
=end code
In this code we first set up our members, C<$.a_string> and
C<$.an_int32>. After that we declare our C<init_struct()> function for
the C<init()> method to wrap around; this function is then called from
C<BUILD> to effectively assign the values before returning the created
object.
=head3 In your C code...
=begin code :lang<C>
typedef struct a_string_and_an_int32_t_ {
char *a_string;
int32_t an_int32;
} a_string_and_an_int32_t;
=end code
Here's the structure. Notice how we've got a C<char *> there.
=begin code :lang<C>
void init_struct(a_string_and_an_int32_t *target, char *str, int32_t int32) {
target->an_int32 = int32;
target->a_string = strdup(str);
return;
}
=end code
In this function we initialize the C structure by assigning an integer
by value, and passing the string by reference. The function allocates
memory that it points <char *a_string> to within the structure as it
copies the string. (Note you will also have to manage deallocation of
the memory as well to avoid memory leaks.)
=begin code :preamble<class AStringAndAnInt {}>
# A long time ago in a galaxy far, far away...
my $foo = AStringAndAnInt.new(a_string => "str", an_int => 123);
say "foo is {$foo.a_string} and {$foo.an_int32}";
# OUTPUT: «foo is str and 123»
=end code
=head1 Typed pointers
You can type your C<Pointer> by passing the type as a parameter. It works with
the native type but also with C<CArray> and C<CStruct> defined types. NativeCall
will not implicitly allocate the memory for it even when calling C<new> on them.
It's mostly useful in the case of a C routine returning a pointer, or if it's a
pointer embedded in a C<CStruct>.
=begin code
use NativeCall;
sub strdup(Str $s --> Pointer[Str]) is native {*}
my Pointer[Str] $p = strdup("Success!");
say $p.deref;
=end code
You have to call X<C<.deref>|deref> on C<Pointer>s to access the embedded type.
In the example above, declaring the type of the pointer avoids typecasting error
when dereferenced. Please note that the original
L<C<strdup>|https://en.cppreference.com/w/c/experimental/dynamic/strdup> returns
a pointer to C<char>; we are using C<Pointer<Str>>.
=begin code :skip-test
my Pointer[int32] $p; #For a pointer on int32;
my Pointer[MyCstruct] $p2 = some_c_routine();
my MyCstruct $mc = $p2.deref;
say $mc.field1;
=end code
It's quite common for a native function to return a pointer to an array of
elements. Typed pointers can be dereferenced as an array to obtain individual
elements.
=begin code :skip-test
my $n = 5;
# returns a pointer to an array of length $n
my Pointer[Point] $plot = some_other_c_routine($n);
# display the 5 elements in the array
for 1 .. $n -> $i {
my $x = $plot[$i - 1].x;
my $y = $plot[$i - 1].y;
say "$i: ($x, $y)";
}
=end code
Pointers can also be updated to reference successive elements in the array:
=begin code :skip-test
my Pointer[Point] $elem = $plot;
# show differences between successive points
for 1 ..^ $n {
my Point $lo = $elem.deref;
++$elem; # equivalent to $elem = $elem.add(1);
my Point $hi = (++$elem).deref;
my $dx = $hi.x = $lo.x;
my $dy = $hi.y = $lo.y;
say "$_: delta ($dx, $dy)";
}
=end code
Void pointers can also be used by declaring them C<Pointer[void]>. Please
consult L<the native types documentation|/language/nativetypes#The_void_type>
for more information on the subject.
=head1 Strings
=head2 Explicit memory management
Let's say there is some C code that caches strings passed, like so:
=begin code :skip-test :lang<C>
#include <stdlib.h>
static char *__VERSION;
char *
get_version()
{
return __VERSION;
}
char *
set_version(char *version)
{
if (__VERSION != NULL) free(__VERSION);
__VERSION = version;
return __VERSION;
}
=end code
If you were to write bindings for C<get_version> and C<set_version>, they would
initially look like this, but will not work as intended:
=begin code :skip-test
sub get_version(--> Str) is native('./version') { * }
sub set_version(Str --> Str) is native('./version') { * }
say set_version('1.0.0'); # 1.0.0
say get_version; # Differs on each run
say set_version('1.0.1'); # Double free; segfaults
=end code
This code segfaults on the second C<set_version> call because it tries to free
the string passed on the first call after the garbage collector had already
done so. If the garbage collector shouldn't free a string passed to a native
function, use C<explicitly-manage> with it:
=begin code :skip-test
say set_version(explicitly-manage('1.0.0')); # 1.0.0
say get_version; # 1.0.0
say set_version(explicitly-manage('1.0.1')); # 1.0.1
say get_version; # 1.0.1
=end code
Bear in mind all memory management for explicitly managed strings must be
handled by the C library itself or through the NativeCall API to prevent memory
leaks.
=head2 Buffers and blobs
L<Blob|/type/Blob>s and L<Buf|/type/Buf>s are the Perl 6 way of storing binary data. We can use them
for interchange of data with native functions and data structures, although not
directly. We will have to use L<C<nativecast>|/routine/nativecast>.
=for code :preamble<use NativeCall;>
my $blob = Blob.new(0x22, 0x33);
my $src = nativecast(Pointer, $blob);
This C<$src> can then be used as an argument for any native function that takes
a Pointer. The opposite, putting values pointed to by a C<Pointer> into a C<Buf>
or using it to inialize a C<Blob> is not directly supported. You might want to
use L<C<NativeHelpers::Blob>|https://github.com/salortiz/NativeHelpers-Blob> to
do this kind of operations.
=for code :skip-test<incomplete code>
my $esponja = blob-from-pointer( $inter, :2elems, :type(Blob[int8]));
say $esponja;
=head1 Function arguments
NativeCall also supports native functions that take functions as arguments. One
example of this is using function pointers as callbacks in an event-driven
system. When binding these functions via NativeCall, one need only provide the
equivalent signature as L<a constraint on the code parameter|/type/Signature.html#Constraining_signatures_of_Callables>:
use NativeCall;
# void SetCallback(int (*callback)(const char *))
my sub SetCallback(&callback:(Str --> int32)) is native('mylib') { * }
Note: the native code is responsible for memory management of values passed to
Perl 6 callbacks this way. In other words, NativeCall will not free() strings passed
to callbacks.
=head1 Library paths and names
The C<native> trait accepts the library name, the full path, or a subroutine
returning either of the two. When using the library name, the name is assumed
to be prepended with "lib" and appended with ".so" (or just appended with
".dll" on Windows), and will be searched for in the paths in the
LD_LIBRARY_PATH (PATH on Windows) environment variable.
=begin code
use NativeCall;
constant LIBMYSQL = 'mysqlclient';
constant LIBFOO = '/usr/lib/libfoo.so.1';
sub LIBBAR {
my $path = qx/pkg-config --libs libbar/.chomp;
$path ~~ s/\/[[\w+]+ % \/]/\0\/bar/;
$path
}
# and later
sub mysql_affected_rows returns int32 is native(LIBMYSQL) {*};
sub bar is native(LIBFOO) {*}
sub baz is native(LIBBAR) {*}
=end code
You can also put an incomplete path like './foo' and NativeCall will automatically put
the right extension according to the platform specification.
BE CAREFUL: the C<native> trait and C<constant> are evaluated at compile time. Don't write a constant
that depends on a dynamic variable like:
# WRONG:
constant LIBMYSQL = %*ENV<P6LIB_MYSQLCLIENT> || 'mysqlclient';
This will keep the value given at compile time. A module will be precompiled and C<LIBMYSQL> will
keep the value it acquires when the module gets precompiled.
=head2 ABI/API version
If you write C<native('foo')> NativeCall will search libfoo.so under Unix like system (libfoo.dynlib on OS X, foo.dll on win32).
In most modern system it will require you or the user of your module to install
the development package because it's recommended to always provide an API/ABI version to a
shared library, so libfoo.so ends often being a symbolic link provided only by a development package.
To avoid that, the C<native> trait allows you to specify the API/ABI version. It can be a full
version or just a part of it. (Try to stick to Major version, some BSD code does not care for Minor.)
use NativeCall;
sub foo1 is native('foo', v1) {*} # Will try to load libfoo.so.1
sub foo2 is native('foo', v1.2.3) {*} # Will try to load libfoo.so.1.2.3
my List $lib = ('foo', 'v1');
sub foo3 is native($lib) {*}
=head2 Routine
The C<native> trait also accepts a C<Callable> as argument, allowing you to provide your
own way to handle the way it will find the library file to load.
use NativeCall;
sub foo is native(sub {'libfoo.so.42'}) {*}
It will only be called at the first invocation of the sub.
=head2 Calling into the standard library
If you want to call a C function that's already loaded, either from the
standard library or from your own program, you can omit the value, so
C<is native>.
For example on a UNIX-like operating system, you could use the following code
to print the home directory of the current user:
use NativeCall;
my class PwStruct is repr('CStruct') {
has Str $.pw_name;
has Str $.pw_passwd;
has uint32 $.pw_uid;
has uint32 $.pw_gid;
has Str $.pw_gecos;
has Str $.pw_dir;
has Str $.pw_shell;
}
sub getuid() returns uint32 is native { * };
sub getpwuid(uint32 $uid) returns PwStruct is native { * };
say getpwuid(getuid()).pw_dir;
Though of course C<$*HOME> is a much easier way :-)
=head1 Exported variables
Variables exported by a library – also named "global" or "extern"
variables – can be accessed using C<cglobal>. For example:
=for code :skip-test
my $var := cglobal('libc.so.6', 'errno', int32)
This code binds to C<$var> a new L<Proxy|/type/Proxy> object that
redirects all its accesses to the integer variable named "errno" as
exported by the "libc.so.6" library.
=head1 C++ support
NativeCall offers support to use classes and methods from C++ as shown in
L<https://github.com/rakudo/rakudo/blob/master/t/04-nativecall/13-cpp-mangling.t>
(and its associated C++ file).
Note that at the moment it's not as tested and developed as C support.
=head1 Helper functions
The C<NativeCall> library exports several subroutines to help you work with
data from native libraries.
=head2 sub nativecast
sub nativecast($target-type, $source) is export(:DEFAULT)
This will I<cast> the Pointer C<$source> to an object of C<$target-type>.
The source pointer will typically have been obtained from a call to a native subroutine
that returns a pointer or as a member of a C<struct>, this may be specified as C<void *>
in the C<C> library definition for instance, but you may also cast from a pointer to
a less specific type to a more specific one.
As a special case, if a L<Signature|/type/Signature> is supplied as C<$target-type> then
a C<subroutine> will be returned which will call the native function pointed to by C<$source>
in the same way as a subroutine declared with the C<native> trait. This is described in
L<Function Pointers|/language/nativecall#Function_pointers>.
=head2 sub cglobal
sub cglobal($libname, $symbol, $target-type) is export is rw
This returns a L<Proxy|/type/Proxy> object that provides access to the C<extern> named C<$symbol> that
is exposed by the specified library. The library can be specified in the same ways that
they can be to the C<native> trait.
=head2 sub nativesizeof
sub nativesizeof($obj) is export(:DEFAULT)
This returns the size in bytes of the supplied object, it can be thought of as being
equivalent to C<sizeof> in B<C>. The object can be a builtin native type such as
C<int64> or C<num64>, a C<CArray> or a class with the C<repr> C<CStruct>, C<CUnion>
or C<CPointer>.
=head2 sub explicitly-manage
sub explicitly-manage($str) is export(:DEFAULT)
This returns a C<CStr> object for the given C<Str>. If the string returned is
passed to a NativeCall subroutine, it will not be freed by the runtime's
garbage collector.
=head1 Examples
Some specific examples, and instructions to use examples above in particular platforms.
=head2 PostgreSQL
The PostgreSQL examples in L<DBIish|https://github.com/perl6/DBIish/blob/master/examples/pg.p6> make use of the NativeCall library and C<is native> to use the native C<_putenv> function call in Windows.
=head2 MySQL
B<NOTE:> Please bear in mind that, under the hood, Debian has substituted MySQL with MariaDB since the Stretch version, so if you want to install MySQL, use L<MySQL APT repository|https://dev.mysql.com/downloads/repo/apt/> instead of the default repository.
To use the MySQL example in L<DBIish|https://github.com/perl6/DBIish/blob/master/examples/mysql.p6>, you'll need to install MySQL server locally; on Debian-esque systems
it can be installed with something like:
=for code :lang<shell>
wget https://dev.mysql.com/get/mysql-apt-config_0.8.10-1_all.deb
sudo dpkg -i mysql-apt-config_0.8.10-1_all.deb # Don't forget to select 5.6.x
sudo apt-get update
sudo apt-get install mysql-community-server -y
sudo apt-get install libmysqlclient18 -y
Prepare your system along these lines before trying out the examples:
=for code :lang<shell>
$ mysql -u root -p
SET PASSWORD = PASSWORD('sa');
DROP DATABASE test;
CREATE DATABASE test;
=head2 Microsoft Windows
Here is an example of a Windows API call:
=begin code :method<False>
use NativeCall;
sub MessageBoxA(int32, Str, Str, int32)
returns int32
is native('user32')
{ * }
MessageBoxA(0, "We have NativeCall", "ohai", 64);
=end code
=head2 Short tutorial on calling a C function
This is an example for calling a standard function and using the returned
information in a Perl 6 program.
C<getaddrinfo> is a POSIX standard function for obtaining network information
about a network node, e.g., C<google.com>. It is an interesting function to look
at because it illustrates a number of the elements of NativeCall.
The Linux manual provides the following information about the C callable function:
=begin code :lang<C>
int getaddrinfo(const char *node, const char *service,
const struct addrinfo *hints,
struct addrinfo **res);
=end code
The function returns a response code 0 = error, 1 = success. The data are
extracted from a linked list of C<addrinfo> elements, with the first element
pointed to by C<res>.
From the table of NativeCall Types we know that an C<int> is C<int32>.
We also know that a C<char *> is one of the forms C for a C C<Str>, which maps
simply to Str.
But C<addrinfo> is a structure, which means we
will need to write our own Type class. However, the function declaration is
straightforward:
=begin code :method :preamble<use NativeCall;
class SockAddr is repr('CStruct') {
has int32 $.sa_family;
has Str $.sa_data;
};
class Addrinfo is repr('CStruct') {
has int32 $.ai_flags;
has int32 $.ai_family;
has int32 $.ai_socktype;
has int32 $.ai_protocol;
has int32 $.ai_addrlen;
has SockAddr $.ai_addr is rw;
has Str $.ai_cannonname is rw;
has Addrinfo $.ai_next is rw;
};
>
sub getaddrinfo( Str $node, Str $service, Addrinfo $hints, Pointer $res is rw )
returns int32
is native
{ * }
=end code
Note that $res is to be written by the function, so it must be labeled as rw.
Since the library is standard POSIX, the library name can be the Type
definition or null.
We now have to handle structure Addrinfo. The Linux Manual provides this
information:
=begin code :lang<C>
struct addrinfo {
int ai_flags;
int ai_family;
int ai_socktype;
int ai_protocol;
socklen_t ai_addrlen;
struct sockaddr *ai_addr;
char *ai_canonname;
struct addrinfo *ai_next;
};
=end code
The C<int, char*> parts are straightforward. Some research indicates that C<socklen_t>
can be architecture dependent, but is an unsigned integer of at least 32 bits.
So C<socklen_t> can be mapped
to the C<uint32> type.
The complication is C<sockaddr> which differs depending on whether C<ai_socktype>
is undefined, INET, or INET6 (a standard v4 IP address or a v6 address).
So we create a Perl 6 C<class> to map to the C C<struct addrinfo>; while we're at it, we also create another class for C<SockAddr> which is needed for it.
=begin code
class SockAddr is repr('CStruct') {
has int32 $.sa_family;
has Str $.sa_data;
}
class Addrinfo is repr('CStruct') {
has int32 $.ai_flags;
has int32 $.ai_family;
has int32 $.ai_socktype;
has int32 $.ai_protocol;
has int32 $.ai_addrlen;
has SockAddr $.ai_addr is rw;
has Str $.ai_cannonname is rw;
has Addrinfo $.ai_next is rw;
}
=end code
The C<is rw> on the last three attributes reflects that these were defined in C to be pointers.
The important thing here for mapping to a C C<Struct> is the structure of the state part of the class, that is the attributes.
However, a class can have methods and C<NativeCall> does not 'touch' them for mapping to C.
This means that we can add extra methods to the class to unpack the attributes in a more
readable manner, e.g.,
=begin code :skip-test<should be compiled as part of Addrinfo defined above>
method flags {
do for AddrInfo-Flags.enums { .key if $!ai_flags +& .value }
}
=end code
By defining an appropriate C<enum>, C<flags> will return a string of keys
rather than a bit packed integer.
The most useful information in the C<sockaddr> structure is the address of
node, which depends on the family of the Socket. So we can add method
C<address> to the Perl 6 class that interprets the address depending on the
family.
In order to get a human readable IP address, there is the C function C<inet_ntop>
which returns a C<char *> given
a buffer with the C<addrinfo>.
Putting all these together, leads to the following program:
=begin code :solo
#!/usr/bin/env perl6
use v6;
use NativeCall;
constant \INET_ADDRSTRLEN = 16;
constant \INET6_ADDRSTRLEN = 46;
enum AddrInfo-Family (
AF_UNSPEC => 0;