Please join in with this interesting project - we need all the help we can get to optimize the code base - so much so that as an incentive to do so, we will be pleased to offer you free Perl and Assembler training to help you make such contributions to this project.
Generate and run X86-64 Advanced Vector Extensions assembler programs using NASM - the Netwide Assember and Perl
Generate and run Intel X86-64 Advanced Vector Extensions assembler programs using Perl as a powerful macro preprocessor for NASM - the Netwide Assember. The Perl module contained in this repository contains useful methods to help you quickly write and debug programs making this system an ideal development environment for any-one who wants to learn how to program effectively in X86-64 assembler code. Full documentation is available at Nasm::X86.
The GitHub Action in this repo shows how to install NASM - the Netwide Assember and the Intel Software Development Emulator used to assemble and then run the programs generated by this module. This repository includes an implementation of 6/13 multi-way trees using Advanced Vector Extensions instructions to perform key comparisons in parallel and relocatable data arenas that are used to contain other data structures. The efficient implementation of such multi-way trees and areas enables the efficient implementation of other dynamic data structures such as strings, stacks, arrays, maps and function libraries all of which are packed into relocatable areas and addressed via trees.
The use of relocatable areas allows data structures to be created in one program then mapped to files via virtual paging or to a socket to enable the data to be reused at a different location in memory by another program.
In particular position independent X86-64 code can be placed in such areas, indexed by a 6/13 tree and then reloaded as a library of functions for reuse elsewhere at a later date to make code generation efficient.
Such relocatable areas work well with parallel processing: each child sub task can run in a separate process that creates an area of dynamic data structures describing the results of the child's processing. The resulting areas can be easily transmitted to the parent process through a file or a socket and then interpreted by the parent regardless of the location in memory at which the child process created the dynamic data structures contained in the transmitted area.
A parser for the UniSyn programming language has been built in assembler using this software. UniSyn implements a generic, universal, utf8 based syntax suitable for constructing programming languages that make extensive use of infix operators. UniSyn enables the definition of new infix operators using selected Unicode points. Each such new infix operator may have one of 12 precedence levels. The precedence level for each infix operator is determined by the alphabet from within Unicode from which its letters are drawn.
For example, the infix operator: 𝕒𝕟𝕕 would have a priority of 3,
whilst 𝗮𝗻𝗱 would have a priority of 11.
The type of each lexical item in a UniSyn program can be determined immediately by examining any character used in its construction.
The canonical "Hello World" in Unisyn is:
Hello World
No quotes are needed because the use of letters drawn from Ascii indicate that
these characters are part of a string. UniSyn prints such strings on
stdout if they could have been preceded and followed by a statement
separator.
ParseUnisyn q(1＋𝗔✕𝗕＋𝗖𝕒𝕟𝕕2✕𝗔＋𝗕＋𝗖);A sample parse looks like this:
𝕒𝕟𝕕
._＋
._._✕
._._._＋
._._._._1
._._._._𝗔
._._._𝗕
._._𝗖
._＋
._._＋
._._._✕
._._._._2
._._._._𝗔
._._._𝗕
._._𝗖And a sample execution:
Ascii: 1
Variable: 𝗔
Add
Variable: 𝗕
Times
Variable: 𝗖
Add
Ascii: 2
Variable: 𝗔
Times
Variable: 𝗕
Add
Variable: 𝗖
Add
AndError tracing with Geany in Perl and Nasm
Get a helpful trace back that translates the location of a failure in a
generated Assembler program with the stack of Perl calls that created the
failing code.
Parse a Unisyn expression in assembly code using NASM - the Netwide Assember and Perl:
Parse a Unisyn expression to create a parse tree in assembly code using NASM - the Netwide Assember and Perl:
my ($s, $l) = constantString "𝗔＝【𝗕＋𝗖】✕𝗗𝐈𝐅𝗘"; # Unisyn expression
my $a = CreateArea; # Area in which we will do the parse
my $p = $a->ParseUnisyn($s, $l); # Parse the utf8 string
$p->tree->dumpParseTree($s); # Dump the parse tree
ok Assemble eq => <<END;
＝
._𝗔
._𝐈𝐅
._._✕
._._._【
._._._._＋
._._._._._𝗕
._._._._._𝗖
._._._𝗗
._._𝗘
ENDPrint some Fibonacci numbers in assembly code using NASM - the Netwide Assember and Perl:
Print the first 11 Fibonacci numbers in assembly code using NASM - the Netwide Assember and Perl:
my $N = 11; # The number of Fibonacci numbers to generate
Mov r13, 0; # First Fibonacci number
Mov r14, 1; # Second Fibonacci
PrintOutStringNL " i Fibonacci"; # The title of the piece
K(N => $N)->for(sub # Generate each Fibonacci number by adding the two previous ones together
{my ($index, $start, $next, $end) = @_;
$index->outRightInDec(2); # Index
Mov rax, r13;
PrintOutRightInDecNL rax, 12; # Fibonacci number at this index
Mov r15, r14; # Next number is the sum of the two previous ones
Add r15, r13;
Mov r13, r14; # Move up
Mov r14, r15;
});
ok Assemble eq => <<END; # Assemble and show expected output
i Fibonacci
0 0
1 1
2 1
3 2
4 3
5 5
6 8
7 13
8 21
9 34
10 55
ENDBinary search in assembly code using NASM - the Netwide Assember and Perl:
Search an array for a specified double word using binary search in assembly code using NASM - the Netwide Assember and Perl:
sub BinarySearchD($$) # Search for an ordered array of double words addressed by r15, of length held in r14 for a double word held in r13 and call the $then routine with the index in rax if found else call the $else routine.
{my ($then, $else) = @_; # Routine to call on matchParameters
my $array = r15, my $length = r14, my $search = r13; # Sorted array to search, array length, dword to search for
my $low = rsi, my $high = rdi, my $loop = rcx, my $range = rdx, my $mid = rax;# Work registers modified by this routine
Mov $low, 0; # Closed start of current range to search
Mov $high, $length; # Open end of current range to search
Cmp $high, 0; # Check we have a none empty array to search
IfEq
Then # Empty array
{Mov rax, -1; # Not found
&$else;
},
Else # Search non empty array
{Dec $high; # Closed end of current range
uptoNTimes # Search a reasonable number of times
{my ($end, $start) = @_; # End, start label
Mov $mid, $low; # Find new mid point
Add $mid, $high; # Sum of high and low
Shr $mid, 1; # Average of high and low is the new mid point.
Cmp dWordRegister($search), "[$array+$mid*4]"; # Compare current element of array with search
Pushfq; # Save result of comparison
IfEq
Then # Found
{Mov rax, $mid unless rax eq $mid;
&$then;
Jmp $end;
};
Mov $range, $high; # Size of remaining range
Sub $range, $low;
Cmp $range, 1;
IfLe
Then # Less than three elements in final range
{Cmp dWordRegister($search), "[$array+$high*4]"; # Compare high end of final range with search
IfEq
Then # Found at high end of final range
{Mov rax, $high;
&$then;
Jmp $end;
};
Cmp dWordRegister($search), "[$array+$low*4]"; # Compare low end of final range with search
IfEq
Then # Found at low end of final range
{Mov rax, $low;
&$then;
Jmp $end;
};
Mov rax, -1; # Not found in final range
&$else;
Jmp $end;
};
Popfq; # Restore results of comparison
IfGt # Search argument is higher so move up
Then
{Mov $low, $mid; # New lower limit
},
Else # Search argument is lower so move down
{Mov $high, $mid; # New upper limit limit
};
} $loop, 999; # Enough to search all the particles in the universe if they could be ordered by some means
};
}
for my $s(1..17)
{Mov r15, Rd(2, 4, 6, 8, 10, 12, 14, 16); # Address array to search
Mov r14, 8; # Size of array
Mov r13, $s; # Value to search for
PrintOutString sprintf "%2d:", $s;
BinarySearchD # Search
Then
{PrintOutString " <= "; PrintOutRaxInDec; PrintOutNL; # Found
},
Else
{PrintOutNL;
};
}
ok Assemble eq => <<END, mix=>1;
1:
2: <= 0
3:
4: <= 1
5:
6: <= 2
7:
8: <= 3
9:
10: <= 4
11:
12: <= 5
13:
14: <= 6
15:
16: <= 7
17:
END
# Test Clocks Bytes Total Clocks Total Bytes Run Time Assembler Perl
# 1 42 3_280 42 3_280 0.1425 0.02 0.00
# 2 2_240 41_336 2_282 44_616 0.1062 0.02 0.05Read lines from stdin and print them out on stdout in assembly code using NASM - the Netwide Assember and Perl:
Read lines of up to 8 characters delimited by a new line character from stdin and print them on stdout in assembly code using NASM - the Netwide Assember and Perl:
my $e = q(readWord);
my $f = writeTempFile("hello\nworld\n");
ReadLine;
PrintOutRaxAsTextNL;
ReadLine;
PrintOutRaxAsTextNL;
Assemble keep => $e;
is_deeply scalar(qx(./$e < $f)), <<END;
hello
world
ENDRead integers in decimal from stdin and print them out on stdout in decimal in assembly code using NASM - the Netwide Assember and Perl:
Read two integers from stdin in decimal, double them then print the doubled integers on stdout in decimal:
my $e = q(readInteger);
my $f = writeTempFile("11\n22\n");
ReadInteger;
Shl rax, 1;
PrintOutRaxInDecNL;
ReadInteger;
Shl rax, 1;
PrintOutRaxInDecNL;
Assemble keep => $e;
is_deeply scalar(qx(./$e < $f)), <<END;
22
44
ENDWrite Unicode characters in assembly code using NASM - the Netwide Assember and Perl:
Generate and write some Unicode utf8 characters:
K( loop => 16)->for(sub
{my ($index, $start, $next, $end) = @_;
$index->setReg(rax);
Add rax, 0xb0; Shl rax, 16;
Mov ax, 0x9d9d; Shl rax, 8;
Mov al, 0xf0;
PrintOutRaxAsText;
});
PrintOutNL;
ok Assemble(debug => 0, trace => 0, eq => <<END);
𝝰𝝱𝝲𝝳𝝴𝝵𝝶𝝷𝝸𝝹𝝺𝝻𝝼𝝽𝝾𝝿
ENDRead a file and print it out in assembly code using NASM - the Netwide Assember and Perl:
Read this file and print it out in assembly code using NASM - the Netwide Assember and Perl:
use Nasm::X86 qw(:all);
Mov rax, Rs $0; # Read this file
ReadFile;
PrintOutMemory; # Print memory occupied by file contents
my $r = Assemble; # Assemble and execute
ok index($r, readFile($0)) > -1; # Output contains this filePrint numbers in decimal in assembly code using NASM - the Netwide Assember and Perl:
Debug your programs quickly with powerful print statements in assembly code using NASM - the Netwide Assember and Perl:
Mov rax, 0x2a;
PrintOutRightInDecNL rax, 16;
PrintOutRightInHexNL rax, 16;
PrintOutRightInBinNL rax, 16;
ok Assemble eq => <<END, avx512=>1;
42
2A
101010
ENDCall functions in Libc in assembly code using NASM - the Netwide Assember and Perl:
Call C functions by naming them as external and including their library in assembly code using NASM - the Netwide Assember and Perl:
my $format = Rs "Hello %s\n";
my $data = Rs "World";
Extern qw(printf exit malloc strcpy); Link 'c';
CallC 'malloc', length($format)+1;
Mov r15, rax;
CallC 'strcpy', r15, $format;
CallC 'printf', r15, $data;
CallC 'exit', 0;
ok Assemble eq => <<END;
Hello World
ENDAvx512 instructions in assembly code using NASM - the Netwide Assember and Perl:
Use Advanced Vector Extensions instructions to compare 64 bytes at a time using the 512 bit wide zmm registers from NASM - the Netwide Assember and Perl:
my $P = "2F"; # Value to test for
my $l = Rb 0; Rb $_ for 1..RegisterSize zmm0; # The numbers 0..63
Vmovdqu8 zmm0, "[$l]"; # Load data to test
PrintOutRegisterInHex zmm0;
Mov rax, "0x$P"; # Broadcast the value to be tested
Vpbroadcastb zmm1, rax;
PrintOutRegisterInHex zmm1;
for my $c(0..7) # Each possible test
{my $m = "k$c";
Vpcmpub $m, zmm1, zmm0, $c;
PrintOutRegisterInHex $m;
}
Kmovq rax, k0; # Count the number of trailing zeros in k0
Tzcnt rax, rax;
PrintOutRegisterInHex rax;
is_deeply [split //, Assemble], [split //, <<END]; # Assemble and test
zmm0: 3F3E 3D3C 3B3A 3938 3736 3534 3332 3130 2F2E 2D2C 2B2A 2928 2726 2524 2322 2120 1F1E 1D1C 1B1A 1918 1716 1514 1312 1110 0F0E 0D0C 0B0A 0908 0706 0504 0302 0100
zmm1: 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F 2F2F
k0: 0000 8000 0000 0000 # Equals
k1: FFFF 0000 0000 0000 # Less than
k2: FFFF 8000 0000 0000 # Less than or equal
k3: 0000 0000 0000 0000
k4: FFFF 7FFF FFFF FFFF # Not equals
k5: 0000 FFFF FFFF FFFF # Greater then or equals
k6: 0000 7FFF FFFF FFFF # Greater than
k7: FFFF FFFF FFFF FFFF
rax: 0000 0000 0000 00$P
ENDCreate a library in assembly code using NASM - the Netwide Assember and Perl:
Create a library with three routines in it and save the library in a file in assembly code using NASM - the Netwide Assember and Perl:
my $library = CreateLibrary # Library definition
(subroutines => # Sub routines in libray
{inc => sub {Inc rax}, # Increment rax
dup => sub {Shl rax, 1}, # Double rax
put => sub {PrintOutRaxInDecNL}, # Print rax in decimal
},
file => q(library),
);
Reuse the code in the library in another assembly: Libraries can be created in one assembly and saved as an area in a file, then reused in a subsequent assembly by reading the area into memory or including the area into the assembly directly.
For example, to create a library of operators to process the parse tree of a UniSyn programming language statement:
my $f = "zzzOperators.lib"; # Methods to be called against each syntactic item
my $library = Subroutine # This subroutine and all of the subroutines it contains will be saved in an area and that area will be written to a file from where it can be included via L<incBin> in subsequent assemblies.
{my ($p, $s, $sub) = @_;
Subroutine # A contained routine that we wish to export to a file
{my ($p, $s, $sub) = @_;
PrintOutString "Ascii: ";
my $parse = $$s{parse}; # Parse
my $source = $parse->source; # Source
$parse->area->getZmmBlock($$p{offset}, 1); # Load current parse tree node
my $w = dSize;
my $length = dFromZ(1, $w * Nasm::X86::Unisyn::Lex::length); # Length of ascii
my $position = dFromZ(1, $w * Nasm::X86::Unisyn::Lex::position); # Position in source
($source+$position)->printOutMemoryNL($length); # Print the ascii string
} name => "Ascii",
structures => {parse => Nasm::X86::Unisyn::DescribeParse},
parameters => [qw(offset)];
Subroutine # Another subroutine that will be exported because it is within the subroutine that is being exported as a library
{my ($p, $s, $sub) = @_;
PrintOutStringNL "Add";
} name => "＋",
structures => {parse => Nasm::X86::Unisyn::DescribeParse},
parameters => [qw(offset)];
Subroutine
{my ($p, $s, $sub) = @_;
PrintOutStringNL "Times";
} name => "✕",
structures => {parse => Nasm::X86::Unisyn::DescribeParse},
parameters => [qw(offset)];
Subroutine
{my ($p, $s, $sub) = @_;
PrintOutStringNL "And";
} name => "𝕒𝕟𝕕",
structures => {parse => Nasm::X86::Unisyn::DescribeParse},
parameters => [qw(offset)];
Subroutine
{my ($p, $s, $sub) = @_;
PrintOutString "Variable: ";
my $parse = $$s{parse}; # Parse
my $source = $parse->source; # Source
$parse->area->getZmmBlock($$p{offset}, 1); # Load current parse tree node
my $w = dSize;
my $length = dFromZ(1, $w * Nasm::X86::Unisyn::Lex::length); # Length of ascii
my $position = dFromZ(1, $w * Nasm::X86::Unisyn::Lex::position); # Position in source
($source+$position)->printOutMemoryNL($length); # Print the ascii string
} name => "Variable",
structures => {parse => Nasm::X86::Unisyn::DescribeParse},
parameters => [qw(offset)];
} name => "operators", parameters=>[qw(a b c)], export => $f;
ok Assemble eq => <<END, avx512=>1;
ENDThe second assembly reuses the library created in the first assembly by reading the area that contains the library into memory so that the contained subroutines can be executed as needed while traversing a parse tree:
my $l = ReadArea $f; # Area containing subroutine library
my ($A, $N) = constantString qq(1＋𝗔✕𝗕＋𝗖𝕒𝕟𝕕2✕𝗔＋𝗕＋𝗖); # Utf8 string to parse
my $p = ParseUnisyn($A, $N); # 10_445 Parse utf8 string 5_340 after single character lexical items, 4_950 after jump table
$p->dumpParseResult;
$p->traverseApplyingLibraryOperators($l); # Traverse a parse tree applying a library of operators where they intersect with lexical items in the parse tree
ok Assemble eq => <<END, clocks=>20_921;
parseChar : .... .... ...1 D5D6
parseFail : .... .... .... ...0
position : .... .... .... ..38
parseMatch : .... .... .... ...0
parseReason: .... .... .... ...0
𝕒𝕟𝕕
._＋
._._✕
._._._＋
._._._._1
._._._._𝗔
._._._𝗕
._._𝗖
._＋
._._＋
._._._✕
._._._._2
._._._._𝗔
._._._𝗕
._._𝗖
Ascii: 1
Variable: 𝗔
Add
Variable: 𝗕
Times
Variable: 𝗖
Add
Ascii: 2
Variable: 𝗔
Times
Variable: 𝗕
Add
Variable: 𝗖
Add
And
ENDCreate a 6/13 multi way tree in an area using SIMD instructions in assembly code using NASM - the Netwide Assember and Perl:
Create a 6/13 multiway tree using Avx512 instructions then iterate through the tree each time an element is deleted in assembly code using NASM - the Netwide Assember and Perl:
my $a = CreateArea;
my $t = $a->CreateTree;
my $N = K loop => 16;
$N->for(sub
{my ($i) = @_;
$t->put($i, $i);
});
$t->printInOrder(" 0"); $t->delete( 0);
$t->printInOrder(" 2"); $t->delete( 2);
$t->printInOrder(" 4"); $t->delete( 4);
$t->printInOrder(" 6"); $t->delete( 6);
$t->printInOrder(" 8"); $t->delete( 8);
$t->printInOrder("10"); $t->delete(10);
$t->printInOrder("12"); $t->delete(12);
$t->printInOrder("14"); $t->delete(14);
$t->printInOrder(" 1"); $t->delete( 1);
$t->printInOrder(" 3"); $t->delete( 3);
$t->printInOrder(" 5"); $t->delete( 5);
$t->printInOrder(" 7"); $t->delete( 7);
$t->printInOrder(" 9"); $t->delete( 9);
$t->printInOrder("11"); $t->delete(11);
$t->printInOrder("13"); $t->delete(13);
$t->printInOrder("15"); $t->delete(15);
$t->printInOrder("XX");
ok Assemble eq => <<END;
0 16: 0 1 2 3 4 5 6 7 8 9 A B C D E F
2 15: 1 2 3 4 5 6 7 8 9 A B C D E F
4 14: 1 3 4 5 6 7 8 9 A B C D E F
6 13: 1 3 5 6 7 8 9 A B C D E F
8 12: 1 3 5 7 8 9 A B C D E F
10 11: 1 3 5 7 9 A B C D E F
12 10: 1 3 5 7 9 B C D E F
14 9: 1 3 5 7 9 B D E F
1 8: 1 3 5 7 9 B D F
3 7: 3 5 7 9 B D F
5 6: 5 7 9 B D F
7 5: 7 9 B D F
9 4: 9 B D F
11 3: B D F
13 2: D F
15 1: F
XX- empty
ENDProcess management in assembly code using NASM - the Netwide Assember and Perl:
Start a child process and wait for it, printing out the process identifiers of each process involved in assembly code using NASM - the Netwide Assember and Perl:
use Nasm::X86 qw(:all);
Fork; # Fork
Test rax,rax;
If # Parent
{Mov rbx, rax;
WaitPid;
PrintOutRegisterInHex rax;
PrintOutRegisterInHex rbx;
GetPid; # Pid of parent as seen in parent
Mov rcx,rax;
PrintOutRegisterInHex rcx;
}
sub # Child
{Mov r8,rax;
PrintOutRegisterInHex r8;
GetPid; # Child pid as seen in child
Mov r9,rax;
PrintOutRegisterInHex r9;
GetPPid; # Parent pid as seen in child
Mov r10,rax;
PrintOutRegisterInHex r10;
};
my $r = Assemble; # Assemble test and run
# r8: 0000 0000 0000 0000 #1 Return from fork as seen by child
# r9: 0000 0000 0003 0C63 #2 Pid of child
# r10: 0000 0000 0003 0C60 #3 Pid of parent from child
# rax: 0000 0000 0003 0C63 #4 Return from fork as seen by parent
# rbx: 0000 0000 0003 0C63 #5 Wait for child pid result
# rcx: 0000 0000 0003 0C60 #6 Pid of parent