Pipefish: A functional and indentation sensitive programming language on BenchGen
Pipefish is a functional, indentation sensitive programming language. It is implemented in Go and aims to facilitate the development of business applications, distributed services, and Domain-Specific Languages (DSLs).
Pipefish is primarily a functional language. It provides support for imperative programming through mutable variables and side effects; however, unlike "multi-paradigm" languages, Pipefish syntactically and semantically separates the purely functional parts of a program from its imperative components.
Pipefish adopts the Functional Core, Imperative Shell paradigm. This means that the business logic the rules governing data processing resides in a pure functional core free from side effects. Only the outer layers of the application use an imperative shell to handle input/output (I/O) operations.
The language prioritizes immutability by default. In Pipefish, all values are immutable, eliminating common bugs related to mutable global state. Values are compared by their content rather than by reference, ensuring consistent and predictable behavior throughout program execution.
Pipefish was designed to facilitate the creation of Domain-Specific Languages (DSLs). The idea is that the resulting service should not merely act as an opaque backend, but also provide an interface for end users, resembling the fluency and readability of an SQL query.
Integrating Pipefish into BenchGen introduced several challenges, primarily because it was necessary to reconcile the growth logic of L-systems which is structured and based on the rewriting of nested blocks and, in many cases, imperative constructs. For example, BenchGen would generate, for the C language, a program similar to the code shown below:
1 if(PATH0 & 0x1)
2 {
3 if(PATH0 & 0x2){
4 Array* array1 = (Array*)malloc(sizeof(Array));
5 for(int i = 0; i < array1.data.size; i++)
6 {
7 array1.data[i] = (int)rand();
8 }
9 }
10 }else{
11 Array* array1 = (Array*)malloc(sizeof(Array));
12 }The equivalent Pipefish program is:
1 def func_4(array1 Array,path_0) -> Array :
2 path_0 and 0x1
3 path_0 and 0x2
4 array1 -> initialize -> insert
5 else
6 array1 -> initialize
7 else:
8 array1 -> initialize
It is important to note that BenchGen was designed with an imperative logic: the generated benchmarks extensively explore mutability and control flow based on loops. In Pipefish, however, such constructs are not encouraged. Pipefish imposes strict rules regarding immutability and referential transparency, making the generation of synthetic code more challenging than in traditional programming languages.
Furthermore, because Pipefish uses indentation as a fundamental syntactic element, our generator cannot simply produce valid commands; it must precisely manage the spacing and hierarchy of every block. Any failure in indentation management during the recursive expansion of the L-system results in syntax errors. Consequently, adaptations were required to ensure that the generated code is not only functional but also semantically coherent and compliant with the language's indentation sensitivity.
To address these challenges, we first created a counter that is incremented whenever a construct requiring indentation is opened, such as an if-then-else, for, function, and so on. This value represents the number of indentation levels that each subsequent line must have until the block is closed. Once the block is finished, the counter is decremented by one.
For illustration purposes, we use a Python example:
1 def main(): # [Counter initialized, counter=0]
2 if PATH0 & 0x1: # [If block detected, counter++, counter=1]
3 if PATH0 & 0x2: # [If block detected, counter++, counter=2]
4 for i in range(data.size): # [For block detected, counter++, counter=3]
5 data[i] = Array() # [Inside for block, counter++, counter=4]
6 else: # [End of for block, counter--, counter=3] & [End of if block, counter--, counter=2]
7 array1 = Array() # [Inside else block, counter++, counter=3]
8 else: # [End of if-then-else block, counter--, counter=2] & [End of if block, counter--, counter=1]
9 array1 = Array() # [Inside else block, counter++, counter=2]
... # [End of if-then-else block, counter--, counter=1] & ...This algorithm is applied line by line and proves to be extremely efficient, since the only data structure required is a single variable that accumulates the indentation level and ultimately produces fully indented code.
The experiments were conducted on a machine with the following specifications:
| Resource Type | Details |
|---|---|
| Processor | AMD Ryzen Threadripper 7970X 32-Cores |
| Memory | 128 GB |
| Disk | 915 GB |
The following programs were used for the experiments and result collection:
| Program name | Program pipefish |
|---|---|
| ex1 | program |
| ex2 | program |
| ex3 | program |
| ex4 | program |
| ex5 | program |
| ex6 | program |
The graph below presents a comparison between six generations of C programs compiled with optimization level -O0 and six generations of Pipefish interpreted programs, both following the same execution flow.
The C programs were executed using the following command:
gcc -O0 ex1.c -o out && ./outThe Pipefish programs were executed using:
pipefish run ex1.pfAs expected, the C programs are faster. However, it is important to note that we are comparing a compiled language with an interpreted language. Therefore, Pipefish execution times also include the overhead of parsing and generating the program's intermediate representation.
The results show that C's performance advantage over Pipefish starts at 34% in the first iteration and remains remarkably stable across subsequent iterations, with a variation of only 0.86 percentage points between the highest and lowest observed gains. This stability indicates that the relative efficiency of both languages remains essentially unchanged, even as the program scales and code complexity increases in terms of lines of code.
Since Pipefish does not perform explicit optimizations and the C code was compiled with the -O0 flag (no optimizations), the execution time difference provides an isolated view of the overhead associated with each environment: the native execution of the binary generated by C versus the interpretation cost incurred by the Pipefish interpreter.
On the other hand, when considering code writing efficiency, Pipefish achieves better results than C. For the generated programs, Pipefish exhibits a reduction of 2,620 lines of code in iteration 6, as illustrated by the graph below.
This is possible because Pipefish provides a syntax that encourages code reuse. In a multi-paradigm language such as C, multiple code fragments inside a function become part of that function itself. In Pipefish, these fragments are often transformed into reusable functions. Example 1 illustrates this behavior with variable initialization. In the C code, variables are declared directly within the function body (lines 4 and 11). In Pipefish, however, an initialization function is created, and only a sequence of function invocations is written on the same line, such as array1 -> initialize -> insert, as shown in line 4 of the Pipefish example.