truffle-arrow-language is a language processing for Apache Arrow and built using Truffle for GraalVM.
truffle-arrow-language is developed as a language for a code generation by a query engine.
This is a sample code for filter-aggregation.
$map = {}; // for aggregation
$out = arrays([key:INT, value:INT]); // for output
load ("target/all_fields.arrow") {
// Allow to reference the field value (F_INT) directly in loop
if ($F_INT < 5) { // filter
$map[$F_INT] = get($map[$F_INT], 0) + 1; // aggregation
}
}
store($out, $map); // storing the result as RecordBatch(VectorSchemaRoot)
return $out;