Browse files


  • Loading branch information...
1 parent d66c52d commit 93417bd51ebd213447ddc948e9b766bba1dcb086 @duncantl committed Jul 14, 2013
Showing with 379 additions and 81 deletions.
  1. +1 −1 NAMESPACE
  2. +5 −4 Paper/OtherFuncs
  3. +307 −59 Paper/Rllvm.tex
  4. +8 −2 R/globalVar.R
  5. +4 −3 R/module.R
  6. +12 −6 R/ostream.R
  7. +4 −1 R/targetMachine.R
  8. +18 −3 inst/TU/clang.R
  9. +6 −1 inst/TU/llvm.c
  10. +13 −0 src/Triple.cpp
  11. +1 −1 src/ostream.cpp
@@ -365,7 +365,7 @@ exportClasses(raw_string_ostream, formatted_raw_ostream)
export(addAnalysisPasses, addPassesToEmitFile)
-export(rawFDOstream, formattedRawOstream, stringRawOstream)
+export(rawFDOstream, formattedRawOstream, rawStringOstream)
9 Paper/OtherFuncs
@@ -31,12 +31,12 @@ Pass manager
addPassesToEmitFile, stringRawOstream, rawFDOstream, formattedRawOstream
Different targets:
- getLLVMTargets
+ [done] getLLVMTargets
- InitializeNVPTXTarget, InitializeCppBackendTarget
+ [done] InitializeNVPTXTarget, InitializeCppBackendTarget
System-level commands
@@ -45,7 +45,7 @@ System-level commands
-Explain module instance is specific to an execution engine
+[Done] Explain module instance is specific to an execution engine
@@ -54,7 +54,8 @@ Module
getModuleFunctions, getModuleGlobals
- global variables, initializer
+ [done] global variables, initializer
366 Paper/Rllvm.tex
@@ -89,24 +89,42 @@
\section{Overview and Motivation}\label{sec:Introduction}
-The potential readers of this paper are people who are interested in
+High-level languages such as \R{} generally improve human efficiency
+in writing code, but suffer from slower performance of the code
+relative to low-level compiled/native code. The ideal situation is to
+be able to develop code in a high-level, interpreted/dynamic language
+but to convert that code into native machine code that executes very
+quickly. There are several different projects exploring approaches to
+compiling \R{} code, most of them using just-in-time compilation
+within the interpreter. In this paper, we'll explore low-level
+\R-level tools that allow us to generate native code directly within
+\R. This allows us to develop compilation strategies within \R{} for
+\R{} code or any other language. We describe the \Rpkg{Rllvm} package
+which is an interface to the Low-level Virtual Machine (\llvm).
+\llvm{} is a compiler toolkit, that is a \Cpp{} library that allows us
+to build compilers for arbitrary languages.
+The target readers of this paper are people who are interested in
exploring how to create machine code directly within R for either
-compiling R code or for developing domain specific languages in R.
-We are using this to compile simple functions in R for speed
-and also to get them to run on the GPU to take advantage
-of massively parallel architectures.
+compiling R code or for developing domain specific languages in R. We
+are using this to compile simple functions in R for speed and also to
+get them to run on the GPU to take advantage of massively parallel
This is more of a manual about how to program with the
-\Rpkg{Rllvm}~\cite{bib:Rllvm} package than what we can do with it our
-how we compile \R{}~\cite{bib:R} code. We will explain some of this
-using example code from the \Rpkg{RLLVMCompile}~\cite{bib:RLLVMCompile} package. However, this
-is not a paper that describes that package and its compilation
+\Rpkg{Rllvm}~\cite{bib:Rllvm} package and how to generate native code
+in \R. We do not focus on how to compile arbitrary \R{} code. We
+leave that to a paper on the
+\Rpkg{RLLVMCompile}~\cite{bib:RLLVMCompile} package. In this paper,
+we will use examples that compile \R{} code, but the intent is to
+illustrate \Rpkg{Rllvm}.
In many respects, this article is an overview of what the \llvm~\cite{bib:llvm}
API provides for applications which use it. We describe the concepts,
classes and methods in that API, but in terms of how we manipulate these
in \R.
+We'll show how to create
We will show how we can call existing \C{} routines using a
combination of \RClang{} and \Rllvm. We also show how we can compile
@@ -121,7 +139,7 @@ \section{Overview and Motivation}\label{sec:Introduction}
(This is done for the sort routine in C code in qsort.c by redefining the
-Show how to make code use multiple cores directly, with shared data?
+%Show how to make code use multiple cores directly, with shared data?
Some readers may say the examples are unrealistic as \R{} already has
fast code to implement, e.g., \Rfunc{sum} and so on. The point is not
@@ -139,14 +157,28 @@ \section{Overview and Motivation}\label{sec:Introduction}
either for \R{} code or other domain specific languages (DSLs) within
We'll use examples from the ``Writing R extensions manual'' also used
in the \Rpkg{Rcpp}~\cite{bib:Rcpp} package to facilitate comparison.
The focus of the paper is generating native code in \R{} using \llvm.
However, before we do this, we have to understand the essential
-concepts of \llvm. Therefore, we start by exploring \llvm{} by
-working with previously generated code. %We will see how we can We may
+elements and concepts of \llvm. Therefore, we start by exploring
+\llvm{} by working with previously generated code that is available in
+\llvm's intermediate representation (IR) form. We then turn our focus
+to generating code in \ref{sec:GeneratingCode}. This provides an
+overview of the basic functionality and classes. In
+\ref{sec:Examples}, we illustrate all of these concepts in an example
+to create a native version of the Normal density function.
+We follow this with a description some additional functionality
+in \Rpkg{Rllvm}.
+%We will see how we can We may
%not regulary need all of these tools for querying \llvm{} objects when
%we generate code, but they allow us to
@@ -482,8 +514,8 @@ \subsection{IRBuilder}
% Constants
-\subsection{Creating Instructions}
-There are many
+%\subsection{Creating Instructions}
@@ -493,76 +525,159 @@ \subsection{Branching}
a branch with \Rfunc{createBranch}, passing it the \Rclass{IRBuilder}
and the block to which we want to jump. This adds the branch
instruction at the end of the current block in the \Rclass{IRBuilder}.
+In many cases, we use a conditional branch.
+This takes a condition value and two blocks.
+If the condition evaluates to true when the instructions are
+the code jumps to the first block. Otherwise, control is transferred to the second block.
+We create this with \Rfunc{createCondBranch}.
\subsection{Intermediate Representation and viewing the code}
% Seen this above
-write code to file.
-read it into R.
+When we have a module, either by reading it from a file or creating it
+and its contents ourselves as we'll see below, we can query it in
+different ways. We have seen how to find the names of the routines or
+non-local variables it contains with a \Rfunc{names} method.
+We also saw how to access routines so that we could invoke them.
+We can also see the IR code in the module with \Rfunc{showModule}.
+This displays the entire contents of the module -- all its routines,
+their blocks and individual instructions and also any metadata in
+the module.
+It is useful to be able to validate a module and ensure that the code
+is valid. The function \Rfunc{verifyModule} does this
+and returns \Rtrue{} if the code is valid.
%\section{Verifying the generated code}
-\section[Connecting LLVM and existing native routines]{Connecting \llvm{} and existing native routines}
+% See below.
+%\section[Connecting LLVM and existing native routines]{Connecting \llvm{} and existing native routines}
% Registering native symbols.
-Passes, pass manager
+\subsection{Optimizing code}
+%Passes, pass manager
+Before we invoke any of the generated routines, we have to compile
+them. This happens automatically when we call \Rfunc{.llvm} or
+\Rfunc{run}. However, we can explicitly compile the routines in a
+module. The function \Rfunc{Optimize} compiles the code. We can
+compile an individual routine (\Rfunc{Function}) or we can compile the
+entire module. It is advisable to create the execution engine with
+which you will invoke the routines and to explicitly provide
+this when calling \Rfunc{Optimize}.
+We can also specify the compilation level to control
+how much optimization is performed.
\subsection{Global Variables}
+We can create global, or more precisely, module-level
+variables with \Rfunc{createGlobalVariable}.
+We specify the name of the variable, the module in which it is
+to be located and the type.
+We can also specify the intial value for the variable.
+(The type is taken from this initial value if the \Rarg{type}
+parameter is not specified in the call.)
+For example, to create an integer variable named \Cvar{counter}
+and a character array with $1000$ elements named \Cvar{line},
+we could use
+mod = Module()
+createGlobalVariable('counter', mod, type = Int32Type,
+ val = createIntegerConstant(0L))
+createGlobalVariable('line', mod, val = string(1000))
+We can also use element assignment as if the
+\Rclass{Module} was a list and use something like
+mod[['counter']] = 0L
+This is merely syntactic convenience. Note
+that \Rfunc{createGlobalVariable} inferred the type
+We can specify the intial value of an existing
+global variable with \Rfunc{setInitializer}.
+This allows us to create the variable in one command
+and intialize it in a separate command.
+For example,
+var = createGlobalVariable("bob", mod, Int32Type)
+setInitializer(var, 2L)
-%clone the module, create new ones with different execution engines.
+Note that if we create a global variable with the name of an existing
+variable in the \Rclass{Module}, \llvm{} will modify the name of the
+new variable (by adding a number to it). This doesn't really affect
+us as we won't refer to the variable by name in our code. Instead, we
+use the object returned by \Rfunc{createGlobalVariable}.
-\section[General LLVM functionality]{General \llvm{} functionality}
-In addition to the specific functions for generating code,
-there are some simple functions in \Rllvm{} that
-allow us to control the \llvm{} and also find the
-version information.
-We can determine the version of the \llvm{} libraries
-with which the \R{} package is compiled with
+We can retrieve a reference to a global variable with
+\Rfunc{getGlobalVariable} or with the \Rop{[[} syntax, e.g.
+getGlobalVariable(mod, "counter")
-This can be important to determine what features are available
-and what forms of code we can read.
+This is actually a reference or pointer to the integer.
+We can verify this with
+which yields a \texttt{PointerTyID} type.
-We can also determine the different targets
-for which we can generate code via the \Rfunc{getLLVMTargets}
-function. This information is determined at installation time
-and returned as a character vector via
+If we want the actual value of the global variable,
+we can use \Rfunc{getGlobalValue} on the reference
+returned by \Rfunc{getGlobalVariable}.
+Alternatively, we can use
+mod[['counter', value = TRUE]]
- [1] "X86" "Sparc" "PowerPC" "ARM" "Mips"
- [6] "CellSPU" "XCore" "MSP430" "CppBackend" "MBlaze"
-[11] "NVPTX" "Hexagon"
-Here we see that we can generate code for various different CPUs
-and also \Cpp{} and \code{PTX} code for Nividia GPUs (Graphics
-Processor Units).
+In both cases, we need an \Rclass{ExecutionEngine} object
+in which to interpret the module and the global variables.
+The subset method will provide a default value, if necessary,
+but this is expensive.
+We can provide an engine in the call as
+mod[['counter', value = TRUE, ee = engine]]
+where \Rvar{engine} is an \Rclass{ExecutionEngine} object.
-Other general top-level functions include
-shutdown, start/stop multi-threaded.
+Note that \Rop{\$} is used to retrieve routines from a \Rclass{Module}
+which \Rop{[[} used for accessing variables.
-\section{Examples and Case Studies}
+It can be important to understand that a global variable in a
+\Rclass{Module} needs to be instantiated within a particular
+\Rclass{ExecutionEngine}. If we access a global variable without an
+explicit \Rclass{ExecutionEngine}, \Rpkg{Rllvm} will create a new
+instance of the global variable and then discard it. This can be
+confusing. However, it can also be useful. It provides a way for us
+to create closures in compiled code without using fully global
+variables. We can create a \Rclass{Module} which has one more
+routines that access shared non-local variables that are in the
+\Rclass{Module}. We can then create separate instances of that
+\Rclass{Module} each with a different \Rclass{ExecutionEngine}. Each
+\Rclass{ExecutionEngine} will have duplicate code, but each will have
+its own separate copy of the non-local variables. This allows them to
+operate independently of each other but still maintain state across
+calls within a module.
+%\section{Examples and Case Studies}
\section{Linking to other routines}
In generating an \llvm{} routine, we sometimes call another routine
-that is available in a library or \R{} itself. In order for \llvm{} to
+that is available in a library or \R{} itself.
+To make a call to this routine, we first have to provide \llvm{}
+with the signature of the routine.
+We can use \Rfunc{declareFuncion}
+In order for \llvm{} to
be able to access this routine, we have to provide it with the address
of the routine. We do this with \Cfunc{llvmAddSymbol}. We can pass
it either the name of the external routine or we can explicitly
@@ -873,23 +988,156 @@ \section{Navigating the instruction hierarchy}
\section[Querying LLVM]{Querying \llvm}
-It is sometimes useful to be able to query and deploy aspects
-of the \llvm{} API itself.
-For instance, we can determine the version with
+It is sometimes useful to be able to query and deploy aspects of the
+\llvm{} API itself. For instance, we can determine the version with
+\Rfunc{llvmVersion}. This can be important to determine what features
+are available and what forms of code we can read.
We can also determine which backend-targets were enabled
-when the \llvm{} distribution was compiled and installed.
+when the \llvm{} distribution was compiled and installed
+and for which we can generate code via the \Rfunc{getLLVMTargets}
We do this with \Rfunc{getLLVMTargets}, which returns
a character vector naming the different targets, e.g.
[1] "X86" "Sparc" "PowerPC" "AArch64"
[5] "ARM" "Mips" "XCore" "MSP430"
[9] "CppBackend" "MBlaze" "NVPTX" "Hexagon"
+Here we see that we can generate code for various different CPUs
+and also \Cpp{} and \code{PTX} code for Nividia GPUs (Graphics
+Processor Units).
We can enable any of these backends with a call to
\Rfunc{Initialize*Target} where \texttt{*} is replaced
with the name of the target, e.g. \texttt{NVPTX}.
+%Other general top-level functions include shutdown, start/stop multi-threaded.
+\section{Generating code for different backends}
+Up to this point, we have described generating routines
+that we can then invoke on the same machine, even in the
+same \R{} session.
+As we saw in the previous section, however, \llvm{}
+has support for different targets or back-ends.
+We can generate code for these different targets.
+The basic mechanism is the same for the different targets.
+For example, we might compile a routine to run on a GPU.
+The target here is \code{NVPTX} - the NVIDA Parallel Thread Execution.
+To generate code for a backend, we start with a \Rclass{Module} containing the code we generated
+to define our routines and any global variables.
+We first initialize our target. For NVPTX, we call
+To generate the code for this target, we need to create
+an object that knows about this target. This is a combination
+of an architecture, environment, operating system, etc.
+Some of these are described by what is called a \Rclass{Triple}.
+We get the target triple object via
+tri = getDefaultTargetTriple()
+Next, we combine the triple and the architecture to
+get the target object:
+trgt = lookupTarget(tri, 'nvptx64')
+This returns an object of class \Rclass{Target}. This is an abstract
+layer that provides methods/routines specific to this particular
+architecture and triple for constructing objects that will create the
+code. We use this to create what \llvm{} terms a
+We call \Rfunc{createTargetMachine} for this,
+passing it the \Rclass{Target}, the triple and
+and string identifying the type of processor (i.e. CPU).
+In the case of a GPU, different processors have different
+capabilities and we need to tell the code generator
+which capabilities we have. We'll use \texttt{sm_20}
+to indicate
+machine = createTargetMachine(trgt, tri, "sm_20")
+Onece we have the abstract target machine object,
+we create a \Rclass{PassManager} which will generate the
+code for our target.
+We call the \Rfunc{PassManager} function to create an instance
+of this pass manager, passing it the module we want to compile.
+We add three different \Rclass{Pass} steps.
+The first provides information about what library routines
+are available for this target.
+We create this library information description for the target with
+\Rfunc{targetLibraryInfo}, passing it the target triple.
+We then add it to the pass manager. The two steps are
+trgtLibInfo = targetLibraryInfo(tri)
+addPass(pm, trgtLibInfo)
+Next we add a a sequence of passes that analyze the code.
+addAnalysisPasses(machine, pm)
+The final pass we add describes the
+data layout characteristics of the target machine.
+We retrieve the data layout from the target machine and
+then call \Rfunc{addPass}
+dataLayout = getDataLayout(machine)
+addPass(pm, dataLayout)
+Before we run these passes on our module, we want to also tell the
+pass manager where to write the code. We can write to either a file
+or to memory by creating different types of streams. The function
+\Rfunc{rawFDOstream} creates a stream to write to a file.
+Alternatively, we create an in-memory stream with
+Regardless of the type of stream, we create a higher-level
+stream that formats the generated code.
+We call \Rfunc{formattedRawOstream} with our lower-level raw stream.
+Once we have the formatted stream, we call
+\Rfunc{addPassesToEmitFile} to have the pass manager write the code
+this stream.
+So the sequence of calls is similar to
+stream = rawStringOstream()
+fstream = formattedRawOstream(stream)
+status = addPassesToEmitFile(machine, pm, fstream, 0L)
+Note that we explicitly create the in-memory stream and assign it to
+an \R{} variable. This allow us to retrieve the generated code later.
+If we are writing to a file, we can create the stream without
+assigning it to an \R{} variable as the code will be written directly
+to that file.
+Regardless of the type of stream, it is (currently) important to explicitly
+assign the formatted stream to a variable so that it is not garbage
+collected until after the code has been generated.
+We now have specified the entire sequence of passes. We can apply
+them to the code by calling the \Rfunc{run} method for the
+\Rclass{PassManager} and \Rclass{module}:
+run(pm, m)
+Assuming there are no errors when applying the passes,
+we can access the generated code from our in-memory stream
+as(stream, 'character')
+If we are writing to a file, it is important to call
+\Rfunc{flush} on the stream or to implictly call \Rfunc{flush}
+by garbage collecting the formatted stream (\Rvar{fstream} above).
% Sharing code with other systems.
\section[Exporting compiled R functions to other languages]{Exporting compiled \R{} functions to other languages}
10 R/globalVar.R
@@ -30,7 +30,8 @@ function(id, mod, type = getType(val), val = NULL, # guessType(val),
type = arrayType(getIntegerType(8L, getContext(mod)), nchar(txt) + 1L) # getType(val) ??
alignment = 1L
- }
+ } else if(is(val, "numeric"))
+ val = createConstant(, val, type, getContext(mod))
if(!is(val, "Constant"))
stop("val must be an object of class Constant")
@@ -63,7 +64,12 @@ function(val)
setInitializer =
function(var, value)
- if(!is(value, "Constant"))
+ if(!is(value, "Constant")) {
+ type = getElementType(getType(var))
+ value = createConstant(, value, type, context = getContext(as(var, "Module")))
+ }
+ if(!is(value, "Constant"))
stop("Need a Constant value")
if(!is(var, "GlobalVariable"))
7 R/module.R
@@ -22,18 +22,19 @@ setGeneric("Optimize",
setMethod("Optimize", "Module",
- function(obj, execEngine = NULL, ...)
+ function(obj, execEngine = NULL, mgr = getPassManager(obj, execEngine, level), level = 3L...)
# should we run optimize on the module. Could it do better in the larger context.
fun = getModuleFunctions(obj)
- mgr = getPassManager(obj, execEngine)
lapply(fun, Optimize, mgr)
c("Function"), # "FunctionPassManager"), # the llvm Function object
- function(obj, passManager = getPassManager(getModule(obj)), ...) {
+ function(obj, execEngine = ExecutionEngine(as(obj, "Module")),
+ passManager = getPassManager(getModule(obj), execEngine, level = level), level = 3L, ...) {
.Call("R_optimizeFunction", obj, passManager)
18 R/ostream.R
@@ -9,10 +9,14 @@ function(filename)
formattedRawOstream =
-function(stream, delete = FALSE, finalize = TRUE)
+function(stream = character(), delete = FALSE, finalize = TRUE)
- if(is.character(stream))
- stream = rawFDOstream(stream)
+ if(is.character(stream)) {
+ if(length(stream) && !is(stream, "AsIs"))
+ stream = rawFDOstream(stream)
+ else
+ stream = rawStringOstream(stream)
+ }
ans = .Call("R_new_formatted_raw_ostream", stream, as.logical(delete))
@@ -23,15 +27,17 @@ function(stream, delete = FALSE, finalize = TRUE)
-stringRawOstream =
+rawStringOstream =
function(value = "")
.Call("R_new_raw_string_ostream", as.character(value))
setAs("raw_string_ostream", "character",
- function(from)
- .Call("R_raw_string_ostream_str", from))
+ function(from) {
+ flush(stream)
+ .Call("R_raw_string_ostream_str", from)
+ })
flush.formatted_raw_ostream =
5 R/targetMachine.R
@@ -7,6 +7,9 @@ function(machine, passManager)
addPassesToEmitFile =
function(machine, passManager, outStream, fileType = 0L)
- .Call("R_TargetMachine_addPassesToEmitFile", machine, passManager, outStream, as.integer(fileType))
+ ans = .Call("R_TargetMachine_addPassesToEmitFile", machine, passManager, outStream, as.integer(fileType))
+ if(ans)
+ stop("failed in addPassesToEmitFile. Is this type of file supported by the manager?")
+ ans
21 inst/TU/clang.R
@@ -1,5 +1,12 @@
+TypeMap = list('llvm::Twine' = list(convertRValue = function(var, rvar, ..., type, typeMap = NULL) { sprintf("%s = makeTwine(%s);", var, rvar)}),
+ 'llvm::StringRef' = list(convertRValue = function(var, rvar, ..., type, typeMap = NULL){
+ sprintf("var = llvm::StringRef(CHAR(STRING_ELT(%s, 0)));", var, rvar)
+ })
+ )
# The -xc++ is critical to get c++ parsing and the header files.
# Otherwise get errors about not finding cstddef.
@@ -26,16 +33,24 @@ o$Module
mod.class = readCppClass(llvm$Module)
irbuilder.class = readCppClass(llvm$IRBuilder)
enums = getEnums(tu)
+# Could do this directly with
+#cc = getEnums("~/llvm-devel/include/llvm/IR/CallingConv.h", args = "-xc++")
+#cat(makeEnumClass(cc$ID, "CallingConv"), sep = "\n", file = "../../R/CallingConvEnum.R")
+cat(makeEnumClass(enums$ID, "CallingConv"), sep = "\n", file = "../../R/CallingConvEnum.R")
+triple = readCppClass(llvm$Triple)
+m = createCppMethod(triple@methods$normalize, typeMap = TypeMap)
-cc = getEnums("~/llvm-devel/include/llvm/IR/CallingConv.h", args = "-xc++")
-cat(makeEnumClass(cc$ID, "CallingConv"), sep = "\n", file = "../../R/CallingConvEnum.R")
#klasses = getCppClasses(tu)
7 inst/TU/llvm.c
@@ -45,16 +45,21 @@
#include <llvm/IR/IRBuilder.h>
#include <llvm/IR/CallingConv.h>
+#include <llvm/ADT/Triple.h>
#include <algorithm>
#include <llvm/ExecutionEngine/ExecutionEngine.h>
+#include <llvm/IR/CallingConv.h>
llvm::Module *m;
llvm::BasicBlock *bl;
llvm::IRBuilder<> *b;
-llvm::FunctionPassManager *mgr;
+llvm::FunctionPassManager *fmgr;
+llvm::PassManager *mgr;
llvm::ExecutionEngine *eng;
13 src/Triple.cpp
@@ -0,0 +1,13 @@
+#include "Rllvm.h"
+#include <llvm/ADT/Triple.h>
+SEXP R_Triple_setTriple(SEXP r_obj, SEXP r_Str)
+ llvm::Triple *obj = GET_REF(r_obj, Triple);
+ llvm::Twine Str;
+ Str = makeTwine(r_Str);
+ obj->setTriple(Str);
+ return(R_NilValue);
2 src/ostream.cpp
@@ -6,7 +6,7 @@ R_new_raw_string_ostream(SEXP r_str)
std::string *str;
if(TYPEOF(r_str) == STRSXP)
- str = new std::string(CHAR(STRING_ELT(r_str, 0)));
+ str = new std::string(Rf_length(r_str) ? CHAR(STRING_ELT(r_str, 0)) : "");
str = (std::string *) getRReference(r_str);

0 comments on commit 93417bd

Please sign in to comment.