The complete set of tools for energy consumption analysis of programming languages, using Computer Language Benchmark Game
Pascal C Makefile Java Python Common Lisp Other
Switch branches/tags
Nothing to show
Clone or download
Latest commit b3d1c0a Jan 29, 2018
Permalink
Failed to load latest commit information.
Ada added missing instruction in 'measure' target Dec 15, 2017
C++ added missing instruction in 'measure' target Dec 15, 2017
C added missing instruction in 'measure' target Dec 15, 2017
CSharp added missing instruction in 'measure' target Dec 15, 2017
Chapel Merge branch 'master' of github.com:greensoftwarelab/Energy-Languages Dec 15, 2017
Dart added missing instruction in 'measure' target Dec 15, 2017
Erlang added missing instruction in 'measure' target Dec 15, 2017
FSharp added missing instruction in 'measure' target Dec 15, 2017
Fortran added missing instruction in 'measure' target Dec 15, 2017
Go added missing instruction in 'measure' target Dec 15, 2017
Hack added missing instruction in 'measure' target Dec 15, 2017
Haskell added missing instruction in 'measure' target Dec 15, 2017
JRuby added missing instruction in 'measure' target Dec 15, 2017
Java added missing instruction in 'measure' target Dec 15, 2017
JavaScript added missing instruction in 'measure' target Dec 15, 2017
Lisp added missing instruction in 'measure' target Dec 15, 2017
Lua added missing instruction in 'measure' target Dec 15, 2017
OCaml added missing instruction in 'measure' target Dec 15, 2017
PHP added missing instruction in 'measure' target Dec 15, 2017
Pascal added missing instruction in 'measure' target Dec 15, 2017
Perl added missing instruction in 'measure' target Dec 15, 2017
Python added missing instruction in 'measure' target Dec 15, 2017
RAPL added all 27 language benchmarks Aug 28, 2017
Racket added missing instruction in 'measure' target Dec 15, 2017
Ruby added missing instruction in 'measure' target Dec 15, 2017
Rust added missing instruction in 'measure' target Dec 15, 2017
Smalltalk added missing instruction in 'measure' target Dec 15, 2017
Swift added missing instruction in 'measure' target Dec 15, 2017
TypeScript added missing instruction in 'measure' target Dec 15, 2017
.gitattributes Added Input Files as Large Files Sep 7, 2017
README.md updated README (#12) Jan 29, 2018
compile_all.py added all 27 language benchmarks Aug 28, 2017
gen-input.sh added input generation script (#11) Jan 29, 2018

README.md

Energy Efficiency in Programming Languages

Checking Energy Consumption in Programming Languages Using the Computer Language Benchmark Game as a case study.

What is this?

This repo contains the source code of 10 distinct benchmarks, implemented in 28 different languages (exactly as taken from the Computer Language Benchmark Game).

It also contains tools which provide support, for each benchmark of each language, to 4 operations: (1) compilation, (2) execution, (3) energy measuring and (4) memory peak detection.

How is it structured and hows does it work?

This framework follows a specific folder structure, which guarantees the correct workflow when the goal is to perform and operation for all benchmarks at once. Moreover, it must be defined, for each benchmark, how to perform the 4 operations considered.

Next, we explain the folder structure and how to specify, for each language benchmark, the execution of each operation.

The Structure

The main folder contains 32 elements:

  1. 28 sub-folders (one for each of the considered languages); each folder contains a sub-folder for each considered benchmark.
  2. A Python script compile_all.py, capable of building, running and measuring the energy and memory usage of every benchmark in all considered languages.
  3. A RAPL sub-folder, containing the code of the energy measurement framework.
  4. A Bash script gen-input.sh, used to generate the input files for 3 benchmarks: k-nucleotide, reverse-complement, and regex-redux.

Basically, the directories tree will look something like this:

| ...
| <Language-1>
	| <benchmark-1>
		| <source>
		| Makefile
		| [input]
	| ...
	| <benchmark-i>
		| <source>
		| Makefile
		| [input]
| ...
| <Language-i>
	| <benchmark-1>
	| ...
	| <benchmark-i>
| RAPL
| compile_all.py
| gen-input.sh

Taking the C language as an example, this is how the folder for the binary-trees and k-nucleotide benchmarks would look like:

| ...
| C
	| binary-trees
		| binarytrees.gcc-3.c
		| Makefile
	| k-nucleotide
		| knucleotide.c
		| knucleotide-input25000000.txt
		| Makefile
	| ...
| ...

The Operations

Each benchmark sub-folder, included in a language folder, contains a Makefile. This is the file where is stated how to perform the 4 supported operations: (1) compilation, (2) execution, (3) energy measuring and (4) memory peak detection.

Basically, each Makefile must contains 4 rules, one for each operations:

Rule Description
compile This rule specifies how the benchmark should be compiled in the considered language; Interpreted languages don't need it, so it can be left blank in such cases.
run This rule specifies how the benchmark should be executed; It is used to test whether the benchmark runs with no errors, and the output is the expected.
measure This rule shows how to use the framework included in the RAPL folder to measure the energy of executing the task specified in the run rule.
mem Similar to measure, this rule executes the task specified in the run rule but with support for memory peak detection.

To better understand it, here's the Makefile for the binary-trees benchmark in the C language:

compile:
	/usr/bin/gcc -pipe -Wall -O3 -fomit-frame-pointer -march=native -fopenmp -D_FILE_OFFSET_BITS=64 -I/usr/include/apr-1.0 binarytrees.gcc-3.c -o binarytrees.gcc-3.gcc_run -lapr-1 -lgomp -lm
	
measure:
	sudo ../../RAPL/main "./binarytrees.gcc-3.gcc_run 21" C binary-trees

run:
	./binarytrees.gcc-3.gcc_run 21

mem:
	/usr/bin/time -v ./binarytrees.gcc-3.gcc_run 21

Running an example.

First things first: generate the input files, like this

./gen-input.sh

This will generate the necessary input files, and are valid for every language.

We included a main Python script, compile_all.py, that you can either call from the main folder or from inside a language folder, and it can be executed as follows:

python compile_all.py [rule]

You can provide a rule from the available 4 referenced before, and the script will perform it using every Makefile found in the same folder level and bellow.

The default rule is compile, which means that if you run it with no arguments provided (python compile_all.py) the script will try to compile all benchmarks.

The results of the energy measurements will be stored in files with the name <language>.csv, where <language> is the name of the running language. You will find such file inside of corresponding language folder.

Add your own example!

Wanna know your own code's energy behavior? We help you!

Follow this steps:

1. Create a folder with the name of you benchmark, such as test-benchmark, inside the language you implemented it.
2. Follow the instructions presented in the Operations section, and fill the Makefile.
3. Use the compile_all.py script to compile, run, and/or measure what you want! Or run it yourself using the make command.

Further Reading

Wanna know more? Check this website!

There you can find the results of a successful experimental setup using the contents of this repo, and the used machine and compilers specifications.

You can also find there the paper which include such results and our discussion on them:

"Energy Efficiency across Programming Languages: How does Energy, Time and Memory Relate?", Rui Pereira, Marco Couto, Francisco Ribeiro, Rui Rua, Jácome Cunha, João Paulo Fernandes, and João Saraiva. In Proceedings of the 10th International Conference on Software Language Engineering (SLE '17)

IMPORTANT NOTE:

The Makefiles have specified, for some cases, the path for the language's compiler/runner. It is most likely that you will not have them in the same path of your machine. If you would like to properly test every benchmark of every language, please make sure you have all compilers/runners installed, and adapt the Makefiles accordingly.

Contacts and References

Green Software Lab

Main contributors: @Marco Couto and @Rui Pereira

The Computer Language Benchmark Game