📝 "End-to-end Deep Learning of Optimization Heuristics" (PACT Best Paper 2017)
Switch branches/tags
Nothing to show
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
code Automated code format Dec 6, 2018
data Add Case Study B Jun 21, 2017
tex Automated code format Dec 6, 2018
.gitignore Initial implementation May 31, 2017
BUILD Add a BUILD file. Jul 17, 2018
LICENSE Add GPLv3 license. Jul 11, 2018
README.md Add contact details to license. Jul 11, 2018
deeptune.png Initial implementation May 31, 2017
paper.pdf Camera ready fixes Jul 18, 2017
paper.png Add image to readme Jul 18, 2017
slides.png Add link to slides Sep 12, 2017


End-to-end Deep Learning of Optimization Heuristics

Chris Cummins, Pavlos Petoumenos, Zheng Wang, Hugh Leather.

Winner of Best Paper Award PACT'17


Accurate automatic optimization heuristics are necessary for dealing with the complexity and diversity of modern hardware and software. Machine learning is a proven technique for learning such heuristics, but its success is bound by the quality of the features used. These features must be hand crafted by developers through a combination of expert domain knowledge and trial and error. This makes the quality of the final model directly dependent on the skill and available time of the system architect.

Our work introduces a better way for building heuristics. We develop a deep neural network that learns heuristics over raw code, entirely without using code features. The neural network simultaneously constructs appropriate representations of the code and learns how best to optimize, removing the need for manual feature creation. Further, we show that our neural nets can transfer learning from one optimization problem to another, improving the accuracy of new models, without the help of human experts.

We compare the effectiveness of our automatically generated heuristics against ones with features hand-picked by experts. We examine two challenging tasks: predicting optimal mapping for heterogeneous parallelism and GPU thread coarsening factors. In 89% of the cases, the quality of our fully automatic heuristics matches or surpasses that of state-of-the-art predictive models using hand-crafted features, providing on average 14% and 12% more performance with no human effort expended on designing features.

  title={End-to-end Deep Learning of Optimization Heuristics},
  author={Cummins, Chris and Petoumenos, Pavlos and Wang, Zheng and Leather, Hugh},

See code/README.md for instructions on re-producing the experiments.



The code for this paper (everything in the directory code) is released under the terms of the GPLv3 license. See LICENSE for details. Everything else (i.e. the LaTeX sources and data sets) are unlicensed, please contact Chris Cummins chrisc.101@gmail.com before using.