Machine Learning and Data Analysis for Nuclear Physics, a Nuclear TALENT Course at the ECT*, Trento, Italy, June 22 to July 10 2020.
Why a course on Machine Learning for Nuclear Physics?
Probability theory and statistical methods play a central role in science. Nowadays we are surrounded by huge amounts of data. For example, there are about one trillion web pages; more than one hour of video is uploaded to YouTube every second, amounting to 10 years of content every day; the genomes of 1000s of people, each of which has a length of more than a billion base pairs, have been sequenced by various labs and so on. This deluge of data calls for automated methods of data analysis, which is exactly what machine learning provides. The purpose of this Nuclear Talent course is to provide an introduction to the core concepts and tools of machine learning in a manner easily understood and intuitive to physicists and nuclear physicists in particular. We will start with some of the basic methods from supervised learning and statistical data analysis, such as various regression methods before we move into deep learning methods for both supervised and unsupervised learning, with an emphasis on the analysis of nuclear physics experiments and theoretical nuclear physics. The students will work on hands-on daily examples as well as projects than can result in final credits. Exercises and projects will be provided and the aim is to give the participants an overview on how machine learning can be used to analyze and study nuclear physics problems (experiment and theory). The major scope is to give the participants a deeper understanding on what Machine learning and Data Analysis are and how they can be used to analyze data from nuclear physics experiments and perform theoretical calculations of nuclear many-body systems.
The goals of the Nuclear Talent course on Machine Learning and Data Analysis are to give the participants a deeper understanding and critical view of several widely popular Machine Learning algorithms, covering both supervised and unsupervised learning. The learning outcomes involve an understanding of the following central methods:
- Basic concepts of machine learning and data analysis and statistical concepts like expectation values, variance, covariance, correlation functions and errors;
- Estimation of errors using cross-validation, blocking, bootstrapping and jackknife methods;
- Optimization of functions
- Linear Regression and Logistic Regression;
- Dimensionality reductions, from PCA to clustering
- Boltzmann machines;
- Neural networks and deep learning;
- Convolutional Neural Networks
- Recurrent Neureal Networks and Autoencoders
- Decisions trees and random forests
- Support vector machines and kernel transformations
We are targeting an audience of graduate students (both Master of Science and PhD) as well as post-doctoral researchers in nuclear experiment and theory.
The teaching teams consists of both theorists and experimentalists. We believe such a mix is important as it gives the students a better understanding on how data are obtained, and what are the limitations and possibilities in understanding and interpreting the experimental information.
Introduction to the Talent Courses
A recently established initiative, Training in Advanced Low Energy Nuclear Theory, aims at providing an advanced and comprehensive training to graduate students and young researchers in low-energy nuclear theory. The initiative is a multinational network between several European and Northern American institutions and aims at developing a broad curriculum that will provide the platform for a cutting-edge theory for understanding nuclei and nuclear reactions. These objectives will be met by offering series of lectures, commissioned from experienced teachers in nuclear theory. The educational material generated under this program will be collected in the form of WEB-based courses, textbooks, and a variety of modern educational resources. No such all-encompassing material is available at present; its development will allow dispersed university groups to profit from the best expertise available.
Aims and Learning Outcomes
This three-week TALENT course on nuclear theory will focus on Machine Learning and Data Analysis algorithms for nuclear physics and to use such methods in the interpretation of data on the structure of nuclear systems.
We propose approximately forty-five hours of lectures over three weeks and a comparable amount of practical computer and exercise sessions, including the setting of individual problems and the organization of various individual projects.
The mornings will consist of lectures and the afternoons will be devoted to exercises meant to shed light on the exposed theory, the computational projects and individual student projects. These components will be coordinated to foster student engagement, maximize learning and create lasting value for the students. For the benefit of the TALENT series and of the community, material (courses, slides, problems and solutions, reports on students' projects) will be made publicly available using version control software like git and posted electronically on github (this site).
As with previous TALENT courses, we envision the following features for the afternoon sessions:
- We will use both individual and group work to carry out tasks that are very specific in technical instructions, but leave freedom for creativity.
- Groups will be carefully put together to maximize diversity of backgrounds.
- Results will be presented in a conference-like setting to create accountability.
- We will organize events where individuals and groups exchange their experiences, difficulties and successes to foster interaction.
- During the school, on-line and lecture-based training tailored to technical issues will be provided. Students will learn to use and interpret the results of computer-based and hand calculations of nuclear models. The lectures will be aligned with the practical computational projects and exercises and the lecturers will be available to help students and work with them during the exercise sessions.
- These interactions will raise topics not originally envisioned for the course but which are recognized to be valuable for the students. There will be flexibility to organize mini-lectures and discussion sessions on an ad-hoc basis in such cases.
- Each group of students will maintain an online logbook of their activities and results.
- Training modules, codes, lectures, practical exercise instructions, online logbooks, instructions and information created by participants will be merged into a comprehensive website that will be available to the community and the public for self-guided training or for use in various educational settings (for example, a graduate course at a university could assign some of the projects as homework or an extra credit project, etc).
At the end of the course the students should have a basic understanding of
- Statistical data analysis, theory and tools to handle large data sets.
- A solid understanding of central machine learning algorithms for supervised and unsupervised learning, involving linear and logistic regression, support vector machines, decision trees and random forests, neural networks and deep learning (convolutional neural networks, recursive neural networks etc)
- Be able to write codes for linear regression, logistic regression and use modern libraries like Tensorflow, Pytorch, Scikit-Learn in order to analyze data from nuclear physics experiments and perform theoretical calculations
- A deeper understanding of the statistical properties of the various methods, from the bias-variance tradeoff to resampling techniques.
Course Content and detailed plan
The lecture plan is as follows
- Monday Linear Regression and intro to statistical data analysis
- Tuesday Logistic Regression and classification problems, intro to gradient methods
- Wednesday More on gradient methods and decision trees
- Thursday Decision Trees, Random Forests and Support vector machines
- Friday Discussion of nuclear experiments and how to analyze data, presentation of simulated data from Active-Target Time-Projection Chamber (AT-TPC)
- Monday Nuclear experiments from AT-TPC and begin Neural Networks
- Tuesday From Neural Networks to Convolutional Neural Networks and how to analyze experiment (classification of events and real data)
- Wednesday Analyzing data and recurrent neural networks
- Thursday Autoencoders and reinforcement learning
- Friday Beta-decay experiments, how to analyze various events
- Monday Beta-decay experiments and deep learning, simulated and real data
- Tuesday Solving many-body problems with Neural Networks
- Wednesday Boltzmann Machines and many-body problems
- Thursday Introduction to exploratory data analysis and unsupervised learning, clustering and dimension reduction
- Friday Future directions in machine learning and summary of course
The course will be taught as an intensive course of duration of three weeks, with a total time of 45 h of lectures, 45 h of exercises and a final assignment of 2 weeks of work. The total load will be approximately 160-170 hours, corresponding to 7 ECTS in Europe. The final assignment will be graded with marks A, B, C, D, E and failed for Master students and passed/not passed for PhD students. A course certificate will be issued for students requiring it from the University of Trento.
The organization of a typical course day is as follows:
Time and Activity
- 9am-12pm Lectures, project relevant information and directed exercises
- 12pm-2pm Lunch
- 2pm-6pm Computational projects, exercises and hands-on sessions
- 6pm-7pm Wrap-up of the day and eventual student presentations
Teachers and organizers
The teachers and organizers are
- Daniel Bazin at Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University, East Lansing, Michigan, USA
- Morten Hjorth-Jensen at Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University, East Lansing, Michigan, USA
- Michelle Kuchera at Physics Department, Davidson College, Davidson, North Carolina, USA
- Sean Liddick at Department of Chemistry and National Superconducting Cyclotron Laboratory, Michigan State University, East Lansing, Michigan, USA
- Raghuram Ramanujan at Department of Mathematics and Computer Science, Davidson College, Davidson, North Carolina, USA
Morten Hjorth-Jensen will also function as student advisor and coordinator.
Audience and Prerequisites
Students and post-doctoral fellows interested in nuclear physics, experiment and theory alike interested in data analysis and machine learning applied to nuclear physics.
The students are expected to have operating programming skills in programming, and in particular on interpreted languages like Python. Preparatory modules on Python programing will be given, idem for a review on linear algebra and statistical data analysis.
Students who have not studied the above topics are expected to gain this knowledge prior to attendance. Additional modules for self-teaching will be provided in good time before the course begins
For more information, please go to https://github.com/NuclearTalent/MachineLearningECT or go to http://nucleartalent.github.io/MachineLearningECT/doc/web/course.html for better display of course material and topics to be covered. Admission The target group is Master of Science students, PhD students and early post-doctoral fellows. Also senior staff can attend but they have to be self-supported.
- Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly
General learning book on statistical analysis:
- Christian Robert and George Casella, Monte Carlo Statistical Methods, Springer
- Peter Hoff, A first course in Bayesian statistical models, Springer
- Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer General Machine Learning Books:
- Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press
- Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer
- David J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press
- David Barber, Bayesian Reasoning and Machine Learning, Cambridge University Press