---

Graph Machine Learning Workshop

Workshop 1 - Intro

1. Graph Machine Learning and Motivations

Workshop 2 - Basic graph representations

2. Graph Representations

Lab 0. Data Handling of Graphs

Workshop 3 - PageRank, message passing - Skip?

3. Link Analysis

Workshop 4 - Graph Neural Networks, graph level prediction

8. Graph Neural Networks

Lab 3: Hands-on GCN, Solution
Lab 3: Node Classification, Solution
Lab 5. Graph Level Prediction, Solution

Workshop 5 - Deep Generative Graph Models, illustrative notebook
14. Learning Deep Generative Models of Graphs.pdf

• Deep Generative Graph Learning Notebook
• Model file

Note: Opening notebooks in colab

Runtime -> Change Runtime Type -> GPU

# !nvidia-smi

# Add this in a Google Colab cell to install the correct version of Pytorch Geometric.
import torch

def format_pytorch_version(version):
  return version.split('+')[0]

TORCH_version = torch.__version__
TORCH = format_pytorch_version(TORCH_version)

def format_cuda_version(version):
  return 'cu' + version.replace('.', '')

CUDA_version = torch.version.cuda
CUDA = format_cuda_version(CUDA_version)

!pip install torch-scatter     -f https://pytorch-geometric.com/whl/torch-{TORCH}+{CUDA}.html
!pip install torch-sparse      -f https://pytorch-geometric.com/whl/torch-{TORCH}+{CUDA}.html
!pip install torch-cluster     -f https://pytorch-geometric.com/whl/torch-{TORCH}+{CUDA}.html
!pip install torch-spline-conv -f https://pytorch-geometric.com/whl/torch-{TORCH}+{CUDA}.html
!pip install torch-geometric 

Select Runtime -> Restart Runtime and start where you left off.

Graph Machine Learning Course

This course provides an introduction to machine learning on graphs.

Paste the following in the first Colab cell to install pytorch geometric (if not present):

Many important real-world datasets can be represented as a graph of relationships between objects. Such networks are a basic tool for modeling social networks, knowledge graphs, the Web, and biological systems such as protein-interaction networks. Until recently, very little attention has been devoted to the generalization of neural network models to such structured datasets.

This course focuses on the computational, algorithmic, and modeling challenges specific to the analysis of graphs. By studying the underlying graph structure and its features, students are introduced to machine learning techniques and data mining tools better able to reveal insights on a variety of networks.

Fundamental questions:
Can we take advantage of graph structure to learn better representations of data? Given better representations, can we make better predictions?

Topics include: representation learning and Graph Neural Networks; algorithms for the World Wide Web; reasoning over Knowledge Graphs; influence maximization; disease outbreak detection, social network analysis.

Lectures are augmented with hands-on tutorials using Jupyter Notebooks. Laboratory assignments will be completed using Python and related packages: PyTorch, PyG, NumPy, Pandas, SciPy, StatsModels, SciKit-Learn, NetworkX, and MatPlotLib.

2-2-3 (class hours/week, laboratory hours/week, credits)

Prerequisites: CS-385 Algorithms, Probability and Statistics; programming maturity, and the ability to program in Python.

ABET: Math/Science, Engineering Topics.

Outcomes:

• Understand the basic process of applying machine learning to graph data.
• The ability to identify, load, and prepare a graph data set for a given problem.
• The ability to analyze a data set including the ability to understand which data attributes (dimensions) affect the outcome.
• The ability to develop and apply graph neural network algorithms for node classifcation, link detection, community detection, and graph generation.
• The ability to apply methods to real world data sets.
• The ability to identify, articulate, and propose a research problem related to graph machine learning.

Tools: Python and related packages for data analysis, machine learning, and visualization. Jupyter Notebooks.

Grading:
Weekly labs and final project: 60%
Midterm: 20%
Final: 20%

Office DH425:
T 3-4pm, Th 3-4pm

References:

Graph Representation Learning by William L. Hamilton

http://www.cs.cornell.edu/home/kleinber/networks-book/Networks, Crowds, and Markets: Reasoning About a Highly Connected World by David Easley and Jon Kleinberg

Network Science by Albert-László Barabási

Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković

Geometric Deep Learning

Stanford Machine Learning on Graphs

PyG - Pytorch Geometric Documentation

---

Week 1: Intro to Network Analysis and Machine Learning on Graphs

Workshop 1 - Intro
1. Graph Machine Learning and Motivations

Workshop 2 - Basic graph representations
2. Graph Representations

Lab 0. Data Handling of Graphs

Lab 1. Graph ML Research Topics

Graph Laplacian Notebook

References:
Graph Representation Learning by William L. Hamilton

TUDataset: A collection of benchmark datasets for learning with graphs

The Emerging Field of Signal Processing on Graphs

Week 2: Link Analysis and Random Walk

Workshop 3 - PageRank for recursive message passing 3. Link Analysis

Lab 2: Link Analysis

Lab 2: PageRank notebook

References:
The Anatomy of a Large-Scale Hypertextual Web Search Engine

Authoritative Sources in a Hyperlinked Environment

Week 3: Node Classification, Intro to Graph Neural Networks

4. Message Passing, Node Embeddings, and Representations

5. Node Embeddings, Random Walk, Node2vec

Hands-on: Node2Vec (optional)

Lab 3: Hands-on GCN

Lab 3: Node Classification

Week 4: Machine Learning Review, Graph Neural Network Intro

6. Machine Learning Intro

Hands-on Gradient Descent Notebook

Hands-on Logistic Regression with Pytorch

XOR and Logistic Regression Proof

XOR and Logistic Regression Notebook

7. Deep Learning Intro

Lab 4. Building a Neural Network in PyTorch

References:

Simplifying Graph Convolutional Networks

SEMI-SUPERVISED CLASSIFICATION WITH GRAPH CONVOLUTIONAL NETWORKS

Week 5: Deep Graph Learning, GraphSage, Applications

8. Graph Neural Network Intro
Slides 1-16.

Workshop 5 - Graph Neural Network
8. Graph Neural Networks

Lab 5. Graph Level Prediction

References:

Graph Neural Networks: A review of Methods and Applications, Zhou, 2018. Graph Representation Learning, Hamilton, 2020.

Graph Representation Learning, Ch. 4 Graph Neural Network Model

Inductive Representation Learning on Large Graphs

Week 6: Graph Convolution Networks, MIDTERM

MIDTERM - study guide

Lab 6. FINAL Project Proposal

Week 7: Graph Convolution Networks, Graph Relational Networks

9. Graph Convolution Networks

Lab 7.1. Hands-on Loading Graphs from CSV

Lab 7.2. Graph ML Project

Optional: Additional examples of Loading Graphs from CSV

10. Relational Graph Networks

Week 8: Graph Attention Networks, Recommender Systems

11. Graph Attention Networks

Lab 7.2. Graph ML Project

12. Graph Recommender Systems

Week 9: Generative Graph Models

13. Deep Generative Models for Graphs

Lab 7.2. Graph ML Project

14. Learning Deep Generative Models of Graphs.pdf

Hands On:

• Deep Generative Graph Learning Notebook
• Model file
• Annimation Gif

References:

• GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models Jiaxuan You, Rex Ying, Xiang Ren, William L. Hamilton, Jure Leskovec
  https://arxiv.org/abs/1802.08773
• Learning Deep Generative Models of Graphs
  https://arxiv.org/pdf/1803.03324.pdf
• AlphaFold

Week 10: Temporal Graph Models, Final Projects, Final Exam Review

15. Temporal Graph Networks

Optional Traffice Notebook!

Student presentations

Final Exam Review

Week 11: Final Exam

CS4981 Thursday 2:00 DH344