-
Notifications
You must be signed in to change notification settings - Fork 559
Effectiveness
Siran Yang edited this page Jun 4, 2019
·
1 revision
In this section, we demonstrate the effectiveness of the Euler algorithm package on both PPI and Reddit data sets. The embedding size of all models is 256, and the training batch size is 512.
Neighbor aggregation algorithms such as GraphSAGE can use either supervised training mode or unsupervised training mode. We use suffixes to distinguish them. Algorithms such as LINE/DeepWalk can only use unsupervised training mode.
- In the supervised training mode, we directly predict the label information. The training epoch number is 20 except for GAT. GAT is trained with 100 epochs for aligning the configuration of the paper.
- In the unsupervised mode, we first train for 20 epochs to produce embedding of nodes. These embeddings are then used as the features of the Logistic Regression model for supervised training of20 epoch.
Below is the micro-F1 scores of all experiments:
| Model | Mirco-F1 scores reported in original papers | Mirco-F1 scores in Euler | Note |
|---|---|---|---|
| Random | 0.396 | 0.415 | |
| DeepWalk | NA | 0.536 | |
| LINE-1stOrder | NA | 0.517 | opt = sgd / lr = 2e-1 |
| LINE-2ndOrder | NA | 0.535 | opt = sgd / lr = 2e-1 |
| GraphSage-GCN | 0.465 | 0.460 | opt = adam / lr = 2e-3 |
| GraphSage-Mean | 0.486 | 0.502 | opt = adam / lr = 1e-3 |
| GraphSage-Meanpool | NA | 0.486 | opt = adam / lr = 1e-3 |
| GraphSage-Maxpool | 0.502 | 0.489 | opt = adam / lr = 1e-3 |
| GraphSage-GCN-Supervised | 0.500 | 0.504 | opt = adam / lr = 1e-2 |
| GraphSage-Mean-Supervised | 0.598 | 0.614 | opt = adam / lr = 1e-2 |
| GraphSage-Meanpool-Supervised | NA | 0.640 | opt = adam / lr = 5e-3 |
| GraphSage-Maxpool-Supervised | 0.600 | 0.634 | opt = adam / lr = 5e-3 |
| ScalableGCN-Mean-Supervised | NA | 0.603 | opt = adam / lr = 2e-1 / store lr = 2e-3 |
| ScalableGCN-Meanpool-Supervised | NA | 0.606 | opt = adam / lr = 5e-3 / store lr = 5e-4 |
| GAT | 0.973 | 0.948 | opt = adam / lr = 5e-3 / head_num=4 / layer_num=3 / sample_neighbor=150 |
| Model | Mirco-F1 scores reported in original papers | Mirco-F1 scores in Euler | Note |
|---|---|---|---|
| Random | 0.043 | 0.120 | |
| DeepWalk | NA | 0.841 | |
| LINE-1stOrder | NA | 0.813 | opt = sgd / lr = 2e-1 |
| LINE-2ndOrder | NA | 0.820 | opt = sgd / lr = 2e-1 |
| GraphSage-GCN-Supervised | 0.930 | 0.917 | opt = adam / lr = 1e-2 |
| GraphSage-Mean-Supervised | 0.950 | 0.933 | opt = adam / lr = 1e-2 |
| GraphSage-Meanpool-Supervised | NA | 0.928 | opt = adam / lr = 5e-3 |
| ScalableGCN-Mean-Supervised | NA | 0.929 | opt = adam / lr = 1e-2 / store lr = 2e-3 |