效果测试
Yi Ren edited this page Jan 17, 2019
·
8 revisions
本节描述Euler算法包在PPI和Reddit两个数据集上的模型调优效果。所有模型的embedding大小均为256。训练的batch size均为512。
GraphSAGE等邻居汇聚算法既可以采用supervised训练方式，也可以用unsupervised训练模式。我们用后缀加以区分。LINE/DeepWalk等算法则仅应用无监督训练模式。
- 在有监督训练模式下，我们直接预测标注信息。除GAT外，训练epoch数为20。GAT为对齐论文的配置训练100epoch。
- 非监督模式下，我们先进行20个epoch训练以生成节点的embedding向量。然后此embedding作为Logistic Regression模型的特征进行20个epoch的supervised训练。
下表所列指标均为测试集的micro-F1。
| 模型 | 论文汇报 F1 | Euler F1 | 备注 |
|---|---|---|---|
| Random | 0.396 | 0.415 | |
| DeepWalk | NA | 0.536 | |
| LINE-1stOrder | NA | 0.517 | opt = sgd / lr = 2e-1 |
| LINE-2ndOrder | NA | 0.535 | opt = sgd / lr = 2e-1 |
| GraphSage-GCN | 0.465 | 0.460 | opt = adam / lr = 2e-3 |
| GraphSage-Mean | 0.486 | 0.502 | opt = adam / lr = 1e-3 |
| GraphSage-Meanpool | NA | 0.486 | opt = adam / lr = 1e-3 |
| GraphSage-Maxpool | 0.502 | 0.489 | opt = adam / lr = 1e-3 |
| GraphSage-GCN-Supervised | 0.500 | 0.504 | opt = adam / lr = 1e-2 |
| GraphSage-Mean-Supervised | 0.598 | 0.614 | opt = adam / lr = 1e-2 |
| GraphSage-Meanpool-Supervised | NA | 0.640 | opt = adam / lr = 5e-3 |
| GraphSage-Maxpool-Supervised | 0.600 | 0.634 | opt = adam / lr = 5e-3 |
| ScalableGCN-Mean-Supervised | NA | 0.603 | opt = adam / lr = 2e-1 / store lr = 2e-3 |
| ScalableGCN-Meanpool-Supervised | NA | 0.606 | opt = adam / lr = 5e-3 / store lr = 5e-4 |
| GAT | 0.973 | 0.948 | opt = adam / lr = 5e-3 / head_num=4 / layer_num=3 / sample_neighbor=150 |
| 模型 | 论文汇报 F1 | Euler F1 | 备注 |
|---|---|---|---|
| Random | 0.043 | 0.120 | |
| DeepWalk | NA | 0.841 | |
| LINE-1stOrder | NA | 0.813 | opt = sgd / lr = 2e-1 |
| LINE-2ndOrder | NA | 0.820 | opt = sgd / lr = 2e-1 |
| GraphSage-GCN-Supervised | 0.930 | 0.917 | opt = adam / lr = 1e-2 |
| GraphSage-Mean-Supervised | 0.950 | 0.933 | opt = adam / lr = 1e-2 |
| GraphSage-Meanpool-Supervised | NA | 0.928 | opt = adam / lr = 5e-3 |
| ScalableGCN-Mean-Supervised | NA | 0.929 | opt = adam / lr = 1e-2 / store lr = 2e-3 |