[Feature] Adjusted homophily and label informativeness #6516

OlegPlatonov · 2023-11-01T15:58:20Z

Description

Hi! In a recent paper Characterizing Graph Datasets for Node Classification: Homophily-Heterophily Dichotomy and Beyond several characteristics of labeled graphs have been proposed. First, it has been shown that a rarely used in graph ML literature measure named adjusted homophily satisfies more desirable properties than other homophily measures, which makes it appropriate for comparing homophily levels across datasets with different number of classes, class sizes, and degree distributions across classes. Further, a new characteristic - label informativeness - has been proposed. It shows how much information about a node's label we get from knowing its neighbor's label. It has been shown that label informativeness is much more correlated with GNN performance than homophily.

In this PR I've added adjusted homophily and label informativeness to DGL.

Checklist

Please feel free to remove inapplicable items for your PR.

The PR title starts with [$CATEGORY] (such as [NN], [Model], [Doc], [Feature]])
I've leverage the tools to beautify the python and c++ code.
The PR is complete and small, read the Google eng practice (CL equals to PR) to understand more about small PR. In DGL, we consider PRs with less than 200 lines of core code change are small (example, test and documentation could be exempted).
All changes have test coverage
Code is well-documented
To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
Related issue is referred in this PR
If the PR is for a new model/paper, I've updated the example index here.

Changes

Added adjusted homophily and tests for it.
Added edge label informativeness and node label informativeness and tests for them.

dgl-bot · 2023-11-01T15:58:51Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-11-01T15:59:03Z

Commit ID: cc7ce6593f897abf364a3f4c57ce4685d7b1de29

Build ID: 1

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

frozenbugs · 2023-11-02T02:15:06Z

@dgl-bot

dgl-bot · 2023-11-02T02:53:50Z

Commit ID: 6c8a959d5bd6152a2453903d05c8ad58df545865

Build ID: 2

Status: ✅ CI test succeeded.

Report path: link

Full logs path: link

rudongyu

Thanks for your contribution! I left a few comments, mainly on the documentation and the coding of loops over nodes and edges. Besides, you can add index to these measures here: https://github.com/dmlc/dgl/blob/master/docs/source/api/python/dgl.rst?plain=1#L207.

python/dgl/homophily.py

python/dgl/label_informativeness.py

…ding to the new implementation

dgl-bot · 2023-12-05T19:28:41Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

…iveness

dgl-bot · 2023-12-05T19:28:54Z

Commit ID: 057b48854a1c454d2d1920a859c7c5863cb2c08c

Build ID: 3

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-12-05T19:29:11Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-12-05T19:29:24Z

Commit ID: dbc9e94

Build ID: 4

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

OlegPlatonov · 2023-12-05T19:43:08Z

@rudongyu Thank you for your review and suggestions! I've rewritten the code to avoid loops, it's much faster now. Please take a look when you have time.

dgl-bot · 2023-12-06T15:24:39Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-12-06T15:24:51Z

Commit ID: ee0bf3a

Build ID: 5

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-12-07T15:40:26Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-12-07T15:40:39Z

Commit ID: d2ebca96bc827ec87678623b6dca4cf75a2df841

Build ID: 6

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-12-07T15:40:48Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-12-11T15:24:26Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

rudongyu · 2023-12-11T15:24:31Z

@dgl-bot

dgl-bot · 2023-12-11T15:24:38Z

Commit ID: babe0a3

Build ID: 21

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-12-11T16:09:11Z

Commit ID: babe0a3

Build ID: 22

Status: ❌ CI test failed in Stage [Torch GPU Unit test].

Report path: link

Full logs path: link

dgl-bot · 2023-12-11T16:46:39Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-12-11T16:46:51Z

Commit ID: a8fa597

Build ID: 23

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

rudongyu · 2023-12-11T16:54:32Z

@dgl-bot

dgl-bot · 2023-12-11T16:54:36Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-12-11T16:54:48Z

Commit ID: 196adc1

Build ID: 24

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-12-11T17:31:49Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-12-11T17:32:01Z

Commit ID: ba97ec2

Build ID: 26

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-12-11T17:37:26Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-12-11T17:37:38Z

Commit ID: 7baf335

Build ID: 27

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-12-11T17:39:15Z

Commit ID: 196adc1

Build ID: 25

Status: ❌ CI test failed in Stage [Torch GPU Unit test].

Report path: link

Full logs path: link

dgl-bot · 2023-12-11T17:41:16Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-12-11T17:41:29Z

Commit ID: c6a6d6e

Build ID: 28

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-12-11T17:42:51Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

rudongyu · 2023-12-11T17:42:58Z

@dgl-bot

dgl-bot · 2023-12-11T17:43:04Z

Commit ID: 803d026

Build ID: 29

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-12-11T18:28:02Z

Commit ID: 803d026

Build ID: 30

Status: ✅ CI test succeeded.

Report path: link

Full logs path: link

rudongyu

@OlegPlatonov Thank you for your contribution! The PR will be merged.

Co-authored-by: rudongyu <ru_dongyu@outlook.com>

OlegPlatonov added 5 commits October 31, 2023 15:41

add adjusted homophily

353df90

add label informativeness

9e8fb70

add tests

a2c43f7

update python/dgl/__init__.py

b09f909

fix code formatting

d7c40db

frozenbugs requested a review from rudongyu November 2, 2023 02:52

rudongyu reviewed Nov 15, 2023

View reviewed changes

OlegPlatonov added 6 commits December 5, 2023 17:00

use more efficient implementation of adjusted homophily

d6e3f42

use more efficient implementations of label informativeness

c2371ab

update docstrings for label informativeness

17d789a

add new functions to dgl.rst

c990bd0

Merge branch 'master' into adjusted_homophily_and_label_informativeness

02251da

slightly change values of label informativeness in the examples accor…

bd5f0b4

…ding to the new implementation

Merge branch 'dmlc:master' into adjusted_homophily_and_label_informat…

dbc9e94

…iveness

Merge branch 'master' into adjusted_homophily_and_label_informativeness

ee0bf3a

OlegPlatonov force-pushed the adjusted_homophily_and_label_informativeness branch from 0af72aa to 0eedce6 Compare December 7, 2023 15:40

Update label_informativeness.py

babe0a3

Update homophily.py

a8fa597

Update label_informativeness.py

196adc1

Update label_informativeness.py

ba97ec2

Update test_label_informativeness.py

7baf335

Update test_label_informativeness.py

c6a6d6e

Update test_label_informativeness.py

803d026

rudongyu approved these changes Dec 11, 2023

View reviewed changes

rudongyu merged commit ffe2871 into dmlc:master Dec 11, 2023
2 checks passed

DominikaJedynak pushed a commit to DominikaJedynak/dgl that referenced this pull request Mar 12, 2024

[Feature] Adjusted homophily and label informativeness (dmlc#6516)

8ab534f

Co-authored-by: rudongyu <ru_dongyu@outlook.com>

[Feature] Adjusted homophily and label informativeness #6516

[Feature] Adjusted homophily and label informativeness #6516

Conversation

OlegPlatonov commented Nov 1, 2023

Description

Checklist

Changes

dgl-bot commented Nov 1, 2023

dgl-bot commented Nov 1, 2023

frozenbugs commented Nov 2, 2023

dgl-bot commented Nov 2, 2023

rudongyu left a comment

Choose a reason for hiding this comment

dgl-bot commented Dec 5, 2023

dgl-bot commented Dec 5, 2023

dgl-bot commented Dec 5, 2023

dgl-bot commented Dec 5, 2023

OlegPlatonov commented Dec 5, 2023

dgl-bot commented Dec 6, 2023

dgl-bot commented Dec 6, 2023

dgl-bot commented Dec 7, 2023

dgl-bot commented Dec 7, 2023

dgl-bot commented Dec 7, 2023

dgl-bot commented Dec 11, 2023

rudongyu commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

rudongyu commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

rudongyu commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

dgl-bot commented Dec 11, 2023

rudongyu left a comment

Choose a reason for hiding this comment