Friendster dataset in Benchmark

[initzhang]

🐛 Bug

There is a potential problem about the friendster dataset used in the benchmark scripts:

dgl/benchmarks/benchmarks/utils.py

Lines 131 to 140 in ae7e3db

	def get_friendster():
	# Same as https://snap.stanford.edu/data/bigdata/communities/com-friendster.ungraph.txt.gz
	_download('https://dgl-asv-data.s3-us-west-2.amazonaws.com/dataset/friendster/com-friendster.ungraph.txt.gz',
	'/tmp/dataset/friendster', 'com-friendster.ungraph.txt.gz')
	df = pandas.read_csv('/tmp/dataset/friendster/com-friendster.ungraph.txt.gz', sep='\t', skiprows=4, header=None,
	names=['src', 'dst'], compression='gzip')
	src = df['src'].values
	dst = df['dst'].values
	print('construct the graph')
	return dgl.graph((src, dst))

the node ID in the com-friendster.ungraph.txt does not increase from 0 and continuously, so the returned graph is also wrong (in terms of number of nodes and edges). Maybe it will affect the benchmark results?

To Reproduce

Steps to reproduce the behavior:

Expected behavior

Environment

• DGL Version (e.g., 1.0):
• Backend Library & Version (e.g., PyTorch 0.4.1, MXNet/Gluon 1.3):
• OS (e.g., Linux):
• How you installed DGL (conda, pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version (if applicable):
• GPU models and configuration (e.g. V100):
• Any other relevant information:

Additional context

[jermainewang]

Hi, thanks for reporting. I'd like to understand how severe is the problem. Do you know how many nodes in the constructed graph are isolated (no in- and out- edges)?

[initzhang]

According to the description in the friendster website (http://snap.stanford.edu/data/com-Friendster.html), the dataset contains 65,608,366 connected nodes with 1,806,067,135 edges.

While the current benchmark script generates a graph with 124,836,180 nodes and 1,806,067,135 edges, where the number of nonzero elements of g.in_degrees() + g.out_degrees() is 65,608,366.

So approximately 50% isolated nodes.

[jermainewang]

I see. The inflation of node ID space may harm locality for reading node features because they will be more scattered. It won't influence the computation amount which depends on the number of edges. I think it is worth improving. Would you like to help? The compact_graphs API should be useful.

[initzhang]

Sure! I will follow up and modify related scripts soon.

[initzhang]

Hi, I have launched a PR at #4009