Implement degree assortativity coefficient app #492

Ningsir · 2021-07-07T13:17:43Z

What do these changes do?

Implement degree assortativity coefficient app with cpp.

implement C++ PIE app with analytical engineAPI;
check the correctness with NetworkX
check the scalability and performance in large graph, like com-friendster dataset
add to python built-in algorithms and add related test case

Related issue number

#453

Experiment

Experiment setting:

Ubuntu20.04;
32G memory;
workers_num = 4;
NetworkX version: 2.6.2;
dataset: Twitter, datagen-7_5-fb, datagen-7_7-zf, datagen-8_0-fb.

Time	GraphScope	NetworkX	GraphScope(weight=1)	NetworkX(weight=1)
Twitter	1.37 S	3.2 S	26.71 S	90.25 S
datagen-7_5-fb	26.72 S	136.07 S	162.67 S	4018.23 S
datagen-7_7-zf	22.27 S	169.96 S	35.86 S	623.15 S
datagen-8_0-fb	58.20 S	844.42 S	543.95 S	OOM

Memory Usage
Twitter	0.96 G	0.578 G	1.38 G	1.31 G
datagen-7_5-fb	8 G	13.28 G	9.6 G	16.42 G
datagen-7_7-zf	23.04 G	25.92 G	24.32 G	26.24 G
datagen-8_0-fb	25.6 G	30.78 G	29.44 G	OOM

CLAassistant · 2021-07-07T13:17:49Z

All committers have signed the CLA.

acezen

nice start.

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h

acezen · 2021-07-09T06:25:57Z

hi, @Ningsir, please assign the cla agreement and pass the ci, especially the code format check.

acezen · 2021-08-02T06:34:01Z

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h

+  using oid_t = typename fragment_t::oid_t;
+  using edata_t = typename fragment_t::edata_t;
+
+  typedef double degree_t;


using degree_t = double;

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h

acezen · 2021-08-04T17:16:09Z

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h

+  void PEval(const fragment_t& frag, context_t& ctx,
+             message_manager_t& messages) {
+    ctx.merge_stage = false;
+    // vid


remove unnecessary comments

acezen · 2021-08-04T17:16:24Z

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h

+    ctx.merge_stage = false;
+    // vid
+    auto inner_vertices = frag.InnerVertices();
+    // v of type: Vertex


analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient_context.h

acezen · 2021-08-04T17:19:27Z

analytical_engine/apps/assortativity/utils.h

+#define ANALYTICAL_ENGINE_APPS_ASSORTATIVITY_UTILS_H_
+
+namespace gs {
+enum class DegreeType { IN, OUT, INANDOUT };


just put enum to context.h, no need to add a new file

DegreeType and some codes about matrix computation can be reused in numeric assortativity app. I will rafactor and put these codes in utils.h

analytical_engine/apps/assortativity/utils.h

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h

acezen · 2021-08-11T02:23:41Z

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h

+  }
+
+  template <typename T>
+  degree_t ComputeWeightedDegree(T adjList) {


add some comment about the function. If the function is simple, we should use inline

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h

acezen

hi, I add some reviews on the latest pr.
Can you add a test case with p2p-31? like

GraphScope/python/graphscope/nx/tests/test_ctx_builtin.py

Lines 219 to 221 in 35b2fdf

    
           def test_katz_centrality(self): 
        
               ans = dict(nx.builtin.katz_centrality(self.p2p, weight="default").values) 
        
               self.assert_result_almost_equal(ans, self.p2p_kz_ans)

acezen

Good job! Thanks for the contribution.

Ningsir added 2 commits July 5, 2021 20:28

create assortativity

f499a5e

add degree assortativity

97a461a

acezen changed the title ~~add degree assortativity in cpp~~ [WIP] Add degree assortativity in cpp Jul 8, 2021

acezen requested changes Jul 8, 2021

View reviewed changes

Ningsir marked this pull request as draft July 8, 2021 13:10

format check

6e8e4f4

Ningsir marked this pull request as ready for review July 9, 2021 07:09

Ningsir and others added 9 commits July 9, 2021 20:38

format check

174b364

format check

5406cc1

implement attribute assortativity

6ceb0cc

format check

9694e0c

add average degree connectivity test

b2fa86e

implement average degree connectivity

29b1e8f

Merge branch 'alibaba:main' into main

6dacecf

make gsa_clformat

a3d99e2

delete attribute and average degree apps

77f7ed6

acezen mentioned this pull request Jul 12, 2021

[WIP] Implement degree assortativity app with cpp #514

Closed

Ningsir and others added 6 commits July 13, 2021 21:58

degree assortativity app refactoring

3911d09

utils format check

05918f7

Modify message fotmat and send method

ea95fa4

implement degree assortativity with python

88519f5

add degree assortativity tests

1e4c313

Merge branch 'main' into main

7e3aeae

yecol marked this pull request as draft July 15, 2021 07:22

Ningsir and others added 3 commits July 16, 2021 11:48

format

29c6a65

format

f14379f

Merge branch 'alibaba:main' into main

eb6920f

acezen reviewed Aug 2, 2021

View reviewed changes

acezen reviewed Aug 4, 2021

View reviewed changes

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h Show resolved Hide resolved

acezen reviewed Aug 4, 2021

View reviewed changes

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient_context.h Show resolved Hide resolved

acezen reviewed Aug 4, 2021

View reviewed changes

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient_context.h Outdated Show resolved Hide resolved

acezen reviewed Aug 4, 2021

View reviewed changes

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient_context.h Outdated Show resolved Hide resolved

acezen reviewed Aug 4, 2021

View reviewed changes

Ningsir changed the title ~~[WIP] Add degree assortativity in cpp~~ [WIP] Implement degree assortativity coefficient app Aug 9, 2021

Ningsir and others added 3 commits August 10, 2021 10:36

Merge branch 'alibaba:main' into main

0628941

remove useless comments and refactor

7418118

Merge branch 'main' of https://github.com/Ningsir/GraphScope into degree

806ce9d

acezen reviewed Aug 11, 2021

View reviewed changes

analytical_engine/apps/assortativity/utils.h Outdated Show resolved Hide resolved

acezen reviewed Aug 11, 2021

View reviewed changes

analytical_engine/apps/assortativity/utils.h Outdated Show resolved Hide resolved

acezen reviewed Aug 11, 2021

View reviewed changes

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h Outdated Show resolved Hide resolved

acezen reviewed Aug 11, 2021

View reviewed changes

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h Outdated Show resolved Hide resolved

acezen reviewed Aug 11, 2021

View reviewed changes

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h Outdated Show resolved Hide resolved

acezen reviewed Aug 11, 2021

View reviewed changes

analytical_engine/apps/assortativity/degree/degree_assortativity_coefficient.h Outdated Show resolved Hide resolved

acezen requested changes Aug 11, 2021

View reviewed changes

Ningsir added 4 commits August 13, 2021 16:47

add p2p-31 test

56544e0

add comments

adf6a62

add p2p-31 test

705f8a6

remove degree assortativity in run_app.cc

019d07a

Ningsir force-pushed the main branch from cacc642 to 019d07a Compare August 17, 2021 10:12

Merge branch 'alibaba:main' into main

0e61e08

acezen changed the title ~~[WIP] Implement degree assortativity coefficient app~~ Implement degree assortativity coefficient app Aug 17, 2021

acezen approved these changes Aug 17, 2021

View reviewed changes

acezen merged commit e092774 into alibaba:main Aug 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement degree assortativity coefficient app #492

Implement degree assortativity coefficient app #492

Ningsir commented Jul 7, 2021 •

edited

Loading

CLAassistant commented Jul 7, 2021 •

edited

Loading

acezen left a comment

acezen commented Jul 9, 2021

acezen Aug 2, 2021 •

edited

Loading

Ningsir Aug 2, 2021

acezen Aug 4, 2021

acezen Aug 4, 2021

acezen Aug 4, 2021

Ningsir Aug 5, 2021

acezen Aug 11, 2021

acezen left a comment

acezen left a comment

	def test_katz_centrality(self):
	ans = dict(nx.builtin.katz_centrality(self.p2p, weight="default").values)
	self.assert_result_almost_equal(ans, self.p2p_kz_ans)

Implement degree assortativity coefficient app #492

Implement degree assortativity coefficient app #492

Conversation

Ningsir commented Jul 7, 2021 • edited Loading

What do these changes do?

Related issue number

Experiment

CLAassistant commented Jul 7, 2021 • edited Loading

acezen left a comment

Choose a reason for hiding this comment

acezen commented Jul 9, 2021

acezen Aug 2, 2021 • edited Loading

Choose a reason for hiding this comment

Ningsir Aug 2, 2021

Choose a reason for hiding this comment

acezen Aug 4, 2021

Choose a reason for hiding this comment

acezen Aug 4, 2021

Choose a reason for hiding this comment

acezen Aug 4, 2021

Choose a reason for hiding this comment

Ningsir Aug 5, 2021

Choose a reason for hiding this comment

acezen Aug 11, 2021

Choose a reason for hiding this comment

acezen left a comment

Choose a reason for hiding this comment

acezen left a comment

Choose a reason for hiding this comment

Ningsir commented Jul 7, 2021 •

edited

Loading

CLAassistant commented Jul 7, 2021 •

edited

Loading

acezen Aug 2, 2021 •

edited

Loading