[SPARK-3650] Fix TriangleCount handling of reverse edges

[jegonzal]

This PR causes the TriangleCount algorithm to remove self-edges, direct edges from low-id to high-id (canonical direction), and then remove duplicate edges, before running the triangle count algorithm.

[SparkQA]

Can one of the admins verify this patch?

[ankurdave]

ok to test

[SparkQA]

QA tests have started for PR 2495 at commit d054d33.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2495 at commit d054d33.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20670/

[jegonzal]

What is the status on this patch?

[SparkQA]

Test build #23301 has finished for PR 2495 at commit 0461ed0.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jegonzal]

@ankurdave take a look at this when you get a chance.

[jegonzal]

@rxin and @ankurdave what is the status on this?

[ankurdave]

Jenkins, retest this please.

[SparkQA]

Test build #25737 has finished for PR 2495 at commit 0461ed0.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[ankurdave]

@jegonzal I fixed the compile error in jegonzal#3.

[jegonzal]

Great! What else needs to be done? There was some discussion about how
this might change the semantics of the triangle count function? Is this
still true?

[ankurdave]

I think it just adds a bit of a slowdown for already-canonicalized graphs, which is probably fine considering how expensive triangle count is anyway.

[pwendell]

I'd like to close this issue pending further updates.

[insidedctm]

@pwendell can this be opened again? As per my discussion on the the JIRA ticket this is an issue that came up on the mailing list recently.