Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Spark Job with new WriteGraph; Resolves #211 #212

wants to merge 2 commits into
base: master


None yet
3 participants
Copy link

commented Nov 28, 2018

This should not be merged until AUT 0.18 is released with the new WriteGraph functionality

GitHub issue(s):


What does this Pull Request do?

This is a relatively straight-forward pull request that changes the syntax in the Spark Job from the older WriteGraphML to the newer WriteGraph.asGraphml. This has the primary effect of turning node IDs for the networks from long hashes to simple numbers.

It also updates the application.yml.example as if users are using a version before 0.18.0 the command won't work.

I figured this should be done on a branch as I wanted to particularly make sure that the ensuing networks work well in Gephi. They appear to do so.

How should this be tested?

I have tested locally but basically we will want to see that (a) the standard suite of derivatives are generated; and (b) the ensuing network files work. The two screenshots show some local testing, showing how the new graphs render well after GraphPass and also work well in Gephi.

screen shot 2018-11-28 at 11 14 28 am

screen shot 2018-11-28 at 11 18 10 am

Additional Notes:

As noted above, only merge once new AUT is released. This should not require new documentation.

Interested parties

@ruebot @greebie

ianmilligan1 added some commits Nov 28, 2018

@ianmilligan1 ianmilligan1 requested a review from ruebot Nov 28, 2018


This comment has been minimized.

Copy link

commented Nov 28, 2018

Codecov Report

Merging #212 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #212   +/-   ##
  Coverage   92.49%   92.49%           
  Files          35       35           
  Lines         533      533           
  Hits          493      493           
  Misses         40       40

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e8f6f26...580d4a7. Read the comment docs.

@ruebot ruebot added the on hold label Dec 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.