Skip to content
This repository has been archived by the owner on Jun 30, 2021. It is now read-only.

Update Spark Job with new WriteGraph; Resolves #211 #212

Merged
merged 4 commits into from Aug 26, 2019
Merged

Conversation

ianmilligan1
Copy link
Member

This should not be merged until AUT 0.18 is released with the new WriteGraph functionality

GitHub issue(s):

#211

What does this Pull Request do?

This is a relatively straight-forward pull request that changes the syntax in the Spark Job from the older WriteGraphML to the newer WriteGraph.asGraphml. This has the primary effect of turning node IDs for the networks from long hashes to simple numbers.

It also updates the application.yml.example as if users are using a version before 0.18.0 the command won't work.

I figured this should be done on a branch as I wanted to particularly make sure that the ensuing networks work well in Gephi. They appear to do so.

How should this be tested?

I have tested locally but basically we will want to see that (a) the standard suite of derivatives are generated; and (b) the ensuing network files work. The two screenshots show some local testing, showing how the new graphs render well after GraphPass and also work well in Gephi.

screen shot 2018-11-28 at 11 14 28 am

screen shot 2018-11-28 at 11 18 10 am

Additional Notes:

As noted above, only merge once new AUT is released. This should not require new documentation.

Interested parties

@ruebot @greebie

@codecov-io
Copy link

codecov-io commented Nov 28, 2018

Codecov Report

Merging #212 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@          Coverage Diff           @@
##           master    #212   +/-   ##
======================================
  Coverage    89.4%   89.4%           
======================================
  Files          35      35           
  Lines         623     623           
======================================
  Hits          557     557           
  Misses         66      66

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e523b6...5a3df25. Read the comment docs.

@ruebot ruebot added the on hold label Dec 11, 2018
@ruebot ruebot merged commit a10db14 into master Aug 26, 2019
@ruebot ruebot deleted the issue-211 branch August 26, 2019 14:03
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants