Skip to content
SparklingGraph provides easy to use set of features that will give you ability to proces large scala graphs using Spark and GraphX.
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
api migration to sbt 1.0.2 May 30, 2018
examples migration to sbt 1.0.2 May 30, 2018
experiments migration to sbt 1.0.2 May 30, 2018
generators
loaders fixing build issues for scala 2.12 Apr 2, 2019
operators commenting out bad test Apr 2, 2019
project fixing git versioning Apr 2, 2019
utils
.gitignore deployment fix Apr 2, 2019
.sbtopts fixing build issues for scala 2.12 Apr 2, 2019
.travis.yml fixing git versioning Apr 2, 2019
CONTRIBUTING.md
LICENSE Updating LICENSE to BSD 2-clause "Simplified" License May 16, 2017
README.md Merge branch 'master' of github.com:sparkling-graph/sparkling-graph Apr 2, 2019
build.sbt versions fix Apr 2, 2019
codecov.yml
sonatypeCredentials.sbt

README.md

sparkling-graph

Build Status codecov Documentation Status Codacy Badge Maven Central MLOSS Spark Packages API Gitter FOSSA Status

SparklingGraph provides easy to use set of features that will give you ability to proces large scala graphs using Spark and GraphX.

Requirements

  • Scala 2.11 or 2.12
  • Spark 2.4.0 (or compatible)

Versioning

Since commit 3246714 project is using git versioning (for example 0.0.7+140-32467140 or 0.0.7+140-32467140+20190402-2057-SNAPSHOT). All artifacts from now one will be published to snapshot without version overriding. New approach will also add abbility to reproduce each version. Release versions will use normal tag based approach.

Dependencies

Since commit 3246714 you can get artifacts for any master branch commits using git describe command.

Snapshot

resolvers +=  "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"
// one or all from:
libraryDependencies += "ml.sparkling" %% "sparkling-graph-examples" % "0.0.8-SNAPSHOT"
libraryDependencies += "ml.sparkling" %% "sparkling-graph-loaders" % "0.0.8-SNAPSHOT"
libraryDependencies += "ml.sparkling" %% "sparkling-graph-operators" % "0.0.8-SNAPSHOT"

Release

// one or all from:
libraryDependencies += "ml.sparkling" %% "sparkling-graph-examples" % "0.0.7"
libraryDependencies += "ml.sparkling" %% "sparkling-graph-loaders" % "0.0.7"
libraryDependencies += "ml.sparkling" %% "sparkling-graph-operators" % "0.0.7"

Current features

  • Loading
    • Formats:
      • CSV
      • GraphML
    • DSL
  • Measures - measures can be configured to treat graphs as directed and undirected
    • Measures DSL - easy to use domain specific language that boost productivity of library
    • Graph
      • Modularity
      • Freeman's network centrality
    • Vertex
      • Closeness
      • Local clustering
      • Eigenvector
      • Hits
      • Neighbor connectivity
      • Vertex embeddedness
      • Betweenness
        • Edmonds
        • Flow
        • Hua
    • Edges
      • Adamic/Adar
      • Common neighbours
  • Comunity detection methods
    • PSCAN (SCAN)
  • Graph coarsening
    • Label Propagation based
  • Link prediction
    • Similarity measure based
  • Generators
    • Ring
    • Watts And Strogatz
  • Experiments
    • Describe graph using all measures to CSV files

Planned features

  • Loading
    • GML
  • Measures
    • Katz
  • Comunity detection methods
    • Modularity maximization
    • Infomap
  • More Generators
  • API
    • Random walk
    • BFS
  • ML
    • Vertex classification

Used by

Supported by:

provides us awesome IDE

How to

Please check API, examples or docs

Citation

If you use SparklingGraph in your research and publish it, please consider citing us, it will help us get funding for making the library better. Currently manuscript is in preparation, so please us following references:

Bartusiak et al. (2017). SparklingGraph: large scale, distributed graph processing made easy. Manuscript in preparation.

@unpublished{sparkling-graph
title={SparklingGraph: large scale, distributed graph processing made easy},
author={Bartusiak R., Kajdanowicz T.},
note = {Manuscript in preparation},
year = {2017}
}

License

FOSSA Status

You can’t perform that action at this time.