Skip to content
Wikipedia graph mining: dynamic structure of collective memory
Jupyter Notebook
Branch: master
Clone or download
Latest commit 60be5d7 Feb 4, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
ipython Viz Feb 4, 2019
project Initial stage. Reading the data. Creating the graph. Learning the gra… Apr 30, 2017
src/main/scala Comments and save signal for students Nov 12, 2018
.gitignore gitignore Feb 3, 2018
LICENSE Initial commit Apr 30, 2017
README.md Update README.md Oct 9, 2017
build.sbt Code cleanup Jul 29, 2017

README.md

WikiBrain

Implementation of the graph learning algorithm presented in Wikipedia graph mining: dynamic structure of collective memory. The learning algorithm is inspired by the Hebbian learning theory.

We also reported the results with interactive graph visualizations in an accompanying blog post.

Dataset

To reproduce the experiments, download the dataset from DOI.

Clone this project and extract the downloaded .zip files to /src/main/resources/ folder.

Change PATH_RESOURCES in Globals.scala to the path to this project on your computer.

Runing the experiments

Open WikiBrainHebbStatic.scala and run the code (Shift+F10 in Intellij Idea).

You may have to change your Spark configuration according to RAM availability on your computer.

val spark = SparkSession.builder
  .master("local[*]") // use all available cores
  .appName("Wiki Brain")
  .config("spark.driver.maxResultSize", "20g") // change this if needed
  .config("spark.executor.memory", "50g") // change this if needed
  .getOrCreate()

Resulting graphs

You will find the resulting graph graph.gexf in PATH_RESOURCES. This file can be opened in Gephi.

You can’t perform that action at this time.