SoNAR (IDH) Visualization Prototypes

About

This repository is a collection of experimental prototypes and the final project results developed in the DFG-funded project SoNAR(IDH).

• the visualizations are based on HTML, CSS and JavaScript using the visualization library D3.js
• while the overview view uses pre-processed data (see Data Analysis), the data for the search-based exploration is retrieved live (see below)
• currently, the prototype visualizations do not use frameworks for more modular DOM management. For a follow-up project we recommend the use of frameworks like Vue.js or Svelte to reduce maintenance effort
• currently the visualizations are mostly based on SVGs, in the future a switch to Canvas or WebGL would be advisable for performance reasons
• currently several queries are sent one after the other and then combined in the browser for the visualization. This leads to duplicates and multiple queried nodes, which slow down the processing and increase the amount of data. With more know-how regarding the production of specific complex queries, the performance could probably be improved significantly, at least in part

Data connection with database

The prototypes use a jquery AJAX request to connect with the neo4J database. The code relies on many hard-coded label names, therefor a use with other data will in most cases need significant editing. The neo4J database can be used local or online, thogh credentials are not hidden and our SoNAR database is set up in read-only mode. To use another database change the "url-link", "neo4jLogin" and "neo4jPassword":

$.ajax({
    url: "https://h2918680.stratoserver.net:7473/db/data/transaction/commit",
    type: 'POST',
    data: JSON.stringify({
      "statements": [{
          "statement": cypherStatement,
          "resultDataContents": ["graph"]
        }
      ],
    }),
    contentType: 'application/json',
    beforeSend: function(xhr) {
    xhr.setRequestHeader('Authorization', 'Basic ' + btoa(neo4jLogin + ':' + neo4jPassword));
    },
    accept: 'application/json; charset=UTF-8'
  }).done(function(data) {
  }

External resources

d3.v5.min.js → D3.js: data-driven visualization library based on JavaScript

jquery.min.js → jQuery: JavaScript library

netClustering.js → netClustering: detects clusters in networks using the Clauset, Newman and Moore community detection algorithm

d3-weighted-voronoi.js and d3-voronoi-map.js → d3-weighted-voronoi and d3-voronoi-map

Google Fonts: Open Sans & Material Icons → Open Sans, Material Icons

Experimental prototypes

The visualization concepts in SoNAR(IDH) were developed iteratively and included many small and experimental prototypes that were not created with the approach the data, open up new pathways, to find problems or challenges with the data, or to communicate with our project partners and domain experts for historical network analysis. For further details see the project documentation of AP3. The following list displays and links to code of selected results (note: the code is not yet fully cleaned/commented):

Visualization

Basic Unfolding Edges

[Demo][Code]

Uncertainty in links as oscillated lines

[Demo][Code]

Simple use of a community cluster algorithm

[Demo][Code]

Scrolling through time

[Demo][Code]

Temporal network cascade by communities

[Demo][Code]

Racing Topic Term Bar Charts with Observable

[Demo/Code]

Topic term by year using a voronoi diagram

[Demo][Code]

Morph between a graph and a timeline

[Demo][Code][Simplified Version with CSV Data (Code & Demo)]

Visualization + Interface

Radial graph layout to visualize shared resources

[Demo][Code]

Data exploration tool

[Demo][Code]

Final project prototype

The finalized prototype is a first attempt to combine the SoNAR (IDH) project results inside one functional prototype. Building on the preceding prototyping process, different views were developed that allow exploration of the data from different perspectives and with a focus on a variety of data dimensions. The web-based visualization is thereby divided into two main components:

• A data overview entry, which allows researchers to get a sense of whether SoNAR might be relevant to their specific research field.
• Search-based views, which are based on targeted queries for an entity, provide different perspectives on the data and offer detailed filtering options.

Views have thus emerged that contrast a whole-data, accumulated overview view as an entry point with individual search-based views. Here, one starts from something small (a search query), and can expand it exploratively as needed.

[Demo][Code]

Data Queries

Search for Persons

In order to be able to operate all filters in the visualization and also to filter further persons in the network according to TopicTerm or GeoName, for example, all links to TopicTerm, GeoName, etc. must also be queried for all persons connected with PerName (via SocialRelations and RelationToPerName). Currently, for performance reasons, only direct connections between the person searched for and other people are rendered in the prototype. However, this would only show connections from/to PerName1. To go beyond a first-person network and get a more accurate picture of the network with the structures and communities, one would also have to query all connections between all persons in the network that do not have to do with PerName1. However, this is currently a low performance query which also increases the complexity of the queries, which is why an optimization of the queries is necessary for future implementation.

Search for Topic Terms

In addition to the person search, a keyword search has been implemented, which makes some additional query additions necessary. Keywords (TopicTerms) can be directly linked to persons via the GND, but more often keywords are linked to resources (e.g. publications). For example, a book on a particular aspect of research is often tagged with a specific keyword, but the author of the book may not be. In order not to miss this richness of keywords, it is necessary to iterate over persons directly connected to the keyword as well as over persons indirectly connected via resources. In addition, as with the person search, all other node types must be queried for all persons in order to make the persons filterable, for example, according to corporate bodies or locations. Finally, it is important, especially for the keyword search, that all connections between the persons appearing in the network are retrieved, so that not only free-floating nodes are generated, but person networks are formed. All in all, this retrieval of data is time-intensive in the current implementation, especially for keywords with very many links. It remains to be examined whether this can be optimized. Furthermore, the described implemented search queries are only to be understood as exemplary implementations. In a further development of the concept, other node types (e.g. search for corporate bodies) and customizable searches (e.g. only a certain time period, minimum number of edges for displayed nodes) should be further considered.

Related publications

Bludau, M.-J., Dörk, M. & Tominski, C. (2021). Unfolding Edges for Exploring Multivariate Edge Attributes in Graphs. Poster at Eurographics Conference on Visualization (EuroVis), Zurich, Switzerland. https://doi.org/10.2312/evp20211070

Bludau, M.-J., Dörk, M., Fangerau, H., Halling, T., Leitner, E., Menzel, S., Müller, G., Petras, V., Rehm, G., Neudecker, C., Zellhöfer, D., & Moreno Schneider, J. (2020). SoNAR (IDH): Datenschnittstellen für historische Netzwerkanalyse. In C. Schöch (Hrsg.), DHd 2020 Spielräume: Digital Humanities zwischen Modellierung und Interpretation. Konferenzabstracts. Tagung des Verbands Digital Humanities, Paderborn, Germany (S. 360–362). Verband Digital Humanities im deutschsprachigen Raum e.V.

Menzel, S., Bludau, M.-J., Leitner, E., Dörk, M., Moreno-Schneider, J., Petras, V. and Rehm, G. (accepted). Graph Technologies for the Analysis of Historical Social Networks Using Heterogeneous Data Sources. In Proceedings of Graph Technologies in the Humanities 2019 and 2020. Graph Technologies in the Humanities 2020. Vienna, AU: CEUR Workshop Proceedings.

Credits

Part of the project SoNAR (IDH), a collobaration between Deutsches Forschungszentrum für Künstliche Intelligenz, FH Potsdam, Humboldt-Universität zu Berlin, Staatsbibliothek zu Berlin and Heinrich-Heine-Universität Düsseldorf.

Project partner responsible for the interface design and data visualization: UCLAB, FH Potsdam – University of Applied Sciences

• Mark-Jan Bludau: concept development & coding
• Marian Dörk: project lead at FH Potsdam

License

MIT License

Copyright (c) Mark-Jan Bludau, Marian Dörk, SoNAR Team and contributors

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.