Skip to content

DominikBoenisch/Training-the-Archive

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

Training the Archive

A research project based at the German Ludwig Forum for International Art Aachen and the Hartware MedienKunstVerein (HMKV) in Dortmund, Germany that combines artificial intelligence and museum collection data through machine learning and object recognition. The project is funded by the Digital Culture Programme of the German Federal Cultural Foundation (Kulturstiftung des Bundes).

Over the next four years (2020-2023), the research project "Training the Archive" will explore the possibilities and risks of AI in relation to the automated structuring of data to support curatorial practice and artistic production. Connected to this is the research question of whether AI can learn research processes so that patterns, connections and associations become recognizable that are not apparent to humans in this form. Together with international artists and curators, a procedure is to be developed that will help to make digital archives - such as the collection of the Ludwig Forum Aachen - accessible in a new way.

Blog: https://trainingthearchive.ludwigforum.de/

Logos Logos

Prototype:

Clustering of images from a museum collection to identify interesting links. Subsequently, the human being is to be brought back into the loop, in which networks for image recognition are retrained with the knowledge of curators, for example, in order to make the clusters more dynamic and personalized.

Step by step guide:

  1. Scraping of data from the Open Source Library of the National Gallery of Denmark (SMK).
  2. Extracting feature vectors from Keras Applications or Tensorflow Hub
  3. Generating a dataset for the training using triplet loss
  4. Training of a network with our annotations, which artworks are related and which are not
  5. Clustering of artworks using different neural networks and visualization of the results
  6. Nearest (or even farthest) neighbors and walk through the latent space

Results of the visualization:

All Images © Dominik Bönisch, Ludwig Forum Aachen using data from the Open Source Library of the National Gallery of Denmark (SMK).

Scatterplot

Cluster Example

Gridplot

Cluster Example

Nearest/Farthest neighbors

Cluster Example

Walkthrough the latent space

Walkthrough Example

Walkthrough Example

Walkthrough the latent space using a tube-shaped scatterplot with an arrowed path

Walkthrough Tube Example

Contribute:

You have suggestions or feedback? We are very happy about that. Feel free to write us a line.

Acknowledgment:

The first prototype was developed within the context of the so-called AI school in the LINK programme of the Lower Saxony Foundation in cooperation with the tutors Dr. rer. nat. Jan Sölter and Dr. rer. nat. Thomas Rost.

License:

This prototype is licensed under the GPL-3.0 License. See the license file for detail.

About

Research project combining artificial intelligence and museum collection data through machine learning and object recognition.

Topics

Resources

License

Stars

Watchers

Forks