cosmo_notebooks

A collection of cosmograph notebooks.

Datasets Overview

Introduction

This repository contains visualization notebooks for various datasets for various projects.

The notebooks all use cosmodata to source, cache, and manage data. The notebooks are self-sufficient (they pip install the stuff they need to install, source the data they need, etc.)

The raw data consists of structured information from sources like academic publications, GitHub repositories, political debates, and Spotify playlists. The prepared datasets (linked in notebooks) feature embeddings and 2D projections that enable scatter and force-directed graph visualizations.

Dataset Descriptions

EuroVis Dataset

• Raw Data: Academic publications metadata from the EuroVis conference, including titles, abstracts, authors, and awards.
• Prepared Data: merged_artifacts.parquet (5599 rows, 18 columns)
  • Potential columns for visualization:
    • X & Y Coordinates: x, y
    • Point Size: n_tokens (number of tokens in the abstract)
    • Color: Cluster labels (cluster_05, cluster_08, etc.)
    • Label: title
  • More:
    • visualization notebook
    • data prep module eurovis.py

Harris vs Trump Debate Dataset

• Raw Data: Transcript of a political debate between Kamala Harris and Donald Trump.
• Prepared Data: harris_vs_trump_debate_with_extras.parquet (1,141 rows, 21 columns)
  • Potential columns for visualization:
    • X & Y Coordinates: tsne__x, tsne__y, pca__x, pca__y
    • Point Size: certainty
    • Color: speaker_color
    • Label: text
  • More:
    • visualization notebook

Spotify Playlists Dataset

• Raw Data: Metadata on popular songs from various playlists, including holiday songs and the greatest 500 songs.
• Prepared Data: holiday_songs_spotify_with_embeddings.parquet (167 rows, 27 columns)
  • Potential columns for visualization:
    • X & Y Coordinates: umap_x, umap_y, tsne_x, tsne_y
    • Point Size: popularity
    • Color: genre (derived from playlist)
    • Label: track_name
  • More:
    • visualization notebook

Quotes Dataset

• Raw Data: Collection of 1,638 famous quotes.
• Prepared Data: micheleriva_1638_quotes_planar_embeddings.parquet (1,638 rows, 3 columns)
  • Potential columns for visualization:
    • X & Y Coordinates: x, y
    • Label: quote
  • More:
    • visualization notebook

Prompt Injections Dataset

• Raw Data: Data related to prompt injection attacks and defenses.
• Prepared Data: prompt_injection_w_umap_embeddings.tsv (662 rows, 6 columns)
  • Potential columns for visualization:
    • X & Y Coordinates: x, y
    • Point Size: size
    • Color: label
    • Label: text
  • More:
    • data prep module prompt_injections.py
    • visualization notebook

LMSys Chat Conversations Dataset

• Raw Data: Conversations from AI chat systems.
• Prepared Data: lmsys_with_planar_embeddings_pca500.parquet (2,835,490 rows, 38 columns)
  • Potential columns for visualization:
    • X & Y Coordinates: x_umap, y_umap
    • Point Size: num_of_tokens
    • Color: model
    • Label: content
  • Related code file: lmsys_ai_conversations.py

HCP Publications Dataset

• Raw Data: Human Connectome Project (HCP) publications and citation networks.
• Prepared Data: aggregate_titles_embeddings_umap_2d_with_info.parquet (340,855 rows, 9 columns)
  • Potential columns for visualization:
    • X & Y Coordinates: x, y
    • Point Size: n_cits (citation count)
    • Color: main_field (research domain)
    • Label: title
  • Related code file: hcp.py

GitHub Repositories Dataset

• Raw Data: GitHub repository metadata including stars, forks, programming languages, and repository descriptions, from kaggle dataset
• Prepared Data: github_repositories.parquet (3,065,063 rows, 28 columns)
  • Potential columns for visualization:
    • X & Y Coordinates: x, y
    • Point Size: stars (star count), forks
    • Color: primaryLanguage
    • Label: nameWithOwner
  • Related code file: github_repos.py