Skip to content

Add visualization tools for sparsity patterns #12

@MaxGhenis

Description

@MaxGhenis

Description

Create utilities to visualize which weights/neurons/filters are being pruned by L0.

Motivation

Understanding sparsity patterns helps debug issues and provides insights into what the model considers important.

Proposed Visualizations

from l0.visualization import plot_sparsity

# 1. Layer-wise sparsity
plot_sparsity.by_layer(model)  # Bar chart of sparsity per layer

# 2. Weight heatmaps
plot_sparsity.weight_heatmap(model.fc1)  # Heatmap of gate values

# 3. Sparsity evolution
plot_sparsity.evolution(checkpoint_dir)  # Sparsity over training

# 4. Filter importance (for CNNs)
plot_sparsity.filter_importance(model.conv1)  # Which filters are kept

# 5. Attention head importance (for Transformers)
plot_sparsity.head_importance(model.attention)

Features

  • Matplotlib/Plotly backends
  • Interactive visualizations
  • Export to images/HTML
  • Integration with TensorBoard/WandB

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions