Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added markdown version of pretrained_saes.yaml #198

Closed
wants to merge 1 commit into from

Conversation

skunnavakkam
Copy link

Description

This adds a markdown file that converts the yaml file in pretrained_saes.yaml to a nice table in the docs. This was referenced in index.md

No code was changed.

@chanind
Copy link
Collaborator

chanind commented Jun 29, 2024

Was this generated via a script? If so it would be great to commit that script as well, and maybe add it as a target in Makefile so we can run something like make update-pretrained-saes-docs to keep this up-to-date in the future as more SAEs are added to the YAML

@skunnavakkam
Copy link
Author

I did the conversion with Claude and then combed over it to make sure no errors, but I think that scripting shouldn't be too hard and I don't mind implementing

@jbloomAus
Copy link
Owner

Scripting would be good. I'd prefer not to quote those L0's / variance explained since they were on a random piece of text for a stop gap testing solution. Will set up benchmarking soon (after I accept Josh's PR) and will report the stats coming out of there. Thanks for this!

@jbloomAus
Copy link
Owner

I've made a script that does this now. https://jbloomaus.github.io/SAELens/sae_table/

Would love any feedback on the page / suggestions! My current vibe is:

  • Maybe one monolithic table would be nicer?
  • We should probably add comments for each SAE to explain which to use.
  • Once we have SAE Benchmarking running across all of these we should have a toggle for "show me config in detail" vs "show me the summary stats" in detail.

@jbloomAus jbloomAus closed this Jul 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants