Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Major Refactoring #260

Merged
merged 31 commits into from
Jan 19, 2024
Merged

Major Refactoring #260

merged 31 commits into from
Jan 19, 2024

Conversation

danbraunai-apollo
Copy link
Contributor

@danbraunai-apollo danbraunai-apollo commented Dec 18, 2023

Major Refactoring

Description

Major refactor. See PRs (and commits) within this PR for descriptions of each change. The main changes are:

  • All rib build experiments are now housed in rib_scripts/rib_build and all ablation experiments in rib_scripts/ablations.
  • LMs and MLPs use the same build and ablation scripts
  • The results of rib builds are now a RibBuildResults pydantic model. Loaded results should be implanted in this model for validation.

Related Issue

Closes #228; Closes #236; Closes #266; Closes #265; Closes #261; Closes #192

Does this PR introduce a breaking change?

Yes. Too many to enumerate. See PRs within this PR for a non-exhaustive list.

danbraunai-apollo and others added 9 commits December 19, 2023 14:10
* Combine lm and mlp ablation scripts

* Use sample graphs in ablation configs
* Move rib build and plot inside rib package

* Rename experiments to rib_scripts and update setup.py

* Move all ablation logic to rib/ablations.py

* Test update venv in CI

* Revert test change in CI

* Fix package finding in setup.py
* Make RibResults a pydantic model

* Save model config object instead of dict in RibResults
* Store edges in Edges pydantic model

* Simplify edge_combiner
* Rename variables and clean docs

* Update rib_builder.py module docstring
@danbraunai-apollo danbraunai-apollo changed the title (WIP) Unification of lm and mlp Major Refactoring Jan 3, 2024
Copy link
Collaborator

@nix-apollo nix-apollo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks very good. I haven't run any non-test code.

I also will make a PR with miscellaneous cleanups.

.github/workflows/checks.yaml Outdated Show resolved Hide resolved
.github/workflows/checks.yaml Outdated Show resolved Hide resolved
rib/ablations.py Outdated Show resolved Hide resolved
rib/ablations.py Outdated Show resolved Hide resolved
rib/analysis_utils.py Outdated Show resolved Hide resolved
rib/rib_builder.py Outdated Show resolved Hide resolved
rib/rib_builder.py Show resolved Hide resolved
rib/data.py Show resolved Hide resolved
rib/plotting.py Outdated Show resolved Hide resolved
rib/plotting.py Outdated Show resolved Hide resolved
danbraunai-apollo and others added 6 commits January 17, 2024 10:07
* ablations cleanups

* ignore old out files

* edge storing loop simplify

* merge model loading

* clean test configs

* revert load dataset chagne

* Fix naming and docs

* important fix

---------

Co-authored-by: Dan Braun <dan@apolloresearch.ai>
* Rename torch types

* Fix raw input torch types

* Fix tuple annotation for unknown emb size

* Fix docs and types
* Remove extra return_set arg and fix model_copy

* Move get_config functions to tests.utils
rib/ablations.py Outdated Show resolved Hide resolved
rib/ablations.py Outdated Show resolved Hide resolved
danbraunai-apollo added a commit that referenced this pull request Jan 18, 2024
* Combine Eigenvectors object with InteractionRotation

* Address misc comments in PR #260

* Rename variables and update docs
@danbraunai-apollo danbraunai-apollo merged commit 648a674 into main Jan 19, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment