Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADD] Minimal prototype for KFAC #43

Merged
merged 8 commits into from
Oct 30, 2023
Merged

[ADD] Minimal prototype for KFAC #43

merged 8 commits into from
Oct 30, 2023

Conversation

f-dangel
Copy link
Owner

Progress on #42.

This adds a first prototype for a linear operator of KFAC. It has very limited support (e.g. only Linear layers, no weight sharing), but I think this is a good milestone to start off with (and to keep the git diffs manageable. The tests compare the per-layer GGN with KFAC for deep linear models with MSELoss for a large number of MC samples from the predictive distribution.

@f-dangel f-dangel requested a review from runame October 27, 2023 19:50
@github-actions
Copy link

github-actions bot commented Oct 27, 2023

Pull Request Test Coverage Report for Build 6671908035

  • 91 of 106 (85.85%) changed or added relevant lines in 4 files are covered.
  • 37 unchanged lines in 2 files lost coverage.
  • Overall coverage decreased (-5.7%) to 87.229%

Changes Missing Coverage Covered Lines Changed/Added Lines %
curvlinops/examples/utils.py 0 3 0.0%
curvlinops/kfac.py 86 98 87.76%
Files with Coverage Reduction New Missed Lines %
curvlinops/examples/utils.py 3 41.67%
curvlinops/fisher.py 34 28.85%
Totals Coverage Status
Change from base Build 6549326356: -5.7%
Covered Lines: 683
Relevant Lines: 783

💛 - Coveralls

Copy link
Collaborator

@runame runame left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I think this is a good starting point and we can start adding the missing cases/functionality. Maybe we can create issues of the missing cases and distribute them among the two of us?

test/conftest.py Show resolved Hide resolved
test/kfac_cases.py Show resolved Hide resolved
test/kfac_cases.py Show resolved Hide resolved
curvlinops/kfac.py Outdated Show resolved Hide resolved
curvlinops/kfac.py Show resolved Hide resolved
curvlinops/kfac.py Outdated Show resolved Hide resolved
test/test_kfac.py Show resolved Hide resolved
@f-dangel f-dangel merged commit a331b05 into development Oct 30, 2023
12 of 13 checks passed
@f-dangel f-dangel deleted the kfac branch October 30, 2023 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants