Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

singular M^T M in G_step #1

Closed
mjvakili opened this issue Apr 7, 2015 · 6 comments
Closed

singular M^T M in G_step #1

mjvakili opened this issue Apr 7, 2015 · 6 comments

Comments

@mjvakili
Copy link
Owner

mjvakili commented Apr 7, 2015

line 109 of lsq.py:

M^T M is not invertible.
@mjvakili
Copy link
Owner Author

mjvakili commented Apr 7, 2015

I'm going to switch to gradient descent for updating G.

@davidwhogg
Copy link
Collaborator

I don't see how that is possible. Can you make the SVD of M^t M and see what the singular values are?

@mjvakili
Copy link
Owner Author

mjvakili commented Apr 7, 2015

eigen_values

@mjvakili
Copy link
Owner Author

mjvakili commented Apr 7, 2015

same plot with logarithmic x-axis only:

eigen_values

@davidwhogg
Copy link
Collaborator

A tiny regularization would fix this. Try that too.

@mjvakili
Copy link
Owner Author

mjvakili commented Apr 8, 2015

Yes, it works.
Ross pointed out that we can accelerate it significantly by bfgs optimizer since we can compute the gradient of the likelihood w.r.t G and give it to the optimizer.

@mjvakili mjvakili closed this as completed Apr 9, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@davidwhogg @mjvakili