Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I think that when updating weights, should not target all nodes. #4

Closed
cuiguohua opened this issue Feb 27, 2019 · 2 comments
Closed
Assignees

Comments

@cuiguohua
Copy link

Hello, first of all, thank you for sharing, and gave me a lot of help.

def update_weights(self, inputVec, sigma, lrate, bmu):

	"""Update the node Weights.

	Args:
		inputVec (np.array): A weights vector whose distance drives the direction of the update.
		sigma (float): The updated gaussian sigma.
		lrate (float): The updated learning rate.
		bmu (somNode): The best matching unit.
	"""

	dist=self.get_nodeDistance(bmu)
	gauss=np.exp(-dist*dist/(2*sigma*sigma))  # I think gauss will always > 0
	if gauss>0:
		for i in range(len(self.weights)):
			self.weights[i] = self.weights[i] - gauss*lrate*(self.weights[i]-inputVec[i])

In somNode::update_weights() , expression ' gauss > 0 ' will always be true.
So throughout the training process, the weights of all nodes will be changed.

I read some literature. The literature says that the node weights near BMU should be updated, and this neighborhood is gradually reduced, eventually containing only BMU itself.
So, I think we should change ' gauss >0' to 'gauss>x ' (0<x<1)

Thank you again for sharing and looking forward to your reply.

@fcomitani
Copy link
Owner

Hello @cuiguohua,

the radius of the Gaussian is determined by the sigma which is reduced at every step by update_sigma, thus effectively reducing the neighborhood set.

You are however right about the Gaussian being always > 0, unless the sigma gets infinitely small.
I haven't looked at the code in a while and I'm not sure why I put that condition. Possibly it was just a debugging device which forgot to remove.

Just let me think it over and do a couple of extra tests. If I still can't find a justification for its presence I will remove it in the next version.

Thanks.

@fcomitani fcomitani self-assigned this Mar 1, 2019
@fcomitani
Copy link
Owner

The redundant condition was removed, I'm closing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants