Skip to content

Commit

Permalink
moved zeroing to blas in batch linear
Browse files Browse the repository at this point in the history
  • Loading branch information
@szagoruyko
szagoruyko committed May 23, 2015
1 parent 3dd5d1d commit 8abe926
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions Linear.lua
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ function Linear:updateOutput(input)
self.output:copy(self.bias:view(1,nunit):expand(#self.output))
self.output:select(2,1):addmv(1, input, self.weight:select(1,1))
else
self.output:zero():addr(1, self.addBuffer, self.bias)
self.output:addmm(1, input, self.weight:t())
self.output:addmm(0, self.output, 1, input, self.weight:t())
self.output:addr(1, self.addBuffer, self.bias)
end
else
error('input must be vector or matrix')
Expand Down

3 comments on commit 8abe926

@hughperkins
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szagoruyko Hi Sergey, I think this breaks Linear:forward on some blases, see this thread karpathy/char-rnn#38 (comment)

@clementfarabet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirming that this is incorrect. self.output is not initialized, and depending on how blas implements the 0*self.output, it may results in NaNs. If no Blas is used, it definitely results in NaNs... This needs to be reverted.

@clementfarabet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proposing a patch here: #351 — it still preserves the blas optimization, but enforces a clean self.output.

Please sign in to comment.