cuDNN R2 #1731

shelhamer · 2015-01-15T07:30:16Z

See #1854 for the initial cuDNN R2 compatible branch by @slayton58.

Although cuDNN R2 is still experimental it is in the release candidate stage of development, so this PR will ready Caffe for R2. This should not be made the default until R2 is done or we add version switching for compatibility.

Note that R2 brings OS X support.

Compatibility

create general tensor, but set 4D tensor
set conv 2D descriptors
set pooling 2D descriptors
replace accumulation flags with alpha, beta scaling
let pooling have padding
pick fastest by default
pick algorithm in Reshape() instead of Forward()
expose choice of the forward convolutional algorithm / preference

Perhaps

keep R1 compatibility #if CUDNN_VERSION < 20 ?
switch to the N-D interface in coordination with Blobs are N-D arrays (for N not necessarily equals 4) #1486
3D convolution

* A sample code was added. * `slice_dim` and `slice_point` attributes were explained.

[docs] brief explanation of SLICE layer's attributes

See https://github.com/BVLC/caffe/blob/master/models/bvlc_reference_caffenet/solver.prototxt

Correct 'epochs' to 'iterations'

Next: release candidater

…current project

fix Imagenet example path

@mees

set the right rpath for tools and examples respectively thanks for the report @mees!

[build] fix dynamic linking of tools

… was overwritten with symlink created at build time and installed with install(DIRECTORY ...)

num/channnels/height/width indexing is valid.

from saved NetParameter Want to keep the param Blob shape the layer has set, and not necessarily adopt the one from the saved net (e.g. want to keep new 1D bias shape, rather than take the (1 x 1 x 1 x D) shape from a legacy net).

Blobs are N-D arrays (for N not necessarily equals 4)

cmake and python3 bugfixes for BVLC#1939 and BVLC#1923

When setting the mean, assert that it is either one pixel or an array with shape equal to the input data size.

Check shape of input mean

add <climits> for INT_MAX

follow-up to BVLC#2031: check that the input and mean channels are compatible in the broadcast channels case.

Give cuDNN {0, 1} constants for controlling accumulation through the alpha and beta coefficients.

ziyuang · 2015-03-19T16:22:33Z

cuDNN v2 is now final (libcudnn.so.6.5.48).

shelhamer added in progress speed-up labels Jan 15, 2015

This was referenced Jan 15, 2015

cuDNN R2 is available #1622

Closed

can't build the python wrapper with make pycaffe #1604

Closed

installing with cuDNN R2 on OS X 10.10 #1556

Closed

bhack mentioned this pull request Jan 15, 2015

Use new cuDNN R2 API #1739

Closed

shelhamer mentioned this pull request Jan 16, 2015

'cudnn.h' file not found error [OS X] #1572

Closed

bhack mentioned this pull request Jan 23, 2015

incompatibility with cudnn v2-rc2 #1792

Closed

shelhamer and others added 7 commits January 24, 2015 18:27

clarify draw_net.py usage: net prototxt, not caffemodel

2f869e7

[docs] ask install + hardware questions on caffe-users

61c63f6

[docs] send API link to class list

4cc8195

[docs] add check mode hint to CPU-only mode error

1f7c3de

Brief explanation of SLICE layer's attributes

8b96472

* A sample code was added. * `slice_dim` and `slice_point` attributes were explained.

lint 1f7c3de

75d0e16

Merge pull request BVLC#1817 from boechat107/patch-1

e3c895b

[docs] brief explanation of SLICE layer's attributes

shelhamer force-pushed the cudnn-r2 branch from 079ff55 to adcaebf Compare February 16, 2015 20:03

shelhamer mentioned this pull request Feb 16, 2015

Use cuDNN R2 RC2 #1854

Closed

Brandon Amos and others added 2 commits February 16, 2015 15:09

Correct 'epochs' to 'iterations'

1e0d49a

See https://github.com/BVLC/caffe/blob/master/models/bvlc_reference_caffenet/solver.prototxt

Merge pull request BVLC#1879 from bamos/patch-1

3e9b050

Correct 'epochs' to 'iterations'

shelhamer force-pushed the cudnn-r2 branch from adcaebf to 4b29af9 Compare February 17, 2015 01:32

shelhamer mentioned this pull request Feb 17, 2015

Next: release candidater #1849

Merged

shelhamer and others added 9 commits February 19, 2015 18:35

Merge pull request BVLC#1849 from BVLC/next

f998127

Next: release candidater

Updated the path for get_ilsvrc_aux.sh to match what is found in the …

af01b9c

…current project

Merge pull request BVLC#1914 from eerwitt/master

5ee85b7

fix Imagenet example path

[build] fix dynamic linking of tools

eabbccd

set the right rpath for tools and examples respectively thanks for the report @mees!

Merge pull request BVLC#1921 from shelhamer/fix-tool-linking

682d9da

[build] fix dynamic linking of tools

check caffe tool runs in runtest

5a26333

ignore pycharm files

a1e951d

set proper CMAKE_INSTALL_RPATH for _caffe.so and tools

fca05c3

fixed bug in install-tree: _caffe.so installed by install(TARGET ...)…

645aa03

… was overwritten with symlink created at build time and installed with install(DIRECTORY ...)

jeffdonahue and others added 25 commits March 3, 2015 15:55

SoftmaxLossLayer generalized like SoftmaxLayer

94d93da

SplitLayer: change Reshape(n,h,c,w) to ReshapeLike(...)

e2bc9f9

HDF5DataLayer shapes output according to HDF5 shape

e6468e9

DataLayer outputs 1D labels

e56377d

MemoryDataLayer outputs 1D labels

7c8725b

ImageDataLayer outputs 1D labels

c87a136

WindowDataLayer outputs 1D labels

9505001

EuclideanLossLayer: generalized Blob axes

fcbb933

DummyDataLayer outputs blobs of arbitrary shape

7462c84

Add CHECK_EQ(4, ...)s to "vision layers" to enforce that the

69fc1f6

num/channnels/height/width indexing is valid.

PyBlobs support generalized axes

269dafa

[pycaffe] expose Blob.reshape as *args function

aa242aa

[pytest] use non-4d blobs in test_python_layer

8c79d65

Merge pull request BVLC#1970 from jeffdonahue/tensor-blob

85bb397

Blobs are N-D arrays (for N not necessarily equals 4)

Merge pull request BVLC#1966 from philkr/python_fixes

136139b

cmake and python3 bugfixes for BVLC#1939 and BVLC#1923

Add error checking for image mean

642619b

When setting the mean, assert that it is either one pixel or an array with shape equal to the input data size.

Merge pull request BVLC#2031 from NVIDIA/image_mean

c0bc17c

Check shape of input mean

fix comment I forgot about from @shelhamer's review of BVLC#1970

dec148e

include/caffe/common.hpp: add <climits> for INT_MAX (now in blob.hpp)

a3b0fbd

Merge pull request BVLC#2035 from jeffdonahue/include-climits

12e7317

add <climits> for INT_MAX

[pycaffe] check mean channels for transformation

ccfa3dc

follow-up to BVLC#2031: check that the input and mean channels are compatible in the broadcast channels case.

switch to cuDNN R2

91a6597

replace cuDNN alphas and betas with coefficient values

2ddbb04

Give cuDNN {0, 1} constants for controlling accumulation through the alpha and beta coefficients.

cuDNN pooling can pad now

4beebcc

shelhamer force-pushed the cudnn-r2 branch from 4b29af9 to 4beebcc Compare March 5, 2015 03:41

shelhamer mentioned this pull request Mar 5, 2015

cuDNN R2 #2038

Merged

6 tasks

shelhamer closed this Mar 5, 2015

lim0606 mentioned this pull request Jun 27, 2015

How to speed up? I'm trying to run GoogLeNet with Batch normalization, but it is not fast enough Lasagne/Lasagne#312

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cuDNN R2 #1731

cuDNN R2 #1731

shelhamer commented Jan 15, 2015

ziyuang commented Mar 19, 2015

cuDNN R2 #1731

cuDNN R2 #1731

Conversation

shelhamer commented Jan 15, 2015

ziyuang commented Mar 19, 2015