Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fully convolutional version of all vgg models via constructor argument #184

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

warmspringwinds
Copy link

Added a feature to use any existing vgg model in a fully convolutional manner.
Backwards compatible, works as an argument fully_conv to any vgg model creator function.
Also the weights are loaded correctly (conversion between Linear to Conv2d. which can be seen in the example images below).

fcn = vgg16(pretrained=True, fully_conv=True)
fcn = fcn.eval()
from torch.autograd import Variable
import torch.nn as nn

import skimage.io as io
import skimage.transform
import numpy as np
import torch

img = skimage.img_as_float(io.imread('bus.jpg'))
img = img - [0.485, 0.456, 0.406]
img = img / [0.229, 0.224, 0.225]
img = img.transpose((2, 0, 1))
img = np.expand_dims(img, 0)
img = img.astype(np.float32)
final_img = torch.from_numpy(img)

fcn.cuda()
final_img = Variable(final_img.cuda())
res = fcn(final_img)

boo = res.cpu().data.numpy()
final = boo.squeeze().argmax(axis=0)
# 779 -- is a class for a bus from imagenet
io.imshow(final == 779)

The following image was evaluated and respective prediction for a bus region was acquired,
which proves that weights from imagenet vgg model were correctly loaded after converting Linear layers to Conv2d.

pytorch_bus
pytorch_bus_pred

Feel free to leave feedback. I can wrap it up with tests if you like it.

@christian-rauch
Copy link

christian-rauch commented Apr 14, 2020

Is there still interest in cleaning the commits and getting this merged? Or is this partially superseded by #643 or other recent changes (like https://pytorch.org/hub/pytorch_vision_fcn_resnet101/)?

@lijiaqi
Copy link

lijiaqi commented May 12, 2020

@christian-rauch Hi. I am also considering adding more flexible ways to instantiate a model. See #2200.

For example, support backbone mode (remove all last fc layers, make pooling layer after last conv layer as optional, etc).

I think keras style is a good implementation. Any thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants