Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed inference with nyud-fcn32s-hha #7064

Open
5 tasks
wangxudong-cq opened this issue Oct 20, 2022 · 0 comments
Open
5 tasks

Failed inference with nyud-fcn32s-hha #7064

wangxudong-cq opened this issue Oct 20, 2022 · 0 comments

Comments

@wangxudong-cq
Copy link

Important - read before submitting

Please read the guidelines for contributing before submitting this issue!

Please do not post installation, build, usage, or modeling questions, or other requests for help to Issues.
Use the caffe-users list instead.
This helps developers maintain a clear, uncluttered, and efficient view of the state of Caffe.

Issue summary

Using the model nyud-fcn32s-hha-heavy.caffemodel to infer images in NYU Depth Dataset V2, the result is wrong. Inference script as blow.

import numpy as np
from PIL import Image

import os 
import sys
current_path = os.path.dirname(__file__)
project_path = os.path.dirname(os.path.dirname(__file__))
sys.path.append(project_path)

import caffe
import vis


def zero_multi_padding(in_array, padding_size=0):
    in_channels, h, w = in_array.shape
    padding_array = np.zeros([in_channels, h + 2 * padding_size, w + 2 * padding_size],dtype=in_array.dtype)
    for i in range(in_channels):
        for xx in range(h):
            for yy in range(w):
                padding_array[i, xx + padding_size, yy + padding_size] = in_array[i, xx, yy]
    return padding_array

# the demo image is "2007_000129" from PASCAL VOC

# load image, switch to BGR, subtract mean, and make dims C x H x W for Caffe
# im = Image.open(project_path+'/demo/image.jpg')
# in_ = np.array(im, dtype=np.float32)
# print(in_.shape)
# in_ = in_[:,:,::-1]
# in_ -= np.array((104.00698793,116.66876762,122.67891434))
# in_ = in_.transpose((2,0,1))
# in_pad = zero_multi_padding(in_, 99)
# print(in_pad.shape)
# print(in_.shape)

im = Image.open('/home/azure002/my_worksapce/fcn.berkeleyvision.org/nyu_images/169.jpg')
# im = Image.open('../demo/image.jpg')
new_image = im.resize((1024,1024), Image.Resampling.BICUBIC)
new_image.save(current_path+'/test_init.png')

in_ = np.array(im, dtype=np.float32)
in_ = in_[:,:,::-1]
in_ -= np.array((104.00698793,116.66876762,122.67891434))
in_ = in_.transpose((2,0,1))

# load net
net = caffe.Net(current_path+'/deploy.prototxt', current_path+'/nyud-fcn32s-hha-heavy.caffemodel', caffe.TEST)
# # shape for input (data blob is N x C x H x W), set data
# net.blobs['None'].reshape(1, *in_.shape)
# net.blobs['None'].data[...] = in_
# net.blobs['data'].reshape(1, *in_pad.shape)
# net.blobs['data'].data[...] = in_pad
net.blobs['data'].reshape(1, *in_.shape)
net.blobs['data'].data[...] = in_
# run net and take argmax for prediction
net.forward()
out = net.blobs['score'].data[0].argmax(axis=0)

import matplotlib.pyplot as plt
plt.imshow(out,cmap='gray') 
plt.axis('off')             
plt.savefig(current_path+'/test.png')  

# visualize segmentation in PASCAL VOC colors
voc_palette = vis.make_palette(40)
print(voc_palette)
out_im = Image.fromarray(vis.color_seg(out, voc_palette))
out_im.save(current_path+'/output.png')
masked_im = Image.fromarray(vis.vis_seg(im, out, voc_palette))
masked_im.save(current_path+'/visualization.jpg')

IMAGE: PROCESS:
image

Steps to reproduce

run the python script

Tried solutions

System configuration

  • Operating system: Linux azure002-System-Product-Name 5.4.0-126-generic Sort out models, data, and tools #142-Ubuntu SMP Fri Aug 26 12:12:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
  • Compiler:
  • CUDA version (if applicable):
  • CUDNN version (if applicable):
  • BLAS:
  • Python version (if using pycaffe): Python 3.8.10
  • MATLAB version (if using matcaffe):

Issue checklist

  • read the guidelines and removed the first paragraph
  • written a short summary and detailed steps to reproduce
  • explained how solutions to related problems failed (tick if found none)
  • filled system configuration
  • attached relevant logs/config files (tick if not applicable)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant