Inference dimension

[laol777]

Can I make the prediction for an image which size is bigger than 512? And how? with assumption that i trained this model on 512x512 images

[laol777]

@nothinglo According to #18 I should make a prediction for global and local features separately, can you describe this more detailed?

[sangenan]

I have the same problem，who can help me thanks

[sangenan]

hello，https://gist.github.com/laol777/a8c6034c3b24e2b3f92cfca2c691312f thank for your help this link has failed. Can you pass it on again? laol <notifications@github.com> 于2019年3月21日周四 下午11:45写道：

…

I figured out the solution: 1. replace FLAGS['data_image_size'], 512 to None, in DATA.py file 2. will run model retraining 3. run this https://gist.github.com/laol777/0ac74698317f890b9a087e354d358239 4. then this https://gist.github.com/laol777/a8c6034c3b24e2b3f92cfca2c691312f — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#51 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH-KSIPNsvNvO-rlAvnxt4tWq1m-YX46ks5vY6kAgaJpZM4acNt_> .

[laol777]

Sorry, i created the wrong snippet, exactly by this reason i deleted it. I will be able to provide correct one in next couple days, please push me, if i will forget about this

[sangenan]

@laol777 Looking forward to your results, you will save my ass

[laol777]

@sangenan This is the main idea, it's quite hard to wrap this code in a good way, so I will put it as is. if you will have any question, feel free to ask.

output2_1 = sess.graph.get_tensor_by_name('netG/netG_var_scope/netG_var_scopeA/netG_2_1/BiasAdd_3:0')
result2_1 = sess.run([output2_1], feed_dict={t: d for t, d in zip(dict_t, dict_d)})
result2_1 = result2_1[0]


a = img[0]
b = np.vstack([a, a, a, a])
c = np.hstack([b, b, b, b])
c = [c]

dict_t_new = (input, output2_1)
dict_d_new = (c, result2_1)


final = sess.graph.get_tensor_by_name('netG/netG_var_scope/netG_var_scopeA/netG_3_1/Add:0')

result1_1 = sess.run([final], feed_dict={t: d for t, d in zip(dict_t_new, dict_d_new)})
result1_1 = result1_1[0]

ttt = result1_1[0]
cv2.imwrite('ttsdf.png', ttt * tf.as_dtype(FLAGS['data_input_dtype']).max)

the sense of this code is computation global features for image with 512x512 shapes and local features for arbitrary size image. dict_t and dict_d you will find in original code. c is manualy generated bigger image.

[sangenan]

@laol777 I'll try it today. Thank you again

[liulizhou]

Hi, @laol777 @sangenan
Thank you for your code, but i am confused by a = img[0] and input, can you explain? In addition, why do vstack on a and do hstack on b?

[sangenan]

@laol777 Thanks again , I have some doubts. How should I use this code? could you provide detailed steps and code? Consider your previous answers, replace FLAGS['data_image_size'], 512 to None, in DATA.py file，and then retrain the model. If so, i need change the model structure

[laol777]

I hope this will be more clear https://gist.github.com/laol777/12ea093104db21f5d6c7be6587895d2b
In this snippet, you will load pd file, which you generated in the previous step (see 3. snippet above)

[laol777]

@liulizhou hstack and vstack is an little trick which will increase original dimensions 4x times

[laol777]

@sangenan and i don't think that without contribution in this project i will be able to describe the whole process, I only tried to give you a gist of this process:

1. retrain model for arbitrary placeholder size (but anyway with this arbitrary placeholder model won't be able to make prediction for arbitrary image)
2. froze model to pb file
3. run prediction in two passes arbitrary resolutions input #18

[liulizhou]

@laol777 @sangenan Really thanks a lot for your sharing, but i'm still confused by following questions :

1. When i change FLAGS['data_image_size'] to None, and retrain model with raw 512 dataset, you said anyway with this arbitrary placeholder model won't be able to make prediction for arbitrary image that means the retrained model still can't predict arbitray image?
2. Retrain model means I still train model using raw 512 dataset?

[laol777]

1. yes, this model consists two step for producing final result - extraction local and global features. Architecture of this model doesn't allow us to predict global features for an arbitrary image (tensor - netG/netG_var_scope/netG_var_scopeA/netG_2_1/BiasAdd_3:0, only for 512x512 ), but a retrained version can predict local features for arbitrary image (tensor netG/netG_var_scope/netG_var_scopeA/netG_3_1/Add:0)
2. yes

[liulizhou]

Hi, @laol777 @sangenan i'm retraining model replacing FLAGS['data_image_size'] with None, have you predicted right enhanced result with the retrain model on the bigger resolution than 512, e.g. 1920x1080? In addition, the image enhancer model may bring some artifacts, like block、noise #41 ，do you have the same problem or solutions about that?

[sangenan]

@liulizhou hi, I'm trying, too. If I make progress, I'll let you know as soon as possible.

[liulizhou]

@liulizhou hi, I'm trying, too. If I make progress, I'll let you know as soon as possible.

@sangenan Thanks, wait for your result, i'm trying too. @laol777 have you get right enhanced result using your method for arbitrary input?

[laol777]

With this detect #41 but it's a common problem for this model

[liulizhou]

Hi, @sangenan @laol777 thanks again. According to @laol777 sharing code, i get the arbitrary input enhanced image.

1. To extract global features, i remove vstack and hstack, and just resize hight resolution image to 512. Remember pad input image width or height which must be divided by 16. For example, if your video image resolution is 1920x1080, you should pad it to 1920x1120 or bigger.
2. Input raw hight resolution to get the local features and output raw resolution enhanced image.

[sangenan]

@laol777 @liulizhou hi,Guys, I succeeded too. Thank you for your help.

[xiaozhi2015]

Much appreciate for your contributions!
https://gist.github.com/laol777/12ea093104db21f5d6c7be6587895d2b This link has failed. Can you pass it on again? @liulizhou
I have some confusions：

1. when retrain, just replacing FLAGS['data_image_size'] with None ,an error occurs in "tf.reshape(, , FLAGS['data_image_size'], ]])", any other parts need to be modified?
2. use "increase original dimensions 4x times" in train or in infer process?

[liulizhou]

Much appreciate for your contributi!
https://gist.github.com/laol777/12ea093104db21f5d6c7be6587895d2b This link has failed. Can you pass it on again?@liulizhou
I have some confusions：

1. when retrain, just replacing FLAGS['data_image_size'] with None ,an error occurs in "tf.reshape(, , FLAGS['data_image_size'], ]])", any other parts need to be modified?
2. use "increase original dimensions 4x times" in train or in infer process?

Hi, @xiaozhi2015
@laol777 has shared the code, you should try it carefully.

1. just replace FLAGS['data_image_size'] to None, check it again.
2. "increase original dimensions 4x times" just a debug trick which resolution is bigger than 512. Actually, it can be arbitrary input which just need to be divided by 16 (unet four times downsample)

[xiaozhi2015]

I've tried with @laol777's shared code. The model trained without replacing FLAGS['data_image_size'] to None also works. Hope for further communication. @liulizhou

I mean you only need to replace FLAGS['data_image_size'] to None in infer process

[nothinglo]

Thank all guys. So sorry for the delay. I just uploaded my inference models and the code which I used in my demo website. Thanks.

[wulei2018]

I've tried with @laol777's shared code. The model trained without replacing FLAGS['data_image_size'] to None also works. Hope for further communication. @liulizhou

I mean you only need to replace FLAGS['data_image_size'] to None in infer process

can you share your code to make this model to support size>512?

[wulei2018]

@laol777 @liulizhou hi,Guys, I succeeded too. Thank you for your help.

can you share your code to make this model to support size>512?