Training data with LR and HR

[zzhan127]

Hello,

I am trying to train the model with my own dataset. Instead of using bicubic, I generated some LR images using my own method. I want to train the model but I cannot find where to put the training LR data.

Thank you!

[RachelCmy]

Hi,

in our code, we only load the HR files and the LR files are generated on the fly.
To load both the LR and HR, you can change the function loadHR() in dataloader.py

Best regards.

[zzhan127]

Thanks.

[zzhan127]

Do you have any suggestions on how to load the LR data? I tried to use loadHR() with different tar_size to load my LR images inside frvsr_gpu_data_loader() and replaced the tf_data_gaussDownby4(curHR, 1.5) with curLR, curLR = LR_images[frame_t] , but the results is weird.

LR_batch_list, LR_num_image_list_HR_t_cur = loadHRfunc(LRFLAGS, 32)
LR_images = list(LR_batch_list[FLAGS.RNN_N::])
def getLR():
    return LR_images[frame_t]
curLR = getLR()
input_images.append( curLR )

Thank you

[RachelCmy]

LR should be loaded at the same time with the HR and they should be preprocessed in the same way consistently. Otherwise, you will get randomly shuffled LR and HR that mismatches.

Hope this file is helpful (not tested!):
loadLRHR.py.txt
line 22 should be changed according to LR paths.
line 34 & 96 makes sure that LR and HR are consistent.

Since LR is pre-generated, it is important that the relationship between LR and HR is invariant to data augmentation methods such as flipping and cropping.
For example, if LR_1 = tf.image.resize_images(HR_1, LRsize), LR_1 and HR_1 are sensitive to flipping,
because tf.image.flip_left_right( LR_1 ) != tf.image.resize_images( tf.image.flip_left_right( HR_1 ), LRsize).

[zzhan127]

I really appreciated that. It works for me with some modification.

Have a nice day,
Zeyu

[caijie0620]

@zzhan127 Hello! I am using this code. Can you tell me where did you make the change? Thanks.

[zzhan127]

Sorry, my code after modification is not available to me now. But I believe some minor modification is needed after you set the correct path. Jie Cai <notifications@github.com> 于 2019年11月7日周四 下午1:50写道：

…

@zzhan127 <https://github.com/zzhan127> Hello! I am using this code. Can you tell me where did you make the change? Thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17?email_source=notifications&email_token=AGF5GYLU35INRW5NQ6CWUOLQSRPNZA5CNFSM4H4IGFQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDNNPFQ#issuecomment-551212950>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGF5GYLKMODW3ZV3ZCVLA4DQSRPNZANCNFSM4H4IGFQQ> .

[alessiapacca]

@zzhan127 @RachelCmy so basically in the code we don't have to call
(main.py, row 280) rdata = frvsr_gpu_data_loader(FLAGS, useValidat) anymore, but
rdata = frvsr_data_loader(FLAGS, useValidat)
?

[alessiapacca]

Hey @zzhan127
can I ask you if it worked for you? I am using that script, and it doesn't give me any error, but once I train with my own images I obtain a super strange result as output. Did you have the chance to retrain it and then test it to see if it worked?

@RachelCmy could you please answer me? I have some questions about this that I would like to discuss with the author of the code

[zzhan127]

Hi, I don't have any problem using it. This model takes several days for training, the results looks reasonable to me after 3 days of training.

…

On Sat, Oct 24, 2020 at 5:20 AM Alessia Paccagnella < ***@***.***> wrote: Hey @zzhan127 <https://github.com/zzhan127> can I ask you if it worked for you? I am using that script, and it doesn't give me any error, but once I train with my own images I obtain a super strange result as output. Did you have the chance to retrain it and then test it to see if it worked? @RachelCmy <https://github.com/RachelCmy> could you please answer me? I have some questions about this that I would like to discuss with the author of the code — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGF5GYJCZ7TMNSI3FN5SPXLSMKL33ANCNFSM4H4IGFQQ> .

[alessiapacca]

Hi, I don't have any problem using it. This model takes several days for training, the results looks reasonable to me after 3 days of training.
…
On Sat, Oct 24, 2020 at 5:20 AM Alessia Paccagnella < @.***> wrote: Hey @zzhan127 https://github.com/zzhan127 can I ask you if it worked for you? I am using that script, and it doesn't give me any error, but once I train with my own images I obtain a super strange result as output. Did you have the chance to retrain it and then test it to see if it worked? @RachelCmy https://github.com/RachelCmy could you please answer me? I have some questions about this that I would like to discuss with the author of the code — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGF5GYJCZ7TMNSI3FN5SPXLSMKL33ANCNFSM4H4IGFQQ .

@zzhan127 Thank you very much for your reply. In your case, how did you obtained your own pairs of HR and LR images?

[zzhan127]

I used some medical imaging data for training. I also changed the images name to col_high_0000 for HR images and col_low_0000 for LR images. On Sun, Oct 25, 2020 at 4:50 PM Alessia Paccagnella < notifications@github.com> wrote:

…

Hi, I don't have any problem using it. This model takes several days for training, the results looks reasonable to me after 3 days of training. … <#m_-3854804937577913515_> On Sat, Oct 24, 2020 at 5:20 AM Alessia Paccagnella < *@*.***> wrote: Hey @zzhan127 <https://github.com/zzhan127> https://github.com/zzhan127 can I ask you if it worked for you? I am using that script, and it doesn't give me any error, but once I train with my own images I obtain a super strange result as output. Did you have the chance to retrain it and then test it to see if it worked? @RachelCmy <https://github.com/RachelCmy> https://github.com/RachelCmy could you please answer me? I have some questions about this that I would like to discuss with the author of the code — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17 (comment) <#17 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGF5GYJCZ7TMNSI3FN5SPXLSMKL33ANCNFSM4H4IGFQQ . @zzhan127 <https://github.com/zzhan127> Thank you very much for your reply. In your case, how did you obtained your own pairs of HR and LR images? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGF5GYOXMF32ZQJBIAXH3W3SMSFSPANCNFSM4H4IGFQQ> .

[alessiapacca]

@zzhan127 thank you very much for your answer, again!
alright, I will try.
Just one last question, that I do not understand. What should tar_size be? In the original code, they wrote

tar_size = (FLAGS.crop_size * 4 ) + int(1.5 * 3.0) * 2 # crop_size * 4, and Gaussian blur margin
k_w_border = int(1.5 * 3.0)

while in the new script, tar_size is only FLAGS.crop_size * 4

do you have any idea of the reason?

[zzhan127]

I am not sure. But Gaussian blur is used to generate LR images in the original code. And in the new code, since we have the LR images, we don’t need to use Gaussian blur anymore.

…

On Mon, Oct 26, 2020 at 5:17 AM Alessia Paccagnella < ***@***.***> wrote: @zzhan127 <https://github.com/zzhan127> thank you very much for your answer, again! alright, I will try. Just one last question, that I do not understand. What should tar_size be? In the original code, they wrote tar_size = (FLAGS.crop_size * 4 ) + int(1.5 * 3.0) * 2 # crop_size * 4, and Gaussian blur margin k_w_border = int(1.5 * 3.0) while in the new script, tar_size is only FLAGS.crop_size * 4 do you have any idea of the reason? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGF5GYKPLWJFK5DF777P4NLSMU5BJANCNFSM4H4IGFQQ> .

[alessiapacca]

@zzhan127 thank you very much for your answers, you were helpful!

May I ask one last thing?
When you trained, if you remember, did you start training from the pretrained model FRVSR or from scratch?