Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

data_size is NAN #18

Closed
ghost opened this issue Sep 29, 2017 · 23 comments
Closed

data_size is NAN #18

ghost opened this issue Sep 29, 2017 · 23 comments

Comments

@ghost
Copy link

ghost commented Sep 29, 2017 •
edited by ghost
Loading

I met some difficuties after use vl_compilenn().

>> Demo_Train_model_64_25_Res_Bnorm_Adam
     layer|      0|      1|      2|      3|      4|      5|      6|      7|      8|      9|     10|     11|     12|     13|     14|     15|     16|     17|
      type|  input|   conv|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|
      name|    n/a| layer1| layer2| layer3| layer4| layer5| layer6| layer7| layer8| layer9|layer10|layer11|layer12|layer13|layer14|layer15|layer16|layer17|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   support|    n/a|      3|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|
  filt dim|    n/a|      1|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|
filt dilat|    n/a|      1|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|
 num filts|    n/a|     64|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|
    stride|    n/a|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
       pad|    n/a|      1|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   rf size|    n/a|      3|      3|      5|      5|      5|      7|      7|      7|      9|      9|      9|     11|     11|     11|     13|     13|     13|
 rf offset|    n/a|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
 rf stride|    n/a|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
 data size|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|
data depth|    NaN|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|
  data num|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
  data mem|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|
 param mem|    n/a|    2KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|

     layer|     18|     19|     20|     21|     22|     23|     24|     25|     26|     27|     28|     29|     30|     31|     32|     33|     34|     35|
      type|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|
      name|layer18|layer19|layer20|layer21|layer22|layer23|layer24|layer25|layer26|layer27|layer28|layer29|layer30|layer31|layer32|layer33|layer34|layer35|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   support|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|
  filt dim|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|
filt dilat|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|
 num filts|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|
    stride|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
       pad|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   rf size|     15|     15|     15|     17|     17|     17|     19|     19|     19|     21|     21|     21|     23|     23|     23|     25|     25|     25|
 rf offset|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
 rf stride|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
 data size|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|
data depth|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|
  data num|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
  data mem|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|
 param mem|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|

     layer|     36|     37|     38|     39|     40|     41|     42|     43|     44|     45|     46|     47|     48|     49|
      type|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|  bnorm|   relu|   conv|   loss|
      name|layer36|layer37|layer38|layer39|layer40|layer41|layer42|layer43|layer44|layer45|layer46|layer47|layer48|layer49|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   support|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|      1|      3|      1|
  filt dim|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|
filt dilat|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|    n/a|      1|    n/a|
 num filts|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|     64|    n/a|    n/a|      1|    n/a|
    stride|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
       pad|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|      0|      1|      0|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
   rf size|     27|     27|     27|     29|     29|     29|     31|     31|     31|     33|     33|     33|     35|     35|
 rf offset|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
 rf stride|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|      1|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
 data size|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|NaNxNaN|
data depth|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|     64|      1|      1|
  data num|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|    128|      1|
----------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
  data mem|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|    NaN|
 param mem|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|  144KB|    1KB|     0B|    2KB|     0B|
@ghost
Copy link
Author

ghost commented Sep 29, 2017 •
edited by ghost
Loading

First step, I run GenerateTrainingPatches.m,Second step ,I run Demo_Train_model_64_25_Res_Bnorm_Adam.m to train.But I find data_size is NAN.what's wrong with it?
then I should run Demo_Test_model_64_25_Res_Bnorm_Adam.m?

@ghost ghost changed the title MatConvNet/matlab/mex/.build/vl_imreadjpeg.o：在函数‘Batch::Item::Item(Batch const&)’中：vl_imreadjpeg.cpp:(.text+0x683)：对‘__warn_memset_zero_len’未定义的引用 data_size is NAN Sep 29, 2017
@ghost
Copy link
Author

ghost commented Sep 29, 2017 •
edited by ghost
Loading

By the way,if I just want to validate your code,could I run Demo_test_DnCNN.m directly? But PSNR and SSIM both are NAN.so,could you figure the problem?

@ghost
Copy link
Author

ghost commented Sep 29, 2017

maybe I solved above problem,but I met the follow problem :

>> Demo_test_DnCNN
引用了不存在的字段 'dilate'。

出错 vl_simplenn (line 303)
        'dilate', l.dilate, ...

出错 Demo_test_DnCNN (line 64)
    res    = vl_simplenn(net,input,[],[],'conserveMemory',true,'mode','test');

maybe I solved above problem,but

@cszn
Copy link
Owner

cszn commented Sep 30, 2017

Train400

See #11

@chenjian123321
Copy link

hello , I run Demo_test_DnCNN.m , But PSNR and SSIM both are NAN.so,could you figure the problem?
and run DnCNN_train ,the result have error and is Input factor is insufficient DnCNN_train(line 75) net = vl_simplenn_tidy(net); can you help me

@ngcthuong
Copy link

I dont have this problem. It seems like you might have some problem with your training. How bout your training error? it should reduce around 1.0.

@chenjian123321
Copy link

what is difference for 1.0 and 1.1? l run the 1.0 also have error.

@chenjian123321
Copy link

l first run Demo_test_DnCNN3.m, the result is -----------------------------------------------
----BSD68------Gaussian Denoising-----

Average PSNR is NaNdB
Average SSIM is NaN

----Set5-----Super-Resolution-----

Average PSNR is NaNdB
Average SSIM is NaN

----Set14-----Super-Resolution-----

Average PSNR is NaNdB
Average SSIM is NaN

----BSD100-----Super-Resolution-----

Average PSNR is NaNdB
Average SSIM is NaN

----Urben100-----Super-Resolution-----

Average PSNR is NaNdB
Average SSIM is NaN

----classic5------Deblocking-----

Average PSNR is NaNdB
Average SSIM is NaN

----LIVE1------Deblocking-----

Average PSNR is NaNdB
Average SSIM is NaN

@ngcthuong
Copy link

ngcthuong commented Oct 11, 2017 •
edited
Loading

For the NAN problem in version 1.1, you should (1) add the path to 'train400' data, then (2) run 'data\GenerateTrainingPatches.m', so that it will generate the training data for you. The training data is storaged in 'data\TrainingPatches\imdb_40_128.mat'. And please check your training data size 'imdb_40_128.mat' whether you really have the data or not.

@chenjian123321
Copy link

thank you very much

@chenjian123321
Copy link

wait , what is (1), 1.1 add or 1.0 add?

@ngcthuong
Copy link

You should add it for TrainingCodes_v1.1 since it doesn't have 'Train400' images in their 'data' folder. The 'Train400' images path is located in 'TrainingCodes_v1.0' folder.

@chenjian123321
Copy link

OK, thank you

@moinul95
Copy link

Hi, I don't understand how the mat files in model are created. Can you please tell in what sequence the code should run?

@chenjian123321
Copy link

hello. first, you should run Demo_test_DnCNN3. if you use GPU, ni should remove "%net = vl_simplenn_tidy(net);" "%", and if have error, in Demo_test_DnCNN3 add your vl_setupnn path. then run GenerateData_model_64_25_Res_Bnorm_Adam.

@moinul95
Copy link

I am actually new to deep learning. I didn't get how the models are created as .mat files. We need to run some code and then save it as mat file. Where is this code to create model? Thank you.

@cszn
Copy link
Owner

cszn commented Oct 18, 2017

@moinul95

DnCNN/TrainingCodes/DnCNN_TrainingCodes_v1.1/DnCNN_train.m

Line 115 in 52faf88

save(modelPath(epoch), 'net')

@cszn cszn closed this as completed Nov 6, 2017
@allenkate12
Copy link

I am new to deep learning. how did u you resolve the data_size and data_meme NaN issue? @goonder .

@YananXM
Copy link

YananXM commented May 9, 2019

也许我解决了上面的问题，但我遇到了以下问题：

>> Demo_test_DnCNN
引用了不存在的字段 'dilate'。

出错 vl_simplenn (line 303)
        'dilate', l.dilate, ...

出错 Demo_test_DnCNN (line 64)
    res    = vl_simplenn(net,input,[],[],'conserveMemory',true,'mode','test');

也许我解决了上面的问题，但是

您好，我现在也遇到了同样的问题，请问您是怎么解决的。

@15230127713
Copy link

I am new to deep learning. how did you resolve the data_size and data_meme NaN issue? @goonder @allenkate12

@chenjian123321
Copy link

chenjian123321 commented May 14, 2019 via email

@191578010 191578010 mentioned this issue Dec 29, 2020
Open
@1804941502
Copy link

maybe I solved above problem,but I met the follow problem :

>> Demo_test_DnCNN
引用了不存在的字段 'dilate'。

出错 vl_simplenn (line 303)
        'dilate', l.dilate, ...

出错 Demo_test_DnCNN (line 64)
    res    = vl_simplenn(net,input,[],[],'conserveMemory',true,'mode','test');

maybe I solved above problem,but

How can you solve this problem,could you tell me? I have met the same problem,my datasiaze is also NAN.

@YananXM
Copy link

YananXM commented May 10, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@ngcthuong @cszn @moinul95 @chenjian123321 @allenkate12 @15230127713 @YananXM @1804941502