Unable to generate tf records

[Zumbalamambo]

Im getting the following error while trying to generate tf records

 File "/Users/anaconda/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "generate_tfrecord.py", line 90, in main
    tf_example = create_tf_example(group, path)
  File "generate_tfrecord.py", line 79, in create_tf_example
    'image/object/class/label': dataset_util.int64_list_feature(classes),
  File "/Users/Documents/models-master/object_detection/utils/dataset_util.py", line 26, in int64_list_feature
    return tf.train.Feature(int64_list=tf.train.Int64List(value=value))
TypeError: None has type NoneType, but expected one of: int, long

[Shivakishore14]

Hi, its easy to fix
https://github.com/datitran/raccoon_dataset/blob/master/generate_tfrecord.py#L30
this function returns None for anything other than 'raccoon', you might want to change that to your desired class and you can add other classes with elif if u have more than 1 class
That function should never return None. It should return a int value depending on the class.

[P-a-i-s]

@Shivakishore14 : that solution didin't work for me, unfortunately. I do have multiple classes and have used elif to describe them.

When checking across different solutions available on the net, most people (including datitran) pointed out that it might be a missing class or a misspell of a class in the train csv file. Am not able to figure that out since the labelling is done using labelImg, it saves these classes as xml, the xml_to_csv.py converts this to a csv. Am not sure under what circumstance I could have had the opportunity to miss out or misspel any class incorrectly.

Here's my error msg:
(OT) nishxxxx@xxx:~/Desktop/OD/models/research/object_detection$ python generate_tfrecord.py --csv_input=data/train_labels.csv --output_path=data/train.record
Traceback (most recent call last):
File "generate_tfrecord.py", line 192, in
tf.app.run()
File "/home/nishxxxx/Desktop/test_OD/OT/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "generate_tfrecord.py", line 184, in main
tf_example = create_tf_example(group, path)
File "generate_tfrecord.py", line 173, in create_tf_example
'image/object/class/label': dataset_util.int64_list_feature(classes),
File "/home/nishxxxx/Desktop/test_OD/models/research/object_detection/utils/dataset_util.py", line 26, in int64_list_feature
return tf.train.Feature(int64_list=tf.train.Int64List(value=value))
TypeError: None has type NoneType, but expected one of: int, long

Has anyone found a solution for this?

[JoltonADsouza]

@P-a-i-s Did you find the solution? I am still unable to do it.

[P-a-i-s]

@JoltonDsouza Hmm.. its been a while since I've completed this project. Trying to re-collect the reason for this error in my mind.

I am guessing you have multiple classes and hence are running into an error when defining the if-else loops. Is that so?

If yes, and I pretty much suspect that. try the following:

if row_label == 'pineapple':
return 1
elif row_label == 'papaya':
return 2
elif row_label == 'mango':
return 3
elif row_label == 'banana':
return 4
elif row_label == 'kiwi':
else:
return 0

Most forums talk about return equal None. It should be zero.

Hope this works for you.

[ruifgmonteiro]

You just need to change the else statement so it returns 0 instead of "None".

def class_text_to_int(row_label): if row_label == 'macncheese': return 1 else: return 0

[mdsyah79]

Hi Admin sorry if i seem trolling old thread.

Hey @P-a-i-s , @rfgm6 , your solution works!. Made me happy until i made this reply just to thank you guys.

Found out the error prompt came out due to it reached EOL hence if 'None' will cause the module to output an error. But despite the error the file test.record is already in the folder.

Anyway i prefer my code to run error free.

[Aishwarya2203]

@Shivakishore14 : that solution didin't work for me, unfortunately. I do have multiple classes and have used elif to describe them.

When checking across different solutions available on the net, most people (including datitran) pointed out that it might be a missing class or a misspell of a class in the train csv file. Am not able to figure that out since the labelling is done using labelImg, it saves these classes as xml, the xml_to_csv.py converts this to a csv. Am not sure under what circumstance I could have had the opportunity to miss out or misspel any class incorrectly.

Here's my error msg:
(OT) nishxxxx@xxx:~/Desktop/OD/models/research/object_detection$ python generate_tfrecord.py --csv_input=data/train_labels.csv --output_path=data/train.record
Traceback (most recent call last):
File "generate_tfrecord.py", line 192, in
tf.app.run()
File "/home/nishxxxx/Desktop/test_OD/OT/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "generate_tfrecord.py", line 184, in main
tf_example = create_tf_example(group, path)
File "generate_tfrecord.py", line 173, in create_tf_example
'image/object/class/label': dataset_util.int64_list_feature(classes),
File "/home/nishxxxx/Desktop/test_OD/models/research/object_detection/utils/dataset_util.py", line 26, in int64_list_feature
return tf.train.Feature(int64_list=tf.train.Int64List(value=value))
TypeError: None has type NoneType, but expected one of: int, long

Has anyone found a solution for this?

Hi, its easy to fix
https://github.com/datitran/raccoon_dataset/blob/master/generate_tfrecord.py#L30
this function returns None for anything other than 'raccoon', you might want to change that to your desired class and you can add other classes with elif if u have more than 1 class
That function should never return None. It should return a int value depending on the class.

I encountered the same issue, also i fixed the return to 0, the problem still persists

[Aishwarya2203]

@JoltonDsouza Hmm.. its been a while since I've completed this project. Trying to re-collect the reason for this error in my mind.

I am guessing you have multiple classes and hence are running into an error when defining the if-else loops. Is that so?

If yes, and I pretty much suspect that. try the following:

if row_label == 'pineapple':
return 1
elif row_label == 'papaya':
return 2
elif row_label == 'mango':
return 3
elif row_label == 'banana':
return 4
elif row_label == 'kiwi':
else:
return 0

Most forums talk about return equal None. It should be zero.

Hope this works for you.

C:\Users\admin\Desktop\object-detection>python generate_tfrecord.py
Traceback (most recent call last):
File "generate_tfrecord.py", line 99, in
tf.app.run()
File "C:\Users\admin\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\platform\app.py", line 125, in run
_sys.exit(main(argv))
File "generate_tfrecord.py", line 85, in main
writer = tf.python_io.TFRecordWriter(FLAGS.output_path)
File "C:\Users\admin\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\lib\io\tf_record.py", line 112, in init
compat.as_bytes(path), compat.as_bytes(compression_type), status)
File "C:\Users\admin\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 526, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a NewWriteableFile: : The system cannot find the path specified.
; No such process

This is the error i'm facing

[Yazmaz98]

Hi Admin sorry if i seem trolling old thread.

Hey @P-a-i-s , @rfgm6 , your solution works!. Made me happy until i made this reply just to thank you guys.

Found out the error prompt came out due to it reached EOL hence if 'None' will cause the module to output an error. But despite the error the file test.record is already in the folder.

Anyway i prefer my code to run error free.

I also found the .record files in my directory even though I got the "TypeError: None has type NoneType, but expected one of: int, long" error, but will those .record files work despite the error?

[AshishGusain17]

@JoltonDsouza Hmm.. its been a while since I've completed this project. Trying to re-collect the reason for this error in my mind.

I am guessing you have multiple classes and hence are running into an error when defining the if-else loops. Is that so?

If yes, and I pretty much suspect that. try the following:

if row_label == 'pineapple':
return 1
elif row_label == 'papaya':
return 2
elif row_label == 'mango':
return 3
elif row_label == 'banana':
return 4
elif row_label == 'kiwi':
else:
return 0

Most forums talk about return equal None. It should be zero.

Hope this works for you.

If we are giving 0 as return, should we add the index in labelmap.pbtxt file too??

[shubhamnagalwade]

Just change the label name whatever you labeling them during crop the image by labelImg tool.

def class_text_to_int(row_label):
    if row_label == 'raccon':
        return 1
    else:
        None

Instead of 'raccon' put label name for ex:- 'car'.