integrating (CTCWordBeamSearch)PureNumpy with (SimpleHTR --wordbeamsearch)

[jjsr]

Dear Sir,
Thank you for making it compatible along with windows , Your kind suggestion is required to implement it in SimpleHTR code --wordbeamsearch option--

1. I followed the steps of read me at successful upto getting the output same as testPybind.py(Since windows does not support custom tf operation as per read me i followed purenumpy instruction)

Code snippet of SimpleHTR-->Model.py
##################################
word_beam_search_module = tf.load_op_library('TFWordBeamSearch.so')
chars = str().join(self.charList)
wordChars = open('../model/wordCharList.txt').read().splitlines()[0]
corpus = open('../data/corpus.txt').read()

decode using the "Words" mode of word beam search

self.decoder =word_beam_search_module.word_beam_search(tf.nn.softmax(self.ctcIn3dTBC, dim=2), 50, 'Words', 0.0, corpus.encode('utf8'), chars.encode('utf8'), wordChars.encode('utf8'))

#################################
What should I replace it with to integrate CTCWordBeamSearch with SimpleHTR

Note-
**Initially i have tried to replace it with WordBeamSearch() , got confused with what to feed
**at my second try I try to convert tf.nn.softmax(self.ctcIn3dTBC, dim=2) into np.array(tf.nn.softmax(self.ctcIn3dTBC, dim=2)) results an error
** called testPybind() with above parameters results in error ..,,
Please may guide to which direction go next

[githubharald]

you have to evaluate the output of the RNN with softmax applied, and then feed this into the decoder. Something like that (not tested):

• After this line, define a new variable self.rnn_out_sm = tf.nn.softmax(self.ctcIn3dTBC, axis=2)
• In the infer function, evaluate the variable defined before, rnn_out_sm_numpy = self.sess.run(self.rnn_out_sm, feedDict)
• Create the word beam search object only once! So e.g. in the __init__ function of the Model
• Then, feed the evaluated numpy array (rnn_out_sm_numpy) to the compute function

[jjsr]

Step-1 Initialize above in init of Model.py
chars = str().join(self.charList) wordChars = open('../model/wordCharList.txt').read().splitlines()[0]
corpus = open('../data/corpus.txt').read() self.wbs=WordBeamSearch(25,'Words',0.0,corpus.encode('utf8'), chars.encode('utf8'), wordChars.encode('utf8'))

Step-2 in Model.py--> setupCTC() defined Point-1 Above
self.rnn_out_sm = tf.nn.softmax(self.ctcIn3dTBC, axis=2)

• Commented everything from L128-L138
  Step-3-
  In inferBtach()
• Commented Line-250 and Line-251 , Line-254, Line-255
• Added self.rnn_out_sm_numpy = self.sess.run(self.rnn_out_sm, feedDict) evalRes=self.wbs.compute(self.rnn_out_sm_numpy) after Line-252
  Result- On running my program in debugging mode my ctcOutput array of Line 186 is [64,61,72,72,64,57]
  Observation-1 I did not get any label index as blank Line196 in my ctcOutput
  Observation-2 on manually finding out the index of ctcOut array in charList i have "mjuumf" instead of "little"

Sidenote- I am running code as(python main.py --wordbeamsearch) (using presaved model ,testing on a single image)
Kindly let me know what i am doing wrong if possible, i know i am messing things up ,a word of help still highly appreciated
I am getting the error on simply running the code after making changes in model,py -- int' object is not iterable -- on Line 196

Modified model.py I have uploaded on my git just for time being after that i will remove , I have also given your reference while uploading model.py

[jjsr]

Thanks again for your earlier response sir

[githubharald]

Hi,

at first sight code looks OK. I guess this line is wrong and can be removed (you want to pass all batch elements to the function decoderOutputToText and not just the one with index 0):
https://github.com/jjsr/SimpleHtr/blob/main/Model.py#L273

If this does not fix the problem: can you upload all of your code into your repository? So that I just have to download it and run it to reprocude the error.

[jjsr]

Sir ,After your input I am able to somehow run it on "Word" , I am not quite confident in what patches i have made during the process is right or Its really worked , Beside it Sir may you please suggest into line segmentation(Breaking a page into lines ) approach ? (The approach you think would work best on IAM(Not quite complex scenario) with preferable code implementation link). Additionally I will upload the codes of the patches i try to work out , Kindly go through it sir once uploaded . Thanks again for your timely valuable feedbacks

[githubharald]

you can try WordDetectorNN to detect handwritten words on a scanned page.

[githubharald]

FYI: SimpleHTR is now using the Python package (numpy operation) instead of the TF operation.

[jjsr]

Thank you so much sir I was in the dire need of same ... Thanks again .. Regards, Vaibhav

…

On Fri, Feb 19, 2021, 01:59 Harald Scheidl ***@***.***> wrote: FYI: SimpleHTR is now using the Python package (numpy operation) instead of the TF operation. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACXE74G3P5LLSOWG4M76AKDS7V2D5ANCNFSM4XBA62RA> .