Error when i run kraken segment ocr

[indykish]

I ran the below command and receive an error as below.

This happens in both python3.6.8 and python3.7.2

 kraken -i bank.png bank.json segment  --remove_hlines --no-script-detect --scale 20 --pad 100 23 ocr --model en-default.pronn 
Loading RNN default	✓
Segmenting	✓
Traceback (most recent call last):
  File "/home/ram/code/lendsmart/py/kraken/venv/bin/kraken", line 10, in <module>
    sys.exit(cli())
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/click/core.py", line 1164, in invoke
    return _process_result(rv)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/click/core.py", line 1102, in _process_result
    **ctx.params)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/kraken/kraken.py", line 220, in process_pipeline
    task(base_image=base_image, input=input, output=output)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/kraken/kraken.py", line 157, in recognizer
    for pred in bar:
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/click/_termui_impl.py", line 285, in generator
    for rv in self.iter:
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/kraken/rpred.py", line 301, in rpred
    preds = network.predict(line)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/kraken/lib/models.py", line 81, in predict
    o = self.forward(line)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/kraken/lib/models.py", line 69, in forward
    o = self.nn.nn(line)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
    input = module(input)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/kraken/lib/layers.py", line 350, in forward
    o, _ = self.layer(inputs)
  File "/home/ram/code/lendsmart/py/kraken/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: 'hidden'

Here is my pip list in venv

Let me know if i missed something.

Is my level of pips correct ?

Should i use a different level for torch ?

[mittagessen]

Are you sure you're running kraken in an environment with pytorch 1.0? The error is caused when running the forward pass of the LSTM layer which required an explicit hidden state argument in the past but doesn't in the current pytorch version.

[indykish]

Here is my full asciicast ran in virtualenv, i used kraken 1.0.1 and torch points to 1.0.0
https://asciinema.org/a/FTrZCrCpF3xGuW9pXZQgOBGlW

Is there something i have missed ?

Is there a temporary patch i can use by sending an explicit hidden parameter ?

May be i will browse the history in git.

[indykish]

Has the lastest 1.0.1 kraken tested ?

Since i tested in other vms/containers 3.6/3.7 level of py. and It had the same error.

What python version was kraken tested ? 2.7 or 3.6 ?

[mittagessen]

It should work with both 3.6 and 3.7 although installation is a bit easier on 3.6 as coremltools hasn't been packaged for 3.7 yet.

[indykish]

Ok. thanks, do we need to run this on GPU only ?

Or CPU works ok ?

[mittagessen]

Either works, for recognition it is certainly not necessary as the throughput of the system is ~1line/second/core. Training is a bit of an annoyance (30min/1000 iterations) on CPU though, with GPU you can train a model from scratch (~20-30 epochs) in <10 minutes.

[indykish]

Thank you very much for the throughput info.

I am running it on cpu currently for text recognition only.

I am ok to test and wait for ~1line/second/core.

[indykish]

I have tried in ArchLinux (bleeding edge) and Ubuntu 18.04.

Was kraken tested in a specific flavor ?

[mittagessen]

It should work on all of them. I know of at least Arch, Debian, and Ubuntu installs that work without trouble. Can you try this English model? Its accuracy is better than the old one anyway.

[indykish]

wget http://homer.dh.uni-leipzig.de/en-default.mlmodel

Maybe you forgot to post the english model in the web server :)

It says 404 in the browser as well.

[mittagessen]

Arrgh, sorry the URL is http://homer.dh.uni-leipzig.de/en_best.mlmodel

[ryanfb]

I'm also seeing this in my Docker environment when trying to reuse any existing .pyrnn.pronn model. If I train a .mlmodel from scratch, it works fine.

[mittagessen]

It's been fixed (a few months old regression) in master in the sense that the forward pass works now but there are apparently slight numerical differences that result in ~5% accuracy drop with the old python models. I might get around to spend a few days debugging that, but the new neural network architecture performs significantly better both in absolute accuracy and generalization so it isn't particularly high on the list.

[indykish]

Hey @mittagessen Great to hear. To sum it up, the problem reported is a regression and was fixed.

We need to try and test with the new model to see it exists. If it still does exists its a low priority problem.:)

[ryanfb]

Thanks—indeed, if I update my Dockerized version of Kraken from c57abdb to f25acc1 (current master), I no longer see the error.

However, one thing that still concerns me about this is when using the existing .pyrnn.pronn models on a test image of Greek, I get completely awful output.

Test image:

Results: https://gist.github.com/ryanfb/4b2fcd8362ec91925221397a6bee2ca5

If I use the .mlmodel from this repo (or one trained from some in-progress training), I get reasonable results: https://gist.github.com/ryanfb/a4d2390d441fa6597b31325322e73764

Also, for the loeb_2013-10-07-12-32c-00071000.pyrnn.pronn model, I still get a different error:

Traceback (most recent call last):
  File "/usr/local/bin/kraken", line 10, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1164, in invoke
    return _process_result(rv)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1102, in _process_result
    **ctx.params)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/kraken/kraken.py", line 225, in process_pipeline
    task(base_image=base_image, input=input, output=output)
  File "/usr/local/lib/python3.6/dist-packages/kraken/kraken.py", line 162, in recognizer
    for pred in bar:
  File "/usr/local/lib/python3.6/dist-packages/click/_termui_impl.py", line 285, in generator
    for rv in self.iter:
  File "/usr/local/lib/python3.6/dist-packages/kraken/rpred.py", line 327, in rpred
    yield bidi_record(ocr_record(pred, pos, conf))
  File "/usr/local/lib/python3.6/dist-packages/kraken/rpred.py", line 102, in bidi_record
    bd.resolve_implicit_levels(storage, False)
  File "/usr/local/lib/python3.6/dist-packages/bidi/algorithm.py", line 466, in resolve_implicit_levels
    '%s not allowed here' % _ch['type']
AssertionError:  not allowed here

I hate to be a nuisance, but I'm just trying to make sure that the old models are still usable going forward, as they might be good for specific fonts/texts and it would be a shame to throw away the work that went into building them. I'd be happy to re-train/convert them and contribute them back as .mlmodel files if the ground truth used to make them is available somewhere and that's the path forward.

[mittagessen]

Retraining them with the new architecture would definitely be the way forward as the accuracy is significantly better. Those models were trained by Bruce Robertson but he hasn't made his training data public. You can ask him if he's willing to make it available for retraining and I can offer some GPU ressources to get it done in a few hours for all of them if that's a concern.

Getting to the root cause of the numerical differences will require comparing the output at each part of the LSTM cell in lib/layer.py (PeepholeBidiLSTM) and the old lib/lstm.py classes. That might take a few hours but I'm terribly swamped for at least the next few weeks and won't get to it.