CTranslate2 layer_norm_gpu.cu:32: cuDNN failed with status CUDNN_STATUS_BAD_PARAM

[5118Python]

if sp=='in' or sp=='out' or sp=='inout':
    s = spm.SentencePieceProcessor()
    s.Load(modelPath + 'all.en.shuffled.filtered.spiece.model')
@app.route('/translate', methods=['Post'])
def trans():
    try:
        line = request.values.get('src')
        
        if sp=='in' or sp=='inout':
            sentence = s.EncodeAsPieces(line)
        else:
            sentence = list(line)

        results = translator.translate_batch([sentence], beam_size=1, max_decoding_length=250, num_hypotheses=1, length_penalty=0, min_decoding_length=1, use_vmap=False, return_attention=False)

        itemResult = ''

        for itemStr in results:
            item = itemStr[0]['tokens']

            if sp=='out' or sp=='inout':
                itemResult = s.DecodePieces(item)
            else:
                itemResult = str(''.join(item))

            # print(result)

        resultHtml = json.dumps([{"tgt": itemResult}], ensure_ascii=False)
    except Exception as e:
        resultHtml = json.dumps(({"error": 1, "message": str(e)}), ensure_ascii=False)

    return resultHtml, 200

server = WSGIServer((args.ip, args.port), app)
print('Server ready!')

server.serve_forever()

When I make a lot of requests, it's a mistake

terminate called after throwing an instance of 'std::runtime_error'
what(): /root/ctranslate2-dev/src/ops/layer_norm_gpu.cu:32: cuDNN failed with status CUDNN_STATUS_BAD_PARAM
Aborted (core dumped)

https://github.com/OpenNMT/CTranslate2/blob/master/src/ops/layer_norm_gpu.cu

[guillaumekln]

It seems you are using on old version as this file is no longer using cuDNN. Can you try the latest version and see if the issue is still present?

[guillaumekln]

Closing. Feel free to reopen if you can reproduce with a more recent version (e.g. 1.2.3).