express_ner中，在数据预测部分出现SystemError: (Fatal) Blocking queue is killed because the data reader raises an exception

[xiaomeng654321]

单独做预测代码，报错SystemError: (Fatal) Blocking queue is killed because the data reader raises an exception.
更改的代码，在数据加载部分，更改 ExpressDataset类，变为无标签预测
变更的代码：
class ExpressDatasettest(paddle.io.Dataset):
def init(self, data_path):
self.word_vocab = load_dict('./conf/word.dic')
self.label_vocab = load_dict('./conf/tag.dic')
self.word_ids = []
self.label_ids = []
with open(data_path, 'r', encoding='utf-8') as fp:
next(fp)
for line in fp.readlines():
# words, labels = line.strip('\n').split('\t')
# words = words.split('\002')
words = line.strip("\n")
words = list(words)
sub_word_ids = convert_tokens_to_ids(words, self.word_vocab,
'OOV')
self.word_ids.append(sub_word_ids)

    self.word_num = max(self.word_vocab.values()) + 1
    self.label_num = max(self.label_vocab.values()) + 1

def __len__(self):
    return len(self.word_ids)

def __getitem__(self, index):
    return self.word_ids[index], len(self.word_ids[index])
    # return self.word_ids[index], len(self.word_ids[index]), self.label_ids[
    #     index]

在预测部分出现的具体错误：

WARNING:root:DataLoader reader thread raised an exception.
Traceback (most recent call last):
File "predict.py", line 183, in
outputs, lens, decodes = model.predict(data_loader)
File "/home/yanwei/anaconda3/envs/paddlenlp36/lib/python3.6/site-packages/paddle/hapi/model.py", line 1703, in predict
logs, outputs = self._run_one_epoch(test_loader, cbks, 'predict')
File "/home/yanwei/anaconda3/envs/paddlenlp36/lib/python3.6/site-packages/paddle/hapi/model.py", line 1779, in run_one_epoch
for step, data in enumerate(data_loader):
File "/home/yanwei/anaconda3/envs/paddlenlp36/lib/python3.6/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 365, in next
return self.reader.read_next_var_list()
SystemError: (Fatal) Blocking queue is killed because the data reader raises an exception.
[Hint: Expected killed != true, but received killed:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:158)

这种情况是什么原因呀，麻烦帮忙解答一下，谢谢

[wawltor]

稍等 我们排查一下

[wawltor]

麻烦对应着把batchify_fn函数改一下，之前是对三个field进行batch化组合，现在只有2个，因此注释一下。
经过实验，您最好load一份已经训练好的参数，这样不会进行出错。

[xiaomeng654321]

好的，非常感谢，但在预测的时候，总是出现keyerror的报错， 如下是主函数部分代码：
paddle.set_device('gpu')

data = "./data/userinput.txt"

modelparams = "./modelspath/final.pdparams"

train_ds_word_num = 20941
train_ds_label_num = 13

label_keys = ['P-B', 'P-I', 'T-B', 'T-I', 'A1-B', 'A1-I', 'A2-B', 'A2-I', 'A3-B', 'A3-I', 'A4-B', 'A4-I', 'O']

data_ds = ExpressDatasettest(data)

batchify_fn = lambda samples, fn=Tuple(
    Pad(axis=0, pad_val=20941),
    Stack()
    # Pad(axis=0, pad_val=13)
): fn(samples)

data_loader = paddle.io.DataLoader(
    dataset=data_ds,
    batch_size=100,
    drop_last=False,
    return_list=True,
    collate_fn=batchify_fn)


weights = paddle.load(modelparams)@

network = BiGRUWithCRF(300, 300, train_ds_word_num, train_ds_label_num)

network.set_state_dict(weights, use_structured_name=False)
model = paddle.Model(network)
model.prepare(optimizer=paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters()),
              loss=LinearChainCrfLoss(network.crf),
              metrics=ChunkEvaluator(label_list=label_keys, suffix=True)
              )

outputs, lens, decodes = model.predict(data_loader)
preds = parse_decodes(data_ds, decodes, lens)
# print("preds: ", preds)

print('\n'.join(preds[:10]))

出现的错误：
Predict begin...
step 5/5 [==============================] - 116ms/step
Predict samples: 424
Traceback (most recent call last):
File "predict.py", line 174, in
preds = parse_decodes(data_ds, decodes, lens)
File "predict.py", line 38, in parse_decodes
tags = [id_label[x] for x in decodes[idx][:end]]
File "predict.py", line 38, in
tags = [id_label[x] for x in decodes[idx][:end]]
KeyError: 13

在原来的训练脚本中，则没有这样的报错，但是预测自己的数据集效果不佳
可以帮忙看下这两个问题吗，谢谢

[wawltor]

好的，非常感谢，但在预测的时候，总是出现keyerror的报错， 如下是主函数部分代码：
paddle.set_device('gpu')

data = "./data/userinput.txt"

modelparams = "./modelspath/final.pdparams"

train_ds_word_num = 20941
train_ds_label_num = 13

label_keys = ['P-B', 'P-I', 'T-B', 'T-I', 'A1-B', 'A1-I', 'A2-B', 'A2-I', 'A3-B', 'A3-I', 'A4-B', 'A4-I', 'O']

data_ds = ExpressDatasettest(data)

batchify_fn = lambda samples, fn=Tuple(
    Pad(axis=0, pad_val=20941),
    Stack()
    # Pad(axis=0, pad_val=13)
): fn(samples)

data_loader = paddle.io.DataLoader(
    dataset=data_ds,
    batch_size=100,
    drop_last=False,
    return_list=True,
    collate_fn=batchify_fn)


weights = paddle.load(modelparams)@

network = BiGRUWithCRF(300, 300, train_ds_word_num, train_ds_label_num)

network.set_state_dict(weights, use_structured_name=False)
model = paddle.Model(network)
model.prepare(optimizer=paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters()),
              loss=LinearChainCrfLoss(network.crf),
              metrics=ChunkEvaluator(label_list=label_keys, suffix=True)
              )

outputs, lens, decodes = model.predict(data_loader)
preds = parse_decodes(data_ds, decodes, lens)
# print("preds: ", preds)

print('\n'.join(preds[:10]))

出现的错误：
Predict begin...
step 5/5 [==============================] - 116ms/step
Predict samples: 424
Traceback (most recent call last):
File "predict.py", line 174, in
preds = parse_decodes(data_ds, decodes, lens)
File "predict.py", line 38, in parse_decodes
tags = [id_label[x] for x in decodes[idx][:end]]
File "predict.py", line 38, in
tags = [id_label[x] for x in decodes[idx][:end]]
KeyError: 13

在原来的训练脚本中，则没有这样的报错，但是预测自己的数据集效果不佳
可以帮忙看下这两个问题吗，谢谢

嗯嗯 这个问题是需要加载一个训练好的参数即可，这样就不会出现上诉的问题；出现上诉的问题报错确实代码报错风格不好，我们会优化。

1. 先不改动示例代码（记得保存之前的已经改动的代码），先训练一个最好的模型出来，模型地址 results/final.pdparams
2. 加上如下图片所示代码
   
3. 开始预测

我们内部讨论一下，看看是否需要加一下预测代码，后续反馈给您

[xiaomeng654321]

好的，已成功，谢谢