PPOCR-V5 识别模型微调，学习到新的字符后，正常文字识别准确率急剧下降

[mudou192]

希望通过微调识别部分emoji字符和一些特殊字体字符，用10张、200张、1000张等进行测试，用10张测试大概20轮，识别任何数据基本都是训练数据中的字符；200张10轮新的字符基本能识别到了，但其他字符识别准确率不足30%，需要补充大量的其他样本数据吗？
尝试很多训练批次和学习率，增大训练数据集，减少训练批次能够提高一定的效果，但想很好的学习到新的字符，必然会导致正常文字识别准确率下降过多。
下面是训练的配置文件，请问需要怎么调整？

Global:
  model_name: PP-OCRv5_server_rec # To use static model for inference.
  debug: false
  use_gpu: true
  epoch_num: 50
  log_smooth_window: 20
  print_batch_step: 100
  save_model_dir: ./model_output/PP-OCRv5_server_rec
  save_epoch_step: 10
  eval_batch_step: [0, 20000000]
  cal_metric_during_train: true
  calc_epoch_interval: 1
  pretrained_model:
  checkpoints:
  save_inference_dir:
  use_visualdl: false
  infer_img: doc/imgs_words/ch/word_1.jpg
  character_dict_path: ./ppocr/utils/dict/ppocrv5_dict.txt
  max_text_length: &max_text_length 25
  infer_mode: false
  use_space_char: true
  distributed: true
  save_res_path: ./model_output/rec/predicts_ppocrv5.txt
  d2s_train_image_shape: [3, 48, 320]


Optimizer:
  name: Adam
  beta1: 0.9
  beta2: 0.999
  lr:
    name: Cosine
    learning_rate: 0.0005
    warmup_epoch: 1
  regularizer:
    name: L2
    factor: 3.0e-05


Architecture:
  model_type: rec
  algorithm: SVTR_HGNet
  Transform:
  Backbone:
    name: PPHGNetV2_B4
    text_rec: True
  Head:
    name: MultiHead
    head_list:
      - CTCHead:
          Neck:
            name: svtr
            dims: 120
            depth: 2
            hidden_dims: 120
            kernel_size: [1, 3]
            use_guide: True
          Head:
            fc_decay: 0.00001
      - NRTRHead:
          nrtr_dim: 384
          max_text_length: *max_text_length

Loss:
  name: MultiLoss
  loss_config_list:
    - CTCLoss:
    - NRTRLoss:

PostProcess:
  name: CTCLabelDecode

Metric:
  name: RecMetric
  main_indicator: acc

Train:
  dataset:
    name: MultiScaleDataSet
    ds_width: false
    data_dir: ./datasets/images_crop_bak/
    ext_op_transform_idx: 1
    label_file_list:
    - ./datasets/images_crop_bak/Label_rec.txt
    transforms:
    - DecodeImage:
        img_mode: BGR
        channel_first: false
    - RecAug:
    - MultiLabelEncode:
        gtc_encode: NRTRLabelEncode
    - KeepKeys:
        keep_keys:
        - image
        - label_ctc
        - label_gtc
        - length
        - valid_ratio
  sampler:
    name: MultiScaleSampler
    scales: [[320, 32], [320, 48], [320, 64]]
    first_bs: &bs 8
    fix_bs: false
    divided_factor: [8, 16] # w, h
    is_training: True
  loader:
    shuffle: true
    batch_size_per_card: *bs
    drop_last: true
    num_workers: 0
Eval:
  dataset:
    name: SimpleDataSet
    data_dir: ./datasets/images_crop_bak/
    label_file_list:
    - ./datasets/images_crop_bak/Label_rec.txt
    transforms:
    - DecodeImage:
        img_mode: BGR
        channel_first: false
    - MultiLabelEncode:
        gtc_encode: NRTRLabelEncode
    - RecResizeImg:
        image_shape: [3, 48, 320]
    - KeepKeys:
        keep_keys:
        - image
        - label_ctc
        - label_gtc
        - length
        - valid_ratio
  loader:
    shuffle: false
    drop_last: false
    batch_size_per_card: 16
    num_workers: 0

另外训练数据能否支持这种垂直方向的文字图像

[ncihnegn]

Working on this as well. But my accuracy is always zero. Could you share your setup?

[julesarts]

publishing your training logs.😂

[julesarts]

yes,if you try to fine-tune the recognition model by customized dataset,it will forget other's accuracy of chars.😮‍💨

[chenxz11]

I've encountered the same issue. Are there any solutions available? Since I don't have the official English dataset, I am unable to perform data fusion operations. :(

[julesarts]

I've encountered the same issue. Are there any solutions available? Since I don't have the official English dataset, I am unable to perform data fusion operations. :(

---

You need to collect enough dataset for model fine-tune.

Another mean is try to adjust the arguments of paddleOCR.

[chenxz11]

How should I adjust the parameters? Any good suggestions? Lowering the learning rate works, but I'm unclear on other approaches. :)