Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] add XFUND dataset and project LayoutLMv3 #1809

Open
wants to merge 58 commits into
base: dev-1.x
Choose a base branch
from
Open

[Feature] add XFUND dataset and project LayoutLMv3 #1809

wants to merge 58 commits into from

Commits on Mar 23, 2023

  1. 阶段性提交

    @KevinNuNu
    KevinNuNu committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    24119ec View commit details
    Browse the repository at this point in the history

  2. Merge branch 'dev-1.x' into layoutlm

    @KevinNuNu
    KevinNuNu committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    38fbba5 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2023

  1. 重构xfund数据集config文件结构

    @KevinNuNu
    KevinNuNu committed Mar 25, 2023
    Configuration menu
    Copy the full SHA
    3ae3f84 View commit details
    Browse the repository at this point in the history

  2. 新增xfund zh数据集

    @KevinNuNu
    KevinNuNu committed Mar 25, 2023
    Configuration menu
    Copy the full SHA
    35d0dd9 View commit details
    Browse the repository at this point in the history

  3. [Fix] 解决jsondumper生成的文件无法正确显示中文的问题

    @KevinNuNu
    KevinNuNu committed Mar 25, 2023
    Configuration menu
    Copy the full SHA
    125cce2 View commit details
    Browse the repository at this point in the history

  4. [Fix] 解决路径拼接异常Bug

    @KevinNuNu
    KevinNuNu committed Mar 25, 2023
    Configuration menu
    Copy the full SHA
    f4f1dac View commit details
    Browse the repository at this point in the history

  5. Merge branch 'dev-1.x' into layoutlm

    @KevinNuNu
    KevinNuNu committed Mar 25, 2023
    Configuration menu
    Copy the full SHA
    ffe8909 View commit details
    Browse the repository at this point in the history

  6. 新增另外6个数据集config文件

    @KevinNuNu
    KevinNuNu committed Mar 25, 2023
    Configuration menu
    Copy the full SHA
    5b203ad View commit details
    Browse the repository at this point in the history

  7. 增加xfund RE任务

    @KevinNuNu
    KevinNuNu committed Mar 25, 2023
    Configuration menu
    Copy the full SHA
    2016921 View commit details
    Browse the repository at this point in the history

  8. pre-commit fix

    @KevinNuNu
    KevinNuNu committed Mar 25, 2023
    Configuration menu
    Copy the full SHA
    717ac03 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2023

  1. [Fix] 简化XFUND parser，优化最终数据集的目录结构

    @KevinNuNu
    KevinNuNu committed Mar 27, 2023
    Configuration menu
    Copy the full SHA
    078cc83 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2023

  1. [Fix] 回退删除huggingface dataset形式，没意义。修改ser/re packer的metainfo信息，阶段性添加S… 

    …ERDataset
    @KevinNuNu
    KevinNuNu committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    d3e16ad View commit details
    Browse the repository at this point in the history

  2. 阶段性完成SERDataset数据集加载

    @KevinNuNu
    KevinNuNu committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    1d0c5e3 View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2023

  1. 优化ser/re packer，根据words关键字是否存在觉得是否加入

    @KevinNuNu
    KevinNuNu committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    deb96cc View commit details
    Browse the repository at this point in the history

  2. 优化xfund数据集的config_generator命名，使config_generator目录结构更清晰

    @KevinNuNu
    KevinNuNu committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    443e979 View commit details
    Browse the repository at this point in the history

  3. 修改SERDataset为XFUNDSERDataset

    @KevinNuNu
    KevinNuNu committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    a88a129 View commit details
    Browse the repository at this point in the history

  4. ser/re packer docstring fix

    @KevinNuNu
    KevinNuNu committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    25f084a View commit details
    Browse the repository at this point in the history

  5. add SERDataSample structure and PackSERInputs transforms

    @KevinNuNu
    KevinNuNu committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    f8f2614 View commit details
    Browse the repository at this point in the history

  6. 初步构建SER部分model文件结构，LayoutLMv3DataPreprocessor参数已与HuggingFace的LayoutLM… 

    …v3ImageProcessor实现数值对齐
    @KevinNuNu
    KevinNuNu committed Mar 30, 2023
    Configuration menu
    Copy the full SHA
    c8a7b68 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2023

  1. Merge branch 'dev-1.x' into layoutlm

    @KevinNuNu
    KevinNuNu committed Apr 10, 2023
    Configuration menu
    Copy the full SHA
    81a4527 View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2023

  1. packer metainfo删除id2label信息

    @KevinNuNu
    KevinNuNu committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    e22e466 View commit details
    Browse the repository at this point in the history

  2. 优化xfund_dataset

    @KevinNuNu
    KevinNuNu committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    ceb66dc View commit details
    Browse the repository at this point in the history

  3. 明确添加的metainfo类型

    @KevinNuNu
    KevinNuNu committed Apr 11, 2023
    Configuration menu
    Copy the full SHA
    2eb79c3 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2023

  1. 简化版layoutlmv3代码

    @KevinNuNu
    KevinNuNu committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    a6bbe12 View commit details
    Browse the repository at this point in the history

  2. 优化layoutlmv3预处理代码，整合到datasets/transforms里，更明确

    @KevinNuNu
    KevinNuNu committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    7951200 View commit details
    Browse the repository at this point in the history

  3. 添加测试脚本

    @KevinNuNu
    KevinNuNu committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    3ddf780 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'dev-1.x' into layoutlm

    @KevinNuNu
    KevinNuNu committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    60b2a52 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2023

  1. 重构xfund数据集mmocr格式

    @KevinNuNu
    KevinNuNu committed Apr 18, 2023
    Configuration menu
    Copy the full SHA
    84be264 View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2023

  1. 简化XFUNDDataset，不再按ser/re任务区分

    @KevinNuNu
    KevinNuNu committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    2767fcc View commit details
    Browse the repository at this point in the history

  2. 将原本在XFUNDDataset内做的预处理全部移到pipeline中，重构预处理代码为LoadProcessorFromPretrain… 

    …edModel / ProcessImageForLayoutLMv3 / ProcessTokenForLayoutLMv3 / ConvertBIOLabelForSER 四个流程
    @KevinNuNu
    KevinNuNu committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    4b4b343 View commit details
    Browse the repository at this point in the history

  3. 更新项目测试脚本

    @KevinNuNu
    KevinNuNu committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    8399f94 View commit details
    Browse the repository at this point in the history

  4. 跑通train.py训练流程

    @KevinNuNu
    KevinNuNu committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    bda6742 View commit details
    Browse the repository at this point in the history

  5. 修改SERDataSample形式

    @KevinNuNu
    KevinNuNu committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    44c68b1 View commit details
    Browse the repository at this point in the history

  6. 修改SERPostprocessor一个命名错误

    @KevinNuNu
    KevinNuNu committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    023b0cf View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2023

  1. Merge branch 'dev-1.x' into layoutlm

    @KevinNuNu
    KevinNuNu committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    de98eb1 View commit details
    Browse the repository at this point in the history

  2. 整理config目录

    @KevinNuNu
    KevinNuNu committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    a05a2e1 View commit details
    Browse the repository at this point in the history

  3. 添加SER任务的评估模块

    @KevinNuNu
    KevinNuNu committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    3664773 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2023

  1. 优化PackSERInputs

    @KevinNuNu
    KevinNuNu committed Apr 29, 2023
    Configuration menu
    Copy the full SHA
    6c1f5be View commit details
    Browse the repository at this point in the history

Commits on May 1, 2023

  1. 将数据处理部分代码移动到project中

    @KevinNuNu
    KevinNuNu committed May 1, 2023
    Configuration menu
    Copy the full SHA
    d21a181 View commit details
    Browse the repository at this point in the history

  2. fix an error

    @KevinNuNu
    KevinNuNu committed May 1, 2023
    Configuration menu
    Copy the full SHA
    40cfe65 View commit details
    Browse the repository at this point in the history

  3. 将ser_data_sample移到projects里

    @KevinNuNu
    KevinNuNu committed May 1, 2023
    Configuration menu
    Copy the full SHA
    d1f43e7 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2023

  1. Merge branch 'dev-1.x' into layoutlm

    @KevinNuNu
    KevinNuNu committed May 8, 2023
    Configuration menu
    Copy the full SHA
    e102ef2 View commit details
    Browse the repository at this point in the history

  2. 规范xfund数据集准备脚本文件

    @KevinNuNu
    KevinNuNu committed May 8, 2023
    Configuration menu
    Copy the full SHA
    50fa7f9 View commit details
    Browse the repository at this point in the history

  3. [Fix]解决推理时存在的一个bug

    @KevinNuNu
    KevinNuNu committed May 8, 2023
    Configuration menu
    Copy the full SHA
    a04cd51 View commit details
    Browse the repository at this point in the history

  4. 使用custom_imports优化自定义模块的导入

    @KevinNuNu
    KevinNuNu committed May 8, 2023
    Configuration menu
    Copy the full SHA
    81b8f86 View commit details
    Browse the repository at this point in the history

  5. 优化SER任务结果可视化效果

    @KevinNuNu
    KevinNuNu committed May 8, 2023
    Configuration menu
    Copy the full SHA
    059e203 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2023

  1. 规范配置文件命名

    @KevinNuNu
    KevinNuNu committed May 25, 2023
    Configuration menu
    Copy the full SHA
    f0a03ac View commit details
    Browse the repository at this point in the history

  2. 化繁为简，优化之前基于default_collate的long_text_data_collate为更明确易理解的ser_collate

    @KevinNuNu
    KevinNuNu committed May 25, 2023
    Configuration menu
    Copy the full SHA
    d9a3a5e View commit details
    Browse the repository at this point in the history

  3. 针对inference阶段没有gt_label的情况针对性修复ser_postprocessor以及ser_visualizer中存在的bug.

    @KevinNuNu
    KevinNuNu committed May 25, 2023
    Configuration menu
    Copy the full SHA
    b04e126 View commit details
    Browse the repository at this point in the history

Commits on May 29, 2023

  1. 优化ser_postprocessor

    @KevinNuNu
    KevinNuNu committed May 29, 2023
    Configuration menu
    Copy the full SHA
    b6f55f8 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2023

  1. [Fix] 修复一个因为分词结果恰好510*n个，剔除收尾None标识后没有结束标志，导致最后一个label无法加入结果的Bug

    @KevinNuNu
    KevinNuNu committed Jun 12, 2023
    Configuration menu
    Copy the full SHA
    edf7fe8 View commit details
    Browse the repository at this point in the history

  2. [Fix] 重置word_biolabels防止重复添加

    @KevinNuNu
    KevinNuNu committed Jun 12, 2023
    Configuration menu
    Copy the full SHA
    8a1e37b View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2023

  1. Merge branch 'dev-1.x' into layoutlm

    @KevinNuNu
    KevinNuNu committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    dbe9145 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2023

  1. 删除项目中所有的绝对路径，补充README.md

    @KevinNuNu
    KevinNuNu committed Jun 27, 2023
    Configuration menu
    Copy the full SHA
    0f0f8ca View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2023

  1. fix lint

    @gaotongxiao
    gaotongxiao committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    ae8c426 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. Merge branch 'ci' into layoutlm

    @gaotongxiao
    gaotongxiao committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    ab14bb0 View commit details
    Browse the repository at this point in the history

  2. fix ci

    @gaotongxiao
    gaotongxiao committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    db5673f View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2023

  1. ci

    @gaotongxiao
    gaotongxiao committed Oct 28, 2023
    Configuration menu
    Copy the full SHA
    c7a3895 View commit details
    Browse the repository at this point in the history