Fix/import error and keep original sample #181

HYLcool · 2024-01-17T12:45:56Z

fix unexpected ImportError from generate_caption_mapper 官方文档，源代码编译后。无法使用 #179
add new arg keep_original_sample for 3 augmentation mappers to control whether to keep the original samples in the final processed datasets.

…dd availability_checking to avoid ImportError

…er to keep original samples in the final datasets.

data_juicer/ops/mapper/nlpaug_en_mapper.py

HYLcool added 4 commits January 17, 2024 17:04

* bug fixed: simhash-pybind is required by generate_caption_mapper. A…

8c1998e

…dd availability_checking to avoid ImportError

+ add new arg keep_original_sample for 3 aug mappers to control wheth…

c429dad

…er to keep original samples in the final datasets.

Merge branch 'main' into fix/import_error_and_keep_original_sample

d89a4e5

* minor fix

0c28a50

HYLcool linked an issue Jan 17, 2024 that may be closed by this pull request

官方文档，源代码编译后。无法使用 #179

Closed

3 tasks

HYLcool requested review from drcege, chenhesen, BeachWang, zhijianma and yxdyc January 17, 2024 12:46

HYLcool self-assigned this Jan 17, 2024

HYLcool added bug Something isn't working enhancement New feature or request labels Jan 17, 2024

zhijianma approved these changes Jan 17, 2024

View reviewed changes

data_juicer/ops/mapper/nlpaug_en_mapper.py Outdated Show resolved Hide resolved

* minor modification on docs

be6769a

zhijianma self-requested a review January 18, 2024 02:18

HYLcool merged commit 22bdb68 into main Jan 18, 2024
5 checks passed

HYLcool deleted the fix/import_error_and_keep_original_sample branch January 18, 2024 02:20

HYLcool mentioned this pull request Jan 18, 2024

官方文档，源代码编译后。无法使用 #179

Closed

3 tasks

Provide feedback