Skip to content

Pull requests: modelscope/data-juicer

New pull request New
19 Open 420 Closed
19 Open 420 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

[NewOp] Add generate_challenging_qa_mapper based on MindGYM principles
#703 opened Jun 14, 2025 by Bat-Reality Loading…
[WIP] Optimization framework
#702 opened Jun 13, 2025 by cyruszhang Loading…
[NewOp] Add domain_diversity_selector based on DaaR principles
#699 opened Jun 12, 2025 by lingzhq Loading…
Add unittest for text embedding module & fix missing param desc
#698 opened Jun 12, 2025 by lingzhq Loading…
MinHash calculation with GPU on Ray dj:dist issues/PRs about distributed data processing dj:efficiency regarding to efficiency issues and enhancements dj:op issues/PRs about some specific OPs dj:tools issues/PRs about specific tools enhancement New feature or request
#694 opened Jun 9, 2025 by cyruszhang Loading…
6
[WIP] Add DJ-MCP server
#690 opened Jun 5, 2025 by cmgzn Loading…
6
support lance data format for ray mode
#687 opened Jun 4, 2025 by Cathy0908 Loading…
6
[WIP] fix calculate_np
#679 opened May 22, 2025 by Cathy0908 Loading…
4
Add RayBTSMinhashDeduplicatorWithUid and DocumentMinhashDeduplicatorWithUid.
#677 opened May 22, 2025 by chenyushuo Loading…
[Tools]Optimize data_resplit, upload a tool to convert parquet to jsonl
#676 opened May 20, 2025 by liuyuhanalex Loading…
1
Add correlation_analysis in the analysis module dj:core issues/PRs about the core functions of Data-Juicer enhancement New feature or request
#663 opened May 7, 2025 by HYLcool Loading…
@HYLcool
1
[WIP] deduping benchmark suite
#607 opened Mar 4, 2025 by cyruszhang Loading…
1
Optimize dedup to avoid oom dj:dist issues/PRs about distributed data processing dj:efficiency regarding to efficiency issues and enhancements dj:tools issues/PRs about specific tools enhancement New feature or request good first issue Good for newcomers
#568 opened Feb 7, 2025 by coolderli Loading…
Add humanvbench operators dj:multimodal issues/PRs about multimodal data processing dj:op issues/PRs about some specific OPs good first issue Good for newcomers
#553 opened Jan 17, 2025 by SYSUzhouting Loading…
@SYSUzhouting
17
Add minhash deduplicator based on RAY and Redis dj:dist issues/PRs about distributed data processing dj:efficiency regarding to efficiency issues and enhancements dj:op issues/PRs about some specific OPs
#489 opened Nov 15, 2024 by pan-x-c Loading…
5
支持RangeSpecifiedFieldSelector使用指定字段的值域进行数据选择
#432 opened Sep 21, 2024 by 2108038773 • Draft
1
Automatically split input dataset in ray mode
#415 opened Sep 4, 2024 by pan-x-c Loading…
1
3
[WIP]Add text tagging by prompt mapper op dj:op issues/PRs about some specific OPs
#408 opened Aug 30, 2024 by garyzhang99 Loading…
1 task
@garyzhang99
8
Add GPT-4V as evaluator dj:multimodal issues/PRs about multimodal data processing enhancement New feature or request stale-pr
#276 opened Mar 22, 2024 by drcege • Draft DJ-SORA
@drcege
13
ProTip! no:milestone will show everything without a milestone.