Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dataset refactor #807

Merged
merged 34 commits into from Mar 22, 2024
Merged

Dataset refactor #807

merged 34 commits into from Mar 22, 2024

Commits on Feb 22, 2024

  1. update

    @wangxingjun778
    wangxingjun778 committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    e34c638 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2024

  1. ok Merge branch 'master' of github.com:modelscope/modelscope into dev… 

    …/dataset_refactor
    @wangxingjun778
    wangxingjun778 committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    91f3481 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2024

  1. add main entry in ms_dataset

    @wangxingjun778
    wangxingjun778 committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    2f0d702 View commit details
    Browse the repository at this point in the history

  2. add main entry in ms_dataset

    @wangxingjun778
    wangxingjun778 committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    3c77cca View commit details
    Browse the repository at this point in the history

  3. update func get_data_patterns import

    @wangxingjun778
    wangxingjun778 committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    64e0aa4 View commit details
    Browse the repository at this point in the history

  4. modify return_config_only

    @wangxingjun778
    wangxingjun778 committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    c10fda5 View commit details
    Browse the repository at this point in the history

  5. update

    @wangxingjun778
    wangxingjun778 committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    f57a2c9 View commit details
    Browse the repository at this point in the history

  6. update

    @wangxingjun778
    wangxingjun778 committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    76475c5 View commit details
    Browse the repository at this point in the history

  7. update

    @wangxingjun778
    wangxingjun778 committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    f90d214 View commit details
    Browse the repository at this point in the history

  8. modify return_config_only to dataset_info_only

    @wangxingjun778
    wangxingjun778 committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    32e21da View commit details
    Browse the repository at this point in the history

  9. udpate version for test

    @wangxingjun778
    wangxingjun778 committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    732984f View commit details
    Browse the repository at this point in the history

  10. del get_logger(__name__)

    @wangxingjun778
    wangxingjun778 committed Mar 7, 2024
    Configuration menu
    Copy the full SHA
    b978553 View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2024

  1. fix py script loading

    @wangxingjun778
    wangxingjun778 committed Mar 10, 2024
    Configuration menu
    Copy the full SHA
    a130f0b View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2024

  1. fix loading py and without py

    @wangxingjun778
    wangxingjun778 committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    b99985b View commit details
    Browse the repository at this point in the history

  2. add subset support

    @wangxingjun778
    wangxingjun778 committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    d7038a3 View commit details
    Browse the repository at this point in the history

Commits on Mar 12, 2024

  1. add hf_datasets_util; refine list_repo_tree_ms; fix private datasets … 

    …loading issue
    @wangxingjun778
    wangxingjun778 committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    ecacc30 View commit details
    Browse the repository at this point in the history

  2. update version to rc5

    @wangxingjun778
    wangxingjun778 committed Mar 12, 2024
    Configuration menu
    Copy the full SHA
    948bed6 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. fix and support preview for dataset_info_only mode

    @wangxingjun778
    wangxingjun778 committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    7cb839b View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. fix urlencode

    @wangxingjun778
    wangxingjun778 committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    fc13737 View commit details
    Browse the repository at this point in the history

  2. update to rc7

    @wangxingjun778
    wangxingjun778 committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    53b6382 View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2024

  1. 1. loading of dataset_infos.json is deprecated; 2. add some ut

    @wangxingjun778
    wangxingjun778 committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    b5f75ca View commit details
    Browse the repository at this point in the history

  2. update version

    @wangxingjun778
    wangxingjun778 committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    1f8cda0 View commit details
    Browse the repository at this point in the history

  3. add escapechar for read_csv and to_csv

    @wangxingjun778
    wangxingjun778 committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    014770a View commit details
    Browse the repository at this point in the history

  4. add params: Source=SDK

    @wangxingjun778
    wangxingjun778 committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    c1f7dad View commit details
    Browse the repository at this point in the history

  5. add create_dataset func

    @wangxingjun778
    wangxingjun778 committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    5b97201 View commit details
    Browse the repository at this point in the history

  6. overwrite _get_paths_info

    @wangxingjun778
    wangxingjun778 committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    ecddc5a View commit details
    Browse the repository at this point in the history

  7. ok Merge branch 'master' of github.com:modelscope/modelscope into dev… 

    …/dataset_refactor
    @wangxingjun778
    wangxingjun778 committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    a4f4584 View commit details
    Browse the repository at this point in the history

  8. update & version

    @wangxingjun778
    wangxingjun778 committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    8830a07 View commit details
    Browse the repository at this point in the history

  9. update

    @wangxingjun778
    wangxingjun778 committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    fb3a025 View commit details
    Browse the repository at this point in the history

  10. update list_repo_tree name

    @wangxingjun778
    wangxingjun778 committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    02e48b1 View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2024

  1. add get_module_with_script, fix download imports

    @wangxingjun778
    wangxingjun778 committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    1e9dc7b View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. fix py script loading issue in dataset_module_factory

    @wangxingjun778
    wangxingjun778 committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    6bee42f View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. fix create dataset

    @wangxingjun778
    wangxingjun778 committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    4e0216d View commit details
    Browse the repository at this point in the history

  2. update log info in api

    @wangxingjun778
    wangxingjun778 committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    09e1f50 View commit details
    Browse the repository at this point in the history