Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the root_dir and dir_fd parameters in glob.glob() #82325

Closed
serhiy-storchaka opened this issue Sep 12, 2019 · 6 comments
Closed

Add the root_dir and dir_fd parameters in glob.glob() #82325

serhiy-storchaka opened this issue Sep 12, 2019 · 6 comments
Labels
3.10 stdlib type-feature

Comments

@serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Sep 12, 2019

BPO 38144
Nosy @akuchling, @serhiy-storchaka
PRs
  • #16075
  • #22805
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2019-09-12.18:31:01.318>
    labels = ['type-feature', 'library', '3.10']
    title = 'Add the root_dir and dir_fd parameters in glob.glob()'
    updated_at = <Date 2020-10-20.16:45:45.999>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2020-10-20.16:45:45.999>
    actor = 'serhiy.storchaka'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2019-09-12.18:31:01.318>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 38144
    keywords = ['patch']
    message_count = 6.0
    messages = ['352223', '370545', '371832', '378993', '379089', '379143']
    nosy_count = 2.0
    nosy_names = ['akuchling', 'serhiy.storchaka']
    pr_nums = ['16075', '22805']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue38144'
    versions = ['Python 3.10']

    @serhiy-storchaka
    Copy link
    Member Author

    @serhiy-storchaka serhiy-storchaka commented Sep 12, 2019

    The pattern for the glob() function can be relative and absolute. If it is relative, paths are searched from the current directory. If you want to search them in other directory, you need either change the current directory (which affects other parts of your program) or pass a concatenation of the escaped root directory and a pattern:

    glob.glob(os.path.join(glob.escape(root_dir), pattern))

    Most code (even in the stdlib and tools) forget to escape the root directory. It works only until it does not contain metacharacters ('*?[').

    When you need paths relative to the root directory, you need to "remove" the root_dir prefix from results (using os.path.relpath() at best).

    The proposed PR adds two new parameters in glob.glob() and glob.iglob(): root_dir and dir_fd.

    root_dir specifies the root directory for relative pattern. Its effect is the same as chdir before calling glob(). It is similar to the root_dir parameter of shutil.make_archive. For example, you can add py-files in the specified directory to the ZIP archive by:

        with zipfile.ZipFile(archive, 'w') as zf:
            for filename in glob.glob('**/*.py', recursive=True, root_dir=root_dir):
                zf.write(os.path.join(root_dir, filename), arcname=filename)

    Adding to archive and copying are simpler if you have paths relative to the specified directory.

    The dir_fd parameter is similar to root_dir, but it specifies the root directory as an open directory descriptor instead of a path (as in many os functions). It adds security (nobody can rename and replace the root directory in process) and performance (because of shorter paths).

    root_dir and dir_fd can be combined. root_dir is relative to dir_fd and the pattern is relative to root_dir.

    If root_dir is absolute, dir_fd is ignored. If the pattern is absolute, root_dir and dir_fd are ignored.

    @serhiy-storchaka serhiy-storchaka added 3.9 stdlib type-feature labels Sep 12, 2019
    @serhiy-storchaka
    Copy link
    Member Author

    @serhiy-storchaka serhiy-storchaka commented Jun 1, 2020

    After adding this feature to iglob I am going to add it to other functions which work recursively with a directory tree.

    The only question: should we add two parameters root_dir and dir_fd or combine them in a single rood_dir (which can be either path or file descriptor or a special object combining a file descriptor and a relative path)? Or maybe merge one variant, test it and change it before feature freeze if needed?

    @serhiy-storchaka serhiy-storchaka added 3.10 and removed 3.9 labels Jun 1, 2020
    @serhiy-storchaka
    Copy link
    Member Author

    @serhiy-storchaka serhiy-storchaka commented Jun 18, 2020

    New changeset 8a64cea by Serhiy Storchaka in branch 'master':
    bpo-38144: Add the root_dir and dir_fd parameters in glob.glob(). (GH-16075)
    8a64cea

    @akuchling
    Copy link
    Member

    @akuchling akuchling commented Oct 19, 2020

    I'm digging around in the sys.audit() feature. Did the sys.audit("glob.glob", ...) call disappear in commit 8a64cea, or is something clever going on that I missed?

    @serhiy-storchaka
    Copy link
    Member Author

    @serhiy-storchaka serhiy-storchaka commented Oct 20, 2020

    Thank you for catching this. It was an error introduced by merge.

    @serhiy-storchaka
    Copy link
    Member Author

    @serhiy-storchaka serhiy-storchaka commented Oct 20, 2020

    New changeset 1d34699 by Serhiy Storchaka in branch 'master':
    bpo-38144: Re-add accidentally removed audition for glob. (GH-22805)
    1d34699

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.10 stdlib type-feature
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants