Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve File Explorer performance #7337

Merged
merged 24 commits into from Feb 13, 2024
Merged

Improve File Explorer performance #7337

merged 24 commits into from Feb 13, 2024

Conversation

aliabid94
Copy link
Collaborator

@aliabid94 aliabid94 commented Feb 7, 2024

Previously we would load the entire directory structure at the root directory and send that to the frontend when the fileexplorer component loaded. This could be prohibitively expensive for higher level directories to crawl through all the nested subdirectories, taking on the order of minutes to load. Now the API call only returns the content of a single level, and when opening a folder, we load the content of that folder. See gif below, showing network requests as well.

Recording 2024-02-06 at 21 08 25

Also fixed root_dir update issue.
Fixes: #7149

Demo code to repro fix (change root_dir values ofc):

import gradio as gr

with gr.Blocks() as app:
    file_explorer = gr.FileExplorer(
        value=["/Users/aliabid/Desktop/gradio/demo/hello_blocks/run.py", "/Users/aliabid/Desktop/gradio/demo/change_vs_input/run.py"],
        root_dir="/Users/aliabid/Desktop/gradio/demo/", 
        file_count="multiple",
        height=200
    )
    select_btn = gr.Button("Select")
    selected_file = gr.Textbox(label="Selected")

    select_btn.click(lambda x:x, inputs=file_explorer, outputs=selected_file)

    change_root_btn = gr.Button("Change Root")
    change_root_btn.click(lambda: gr.FileExplorer(root_dir="/Users/aliabid/Desktop/gradio/"), outputs=file_explorer)


if __name__ == "__main__":
    app.launch()

@gradio-pr-bot
Copy link
Contributor

gradio-pr-bot commented Feb 7, 2024

🪼 branch checks and previews

Name Status URL
Spaces ready! Spaces preview
Website ready! Website preview
Storybook ready! Storybook preview
🦄 Changes detected! Details

Install Gradio from this PR

pip install https://gradio-builds.s3.amazonaws.com/712a7fd4f92f71521d324e0eb6fd0db8824638b8/gradio-4.18.0-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@712a7fd4f92f71521d324e0eb6fd0db8824638b8#subdirectory=client/python"

@gradio-pr-bot
Copy link
Contributor

gradio-pr-bot commented Feb 7, 2024

🦄 change detected

This Pull Request includes changes to the following packages.

Package Version
@gradio/fileexplorer patch
gradio patch
  • Maintainers can select this checkbox to manually select packages to update.

With the following changelog entry.

Improve File Explorer performance

Maintainers or the PR author can modify the PR title to modify this entry.

Something isn't right?

  • Maintainers can change the version label to modify the version bump.
  • If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

@abidlabs
Copy link
Member

abidlabs commented Feb 7, 2024

Nice! There are some bugs related to the FileExplorer maybe it would make sense to include fixes in this PR:

(Edit: it looks like @freddyaboulton has already addressed the first one.)

@aliabid94 aliabid94 marked this pull request as ready for review February 8, 2024 22:39
@aliabid94
Copy link
Collaborator Author

ready for review

@abidlabs abidlabs added the windows-tests Run backend tests on Windows on this PR (by default, applied only to the changeset release PR) label Feb 13, 2024
gradio/components/file_explorer.py Show resolved Hide resolved
gradio/components/file_explorer.py Outdated Show resolved Hide resolved
gradio/components/file_explorer.py Outdated Show resolved Hide resolved
demo/file_explorer_component_events/run.py Show resolved Hide resolved
@abidlabs
Copy link
Member

abidlabs commented Feb 13, 2024

Great PR @aliabid94! Everything looks good to me, just left a few nits on the code above. I also added the windows-tests label to test the logic on windows paths.

On the UI side, I preferred having the checkboxes next to everything, even if they weren't clickable. Now, you have this empty space, which looks odd imo:

image

@@ -59,7 +58,7 @@ def __init__(
value: The file (or list of files, depending on the `file_count` parameter) to show as "selected" when the component is first loaded. If a callable is provided, it will be called when the app loads to set the initial value of the component. If not provided, no files are shown as selected.
file_count: Whether to allow single or multiple files to be selected. If "single", the component will return a single absolute file path as a string. If "multiple", the component will return a list of absolute file paths as a list of strings.
root_dir: Path to root directory to select files from. If not provided, defaults to current working directory.
ignore_glob: The glob-tyle pattern that will be used to exclude files from the list. For example, "*.py" will exclude all .py files from the list. See the Python glob documentation at https://docs.python.org/3/library/glob.html for more information.
ignore_glob: The glob-style pattern that will be used to exclude files from the list. For example, "*.py" will exclude all .py files from the list. See the Python glob documentation at https://docs.python.org/3/library/glob.html for more information.
label: The label for this component. Appears above the component and is also used as the header if there are a table of examples for this component. If None and used in a `gr.Interface`, the label will be the name of the parameter this component is assigned to.
every: If `value` is a callable, run the function 'every' number of seconds while the client connection is open. Has no effect otherwise.sed (e.g. to cancel it) via this component's .load_event attribute.
show_label: if True, will display label.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we properly document interactive parameter below and fix #6627?

@pngwn
Copy link
Member

pngwn commented Feb 13, 2024

Yeah I liked the disabled checkboxes, they make the alignment tidier and hint more clearly that selecting the entire folder is explicitly not allowed on this app.

@aliabid94
Copy link
Collaborator Author

I think it's hard to tell the checkboxes are disabled when all of them are disabled, e.g.

Screenshot 2024-02-13 at 4 35 08 PM

but don't feel strongly, will revert

@aliabid94 aliabid94 merged commit 65437ce into main Feb 13, 2024
9 checks passed
@aliabid94 aliabid94 deleted the simplify_file_explorer branch February 13, 2024 22:51
@pngwn pngwn mentioned this pull request Feb 13, 2024
@shimizust
Copy link

Thanks so much for the fix @aliabid94 ! I was just running into this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
windows-tests Run backend tests on Windows on this PR (by default, applied only to the changeset release PR)
Projects
None yet
5 participants