Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change S3 AwsS3DirectoryBrowser to apply FilePrefix filtering at the server #8

Closed
SRJames opened this issue Feb 3, 2023 · 1 comment
Labels
AWS S3 enhancement New feature or request

Comments

@SRJames
Copy link

SRJames commented Feb 3, 2023

Currently the fileprefix is applied client side. If there are many files in a path then this can be very slow.

A small change to AWSS3DirectoryBrowser would allow the filtering to be applied on the server.
E.g.

Task ListFolderAsync(List<Blob> container, string path, ListOptions options, CancellationToken cancellationToken)
      {
         var request = new ListObjectsV2Request()
         {
            BucketName = _bucketName,
            Prefix = FormatFolderPrefix(path),
            Delimiter = "/"   //this tells S3 not to go into the folder recursively
         };

         // if we have provided a file prefix, then append it on.
         // as S3 can perform the filtering at source
         if (!string.IsNullOrEmpty(options.FilePrefix))
         {
            request.Prefix += options.FilePrefix;
         }

@robinrodricks
Copy link
Owner

Looks good, can you file a PR?

@robinrodricks robinrodricks added enhancement New feature or request AWS S3 labels Feb 4, 2023
@SRJames SRJames closed this as completed May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AWS S3 enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants