Skip to content

refactor: dont list remote storages when precise file path is used#2331

Merged
norberttech merged 1 commit into1.xfrom
filesystem-streams-optimization
Apr 25, 2026
Merged

refactor: dont list remote storages when precise file path is used#2331
norberttech merged 1 commit into1.xfrom
filesystem-streams-optimization

Conversation

@norberttech
Copy link
Copy Markdown
Member

Change Log


Added

Fixed

Changed

  • Avoid listing the remote storages when direct file path is used

Removed

Deprecated

Security

This behavior was noticed in one of the projects I have been consulting.
What happens is that even when the extractor has a direct file path s3://some-bucket/some-folder/file.xml for example the XMLParserExtrator is using list action on Streams

foreach ($context->streams()->list($this->path, $this->filter()) as $stream) {

And then in FilesystemStreams regardless is this is direct file path or folder or pattern we stilld call list.

public function list(Path $path, Filter $pathFilter) : \Generator
{
$fs = $this->fstab->for($path);
foreach ($fs->list($path, $pathFilter) as $file) {
yield $fs->readFrom($file->path);
}

This PR changes that behavior, now list methods of AzureBlob and AsyncAWSS3Filesystem will first (if that option is enabled) check if path is pointing to a file.

This way it wont need an actual list bucket / container permission

@norberttech norberttech added this to the 0.37.0 milestone Apr 25, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 25, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.29%. Comparing base (0a7cf1f) to head (fbea7de).
⚠️ Report is 1 commits behind head on 1.x.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@             Coverage Diff             @@
##              1.x    #2331       +/-   ##
===========================================
+ Coverage   10.84%   84.29%   +73.45%     
===========================================
  Files        1540     1540               
  Lines       58115    58157       +42     
===========================================
+ Hits         6300    49022    +42722     
+ Misses      51815     9135    -42680     
Components Coverage Δ
etl 90.61% <ø> (+90.55%) ⬆️
cli 85.76% <ø> (+85.76%) ⬆️
lib-array-dot 94.82% <ø> (+94.82%) ⬆️
lib-azure-sdk 61.13% <ø> (+61.13%) ⬆️
lib-doctrine-dbal-bulk 95.16% <ø> (+95.16%) ⬆️
lib-filesystem 85.03% <ø> (+75.84%) ⬆️
lib-types 89.27% <ø> (+88.55%) ⬆️
lib-parquet 70.18% <ø> (+0.50%) ⬆️
lib-parquet-viewer 83.04% <ø> (+83.04%) ⬆️
lib-snappy 90.65% <ø> (+0.93%) ⬆️
lib-dremel 0.00% <ø> (ø)
lib-postgresql 88.66% <ø> (+88.66%) ⬆️
lib-telemetry 89.57% <ø> (+89.57%) ⬆️
bridge-filesystem-async-aws 92.63% <100.00%> (+92.63%) ⬆️
bridge-filesystem-azure 91.48% <100.00%> (+91.48%) ⬆️
bridge-monolog-http 97.45% <ø> (+97.45%) ⬆️
bridge-openapi-specification 91.50% <ø> (+91.50%) ⬆️
symfony-http-foundation 79.45% <ø> (+79.45%) ⬆️
bridge-psr18-telemetry 100.00% <ø> (+100.00%) ⬆️
bridge-psr7-telemetry 100.00% <ø> (+100.00%) ⬆️
bridge-telemetry-otlp 89.29% <ø> (+89.29%) ⬆️
bridge-symfony-http-foundation-telemetry 88.23% <ø> (+88.23%) ⬆️
bridge-symfony-filesystem-bundle 93.67% <ø> (+93.67%) ⬆️
bridge-symfony-postgresql-bundle 95.42% <ø> (+95.42%) ⬆️
bridge-symfony-postgresql-messenger 99.59% <ø> (+99.59%) ⬆️
bridge-symfony-telemetry-bundle 67.73% <ø> (+67.73%) ⬆️
adapter-chartjs 84.72% <ø> (+84.72%) ⬆️
adapter-csv 88.65% <ø> (+88.65%) ⬆️
adapter-doctrine 92.00% <ø> (+92.00%) ⬆️
adapter-elasticsearch 92.78% <ø> (+92.78%) ⬆️
adapter-google-sheet 99.02% <ø> (+99.02%) ⬆️
adapter-http 67.91% <ø> (+67.91%) ⬆️
adapter-json 87.61% <ø> (+87.61%) ⬆️
adapter-logger 47.36% <ø> (+47.36%) ⬆️
adapter-parquet 78.89% <ø> (+78.89%) ⬆️
adapter-text 81.25% <ø> (+81.25%) ⬆️
adapter-xml 82.61% <ø> (+82.61%) ⬆️
adapter-avro 0.00% <ø> (ø)
adapter-excel 94.02% <ø> (+94.02%) ⬆️
adapter-postgresql 0.00% <ø> (ø)
bridge-phpunit-postgresql 72.97% <ø> (+41.89%) ⬆️
bridge-phpunit-telemetry 60.33% <ø> (+58.37%) ⬆️
bridge-postgresql-valinor 100.00% <ø> (+100.00%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@norberttech norberttech merged commit 3e84239 into 1.x Apr 25, 2026
31 checks passed
@norberttech norberttech deleted the filesystem-streams-optimization branch April 25, 2026 16:36
@github-project-automation github-project-automation Bot moved this from Todo to Done in Roadmap Apr 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant