Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH Improve performance of InterventionBackend::getImageResource() #409

Closed

Conversation

christopherdarling
Copy link
Contributor

@christopherdarling christopherdarling commented Jun 26, 2020

when using the default local filesystem

From my tests, I'm seeing a speed improvement of 10-20% on requests that require the generation of Intervention_Mamipulation cache files (see related merged PR #362) or image manipulations performed by GD.

n.b. the 10-20% figure was based on a page using three reasonably sized original images (~3000px square), I presume the % will increase with the number of images on the page.

@kinglozzer
Copy link
Member

The lines above say:

Currently exif data can only be read from file paths and not streams

I assume that’s why we write a local file and only use streams as a fallback - we need exif data to correct orientation. Would that be broken by this change?

@christopherdarling
Copy link
Contributor Author

christopherdarling commented Jun 27, 2020

The lines above say:

Currently exif data can only be read from file paths and not streams

I assume that’s why we write a local file and only use streams as a fallback - we need exif data to correct orientation. Would that be broken by this change?

Ah good point. Isn't EXIF data from streams supported from PHP 7.2 now? https://bugs.php.net/bug.php?id=65187

related: @dhensby looks like he was looking into using local paths where available... #34 (comment)

@kinglozzer
Copy link
Member

@christopherdarling that would be good as presumably it would save an unnecessary copy into the temp directory. I wonder if we could somehow grab the underlying Flysystem adapter and check $adapter instanceof Local?

@dhensby
Copy link
Contributor

dhensby commented Jun 29, 2020

This needs linting.

So is this speed improvement only for when the cropped images need to be generated and otherwise not a problem? If so, I feel like it's of limited importance, but if we can improve it easily we should.

If exif data can be read from streams in 7.2 that's good, but we still support lower PHP versions and I think that we need to make sure that those don't see a regression.

@christopherdarling
Copy link
Contributor Author

christopherdarling commented Jun 29, 2020

So is this speed improvement only for when the cropped images need to be generated and otherwise not a problem? If so, I feel like it's of limited importance, but if we can improve it easily we should.

I don't think so. I think it's also an improvement for when a dimension (width/height) is needed which relies on getImageResource() also.

If exif data can be read from streams in 7.2 that's good, but we still support lower PHP versions and I think that we need to make sure that those don't see a regression.

Looking into this today, hopefully.

@christopherdarling
Copy link
Contributor Author

christopherdarling commented Jun 30, 2020

I've just sent up an alternative version which maintains the use of streams, no temp files in any case and extracts exif data and handles rotation (I think for iPhones). Some quick stats from a local dev site I'm working on at the mo'.

No Intervention_Manipulations cache & Resampling 5 images 5 times (total 25 resampled images)

Run# Before After Difference
1 14.25s 10.89s -3.36s
2 14.22s 13.67s -0.55s
3 15.57s 10.50s -5.07s

No Intervention_Manipulations cache & Resampling 1 image 5 times (total 5 resampled images)

Run# Before After Difference
1 3.57s 3.43s -0.14s
2 3.57s 3.39s -0.18s
3 3.65s 3.76s +0.11s

No Intervention_Manipulations cache & Resampling files already generated - 1 image 5 resampled versions

Run# Before After Difference
1 1.44s 960ms -0.48s
2 1.48s 895ms -0.585s
3 1.31s 952ms -0.358s

So some quite considerable improvements in most cases

Note: I haven't done any testing with any other asset stores like S3, just the default.

@kinglozzer
Copy link
Member

@christopherdarling So that’s failing on PHP 7.1 (as you pointed out, exif_read_data() doesn’t support streams prior to 7.2). I’m not too keen on us handling orientation ourselves, but given the performance benefits I think it’s acceptable until there’s (hopefully) a fix in the intervention lib. I think we have a choice to make:

  1. Switch to use streams for PHP 7.2+, provide BC for PHP 7.1 (via the existing file-based method)
  2. Switch to only use streams, allow PHP 7.1 to operate but with potentially broken orientation
  3. Drop PHP 7.1 support entirely (this would need wider discussion)
  4. Leave as-is

My order of preference is 3 > 1 > 4 > 2. Thoughts @silverstripe/core-team?

@maxime-rainville
Copy link
Contributor

Just had a look at our stats for PHP7.1:

  • 29% of SCP stacks are on PHP7.1
  • 20% of CWP stacks are on PHP7.1

@maxime-rainville
Copy link
Contributor

Maybe, I'm misreading this, but looking at the Intervention implementation, sounds like it would make more sense to do a PR there to fix the underlying problem that forces us to create a physical file.

https://github.com/Intervention/image/blob/master/src/Intervention/Image/Commands/ExifCommand.php

@christopherdarling
Copy link
Contributor Author

While i agree, it looks like @dhensby was looking into this some time ago Intervention/image#745 so figured bed to resolve it here for the time being.

@dhensby
Copy link
Contributor

dhensby commented Jul 2, 2020

I agree we should be fixing this in the upstream library - that's the whole point of open source; rather than implementing our own specific fix we can apply a generic fix in the upstream that helps everyone.

// somewhere along that process the stream changes and exif becomes
// unreadable
try {
$exif = @exif_read_data($stream);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't be using @ to silence errors

$exif = @exif_read_data($stream);

if (!$exif || !array_key_exists('Orientation', $exif)) {
throw new NotSupportedException('missing exif or exif[Orientation]');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we throwing and then catching (and ignoring) this exception?

@christopherdarling
Copy link
Contributor Author

Looks like @tractorcow had some ideas of how to implement this on your issue thread @dhensby. I presumed a jump from PHP 5.4 (in their composer.json) to 7.2 would have been a blocker?

I won't bother implementing your feedback Daniel if the consensus is not to merge anyway? Not sure I'll be able to do the PR for the Intervention/Image repo and for the meantime I'll be running my fork, but if I do figure it out before anyone else gets to it I'll update this PR

@emteknetnz
Copy link
Member

This pull request hasn't had any activity for a while. Are you going to be doing further work on it, or would you prefer to close it now?

@christopherdarling
Copy link
Contributor Author

Quite happy to do further work on the PR if it's likely to be accepted? I got the impression that it wouldn't be accepted so have since continued to use my fork, which seems to be working well on a number of sites.

@emteknetnz
Copy link
Member

Looking back through the previous comments, you're right it doesn't seem likely. I'll close this one for now though it could be resurrected in the future if people feel this should be progressed further.

@emteknetnz emteknetnz closed this Sep 10, 2020
@christopherdarling
Copy link
Contributor Author

christopherdarling commented Sep 10, 2020 via email

@sminnee
Copy link
Member

sminnee commented Sep 11, 2020

From my perspective the main blocker was PHP 7.1 support. I can live with doing without the upstream patch if that seems impractical. I know that PHP 7.1 is EOL, but that occurred less than a year ago and dropping support for it will probably frustrate more people than the issue this PR resolves.

InterventionBackend is designed to be a pluggable service. Perhaps the best bet is to offer this implementation and a second service and retain the previous code as LegacyInterventionBackend.

If we implemented http://github.com/silverstripe/silverstripe-framework/issues/9683 then the yaml config could be smart enough to choose the legacy implementation for PHP 7.1.

@dhensby
Copy link
Contributor

dhensby commented Sep 11, 2020

I still believe this needs to be fixed in the upstream library and the effort should be focused there - I've put some time into it myself, but it's a bit complicated because of the way intervention image is designed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants