Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the performance of CachedIdentifiersExtractor #1257

Merged
merged 1 commit into from
Jul 20, 2017

Conversation

dunglas
Copy link
Member

@dunglas dunglas commented Jul 19, 2017

Q A
Bug fix? no
New feature? no
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets n/a
License MIT
Doc PR api-platform/docs#1234

Follows #1256.

@soyuka
Copy link
Member

soyuka commented Jul 19, 2017

hmm tests failed though

@dunglas
Copy link
Member Author

dunglas commented Jul 19, 2017

I'm on it

@meyerbaptiste
Copy link
Member

Could we not do the same for all Cached* classes?

@@ -33,6 +33,7 @@
private $cacheItemPool;
private $propertyAccessor;
private $decorated;
private $memoryCache = [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming... See #1256 (comment)

@dunglas dunglas force-pushed the improve_cached_identifiers branch from 649d81d to 4d4b5c4 Compare July 19, 2017 20:42
@dunglas
Copy link
Member Author

dunglas commented Jul 19, 2017

I've tried something but I would like a review @soyuka. I'm not sure I respect your initial intent.

@api-platform api-platform deleted a comment from bco-trey Jul 19, 2017
@dunglas dunglas force-pushed the improve_cached_identifiers branch from 4d4b5c4 to b68cbbf Compare July 19, 2017 21:21
$cacheIsHit = false;
$keys = $this->getKeys($item, function ($item) use (&$identifiers) {
return $identifiers = $this->decorated->getIdentifiersFromItem($item);
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is cool stuff!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤷‍♂️

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a reference to an unknown variable and declaring it in a closure looks cool to me haha 😅

@dunglas dunglas merged commit c6a7bef into api-platform:master Jul 20, 2017
@dunglas dunglas deleted the improve_cached_identifiers branch July 20, 2017 07:50
@soyuka
Copy link
Member

soyuka commented Jul 20, 2017

@meyerbaptiste this one is slightly different and more complex then metadata classes. Initial PR #997

@meyerbaptiste
Copy link
Member

Yep @soyuka, I posted my message before he refactored his PR. My previous comment is now more relevant for #1256.

@soyuka
Copy link
Member

soyuka commented Jul 20, 2017

Yeah I agree we have a lot of repetition in those "cached" classes, finally it's always the same:

  1. check cache (key is PREFIX + md5(resourceClass))
  2. if cached return cached value
  3. if no cache call decorator and cache

Do you have an idea to remove the duplicated code (only diff is the injected decorator and the cache key prefix)?

@teohhanhui
Copy link
Contributor

Is it safe for us to use Symfony\Component\Cache\Adapter\ArrayAdapter with $storeSerialized = false? Since our metadata classes are all immutable... And Symfony\Component\PropertyInfo\Type is immutable too.

@teohhanhui
Copy link
Contributor

In the case of CachedIdentifiersExtractor we are storing an array of strings, so it's "immutable" too.

@dunglas
Copy link
Member Author

dunglas commented Jul 20, 2017

ArrayAdapter will be slower (more function calls) and - as pointed out by Nicolas Grekas on Twitter - is not part of cache.system because it's always slower than a local cache. Symfony components use a memory cache too.

@dunglas
Copy link
Member Author

dunglas commented Jul 20, 2017

We can probably go one step further for this class: it should be possible to cache (in a memory cache) the results as well.

$this->assertEquals(['id' => 1], $identifiersExtractor->getIdentifiersFromItem($dummy));
$expectedResult = ['id' => 1];
$this->assertEquals($expectedResult, $identifiersExtractor->getIdentifiersFromItem($dummy));
$this->assertEquals($expectedResult, $identifiersExtractor->getIdentifiersFromItem($dummy), 'Trigger the local cache');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this test verify that we are triggering the local cache? Also, it should not matter in a unit test.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

He's hacking coverage to please @Simperfit 😆

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this second call is crucial to test correct behaviour, i.e. that we return the correct value when cache is used. But it's not testing whether we're triggering the local cache.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait, we already have different test cases for that... Lol

Copy link
Member Author

@dunglas dunglas Jul 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really for the coverage. You're right @teohhanhui, it doesn't verify that the local cache actually works (it's very hard to test reliably), however it tests if the local cache doesn't introduce a bug.

For instance, if some one contribute this patch:

-return $this->memoryCache[$resourceClass];
+return $this->memoryCache[$resourceC];

The test will detect the problem. Without this line, it will not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dunglas Doesn't testSecondPass already do that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. testSecondPass should be update to bypass the local cache IMO (by creating a new instance).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$localCache is unambiguous. #preach

@soyuka
Copy link
Member

soyuka commented Jul 20, 2017

@dunglas I don't think that caching values is worth it. Also cache invalidation will be an issue. This is already improving performances a lot because we avoid calling the whole metadataFactory thing from IdentifierExtractor to find identifiers (simple and sufficient IMO).

hoangnd25 pushed a commit to hoangnd25/core that referenced this pull request Feb 23, 2018
…tifiers

Improve the performance of CachedIdentifiersExtractor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants