Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EZP-20193: Spi persistence cache using Stash #198

Merged
merged 52 commits into from Mar 7, 2013
Merged

Conversation

andrerom
Copy link
Contributor

@andrerom andrerom commented Jan 9, 2013

This PR adds Persistence cache (aka "StorageEngine" cache) as a layer above the storage engine using a caching library called Stash which supports both APC, memcache(d) and others.

Note: Related to ezsystems/ezpublish-community#42

The benefit of doing this caching at this layer is that you don't have to care about permissions and can easily have one big pool of cache for all users, witch makes it much more memory efficient.

TODO:

  • Persistence factory to lazy load the "inner" persistence handler, aka storage engine*
  • Cache:
    • Section
    • Content Type (published)
    • Location
    • Locations ( loadLocationsByContent() as used by limitations )
    • Language
    • Content (published)
    • Content Info (published)
    • Role(Assignments) (loadRole & loadRoleAssignmentsByGroup as used by permission system)
    • UrlALias ( lookup(), listURLAliasesForLocation() & loadUrlAlias() )
  • Refactoring to improve cache efficiency:
    • Remove unused SPI Location->mainLocationId to avoid aggressive cache clearing on location changes
    • Change RoleAssignment->role to ->roleId to avoid having to swap value for id in cache system
  • SPI logger to profile if other parts should be cached
  • Legacy integration (Clearing cache on changes in legacy when there is an event for it, otherwise have low ttl and document)
  • Change setup wizard to generate Multisite safe default config (APC if available, or dev/null if not)
  • Doc (in issue)

TODO's for another time:

  • Add requirement to Persistence field type values that their ->value must be serializable (see current work-around for DOM object's)
  • Relations Cache
  • ObjectState(Group) ( loadAllGroups(), loadObjectStates(), getContentState() as used by limitations) cache
  • UrlWildcard Cache

* This has the benefit of solving our issue with database connection being openend when repository is loaded. .

@ghost ghost assigned andrerom Jan 16, 2013
*/
public function testGetName()
{
$this->assertEquals( 'PersistenceLogger', $this->logger->getName() );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a class constant instead ?

@lolautruche
Copy link
Contributor

Besides my several comments, looks pretty good. Good job @andrerom !

}
else
{
/** @var $urlAliasUrlLookupCache int */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hinting non-existent variable.

Why?
- All other handlers are interfaces
- There is no base functionality in it
- I need to do this to be able to extend base handler stuff from all handler impl (cache in this case)
@@ -82,6 +82,10 @@ services:
arguments: [@service_container]

ezpublish.api.persistence_handler:
#To disable cache, switch alias to ezpublish.api.storage_engine
alias: ezpublish.spi.persistence.cache
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A semantic config for enabling/disabling cache would be nice 😃

@lolautruche
Copy link
Contributor

So as far as I understood, only APC caching is working with StashBundle ? If so, we might have an issue since APC cache is not shared between workers in FastCGI/FPM mode.
So we really need to get Memcached and File drivers working (the Ephemeral one could be nice as well).

$location = $this->persistenceFactory->getLocationHandler()->create( $locationStruct );

$this->cache->getItem( 'location', $location->id )->set( $location );
$this->cache->clear( 'content', 'locations', $location->contentId );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not take care of content locations cache with root key, set in loadLocationsByContent().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does afaik, Stash cache is hierarchical.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only problem being root key is not separate argument but concatenated to Content id :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction, looked at the doc and you are indeed right.

@andrerom
Copy link
Contributor Author

andrerom commented Mar 6, 2013

Ephemeral is enabled under the hood if InMemory setting is enabled, as it is by default in combination with BlackHole.
I would have preferred to just have Ephemeral, and no BlackHole, as it is pointless unless you really don't want caching at all. But for some reason the author of stash bundle didn't see the need for that, so changing this would mean breaking it.

As for the other issues:

  • File driver should create the stash cache folder for you, needs pull request against stash or stash bundle and new release.
  • Memcache(d), I think it works, it's just that I don't have the extension installed, so it won't accept memcache settings

{
$this->logger->logCall( __METHOD__, array( 'type' => $identifier ) );
$type = $this->persistenceFactory->getContentTypeHandler()->loadByIdentifier( $identifier );
$cache->set( $type->id );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loaded ContentType can be cached here as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, but it might already be cached, if not it will be fetched on next call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in a244002

@lolautruche
Copy link
Contributor

👍

@andrerom
Copy link
Contributor Author

andrerom commented Mar 7, 2013

Thanks, that is it then, to get QA to get access to this I will now merge.

The remaining (non blocking) open topics will be handled separately when time permits:

  • SPI interface for extending the eZ tab in the web profiler
  • Get rid of custom branch of StashBundle to have the stash fix mentioned above
  • Get FileDriver working better by making pull to upstream to make it create the the cache folder during compilation

Other remaining issues left for future versions are written in the description of this PR.

andrerom added a commit that referenced this pull request Mar 7, 2013
EZP-20193: Spi persistence cache using Stash
@andrerom andrerom merged commit 1dbc98b into master Mar 7, 2013
@andrerom andrerom deleted the spiPersistenceCache branch March 7, 2013 10:57
@coveralls
Copy link

Coverage Status

Changes Unknown when pulling 6254c8f on spiPersistenceCache into * on master*.

ViniTou pushed a commit that referenced this pull request Feb 23, 2022
Looks like a83c3ec for Symfony 5.2 was applied by accident on the Kernel branch 1.2 which supports Symfony 5.1.x.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
8 participants