Mimi

[sherakama]

READY FOR REVIEW

Summary

• // Some sites have wwaaayyy too many items in the capx_profiles table for this
• // to load with an appropriate amount of memory. You can thank Zach Chandler
• // for finding out this edge case. As such, we will apply two different
• // stategies for fetching the CAPx metadata. If the amount of profile entries
• // are over a value then switch from one big cached call to multiple pings of
• // the database. We are trading PHP memory for hammering the DB with multiple
• // queries.

Needed By (Date)

• July 1st.

Urgency

• Moderate.

Steps to Test

1. Clone https://sites.stanford.edu/mimisbrunnr/ to local
2. Check out this branch
3. Play with threshold setting

Affected Projects or Products

• Not really. Just Mimi.

Associated Issues and/or People

• None.

See Also

• PR Checklist

[sherakama]

@boznik,

I've added you to review this approach. So this issue was caused by someone importing ALL publications from EVERYONE on the CAP API. Yep, that was Zach. Anyhow, this caused the capx_profiles table to be very large (~240000). That amount of data is too large to pull all at once but a single request is more efficient when the amount of data is lower. I have introduced a threshold variable to swap out strategies. Please have a review.

[jbickar]

Undefined offset: 4 in stanford_capx_entity_load() (line 292 of stanford_capx/stanford_capx.entity.hooks.inc

[jbickar]

I'm trying to reproduce the Undefined offset, but I also get this with drush rr (on mimi):

% drush rr
The registry has been rebuilt via registry_rebuild (A).                                                                                                                                                                    [success]
array_flip(): Can only flip STRING and INTEGER values! entity.controller.inc:742                                                                                                                                           [warning]
array_flip(): Can only flip STRING and INTEGER values! entity.controller.inc:219                                                                                                                                           [warning]
array_flip(): Can only flip STRING and INTEGER values! entity.controller.inc:742                                                                                                                                           [warning]
array_flip(): Can only flip STRING and INTEGER values! entity.controller.inc:219                                                                                                                                           [warning]
All caches have been cleared with drush_registry_rebuild_cc_all.                                                                                                                                                           [success]
The registry has been rebuilt via drush_registry_rebuild_cc_all (B).                                                                                                                                                       [success]
array_flip(): Can only flip STRING and INTEGER values! entity.controller.inc:742                                                                                                                                           [warning]
array_flip(): Can only flip STRING and INTEGER values! entity.controller.inc:219                                                                                                                                           [warning]
array_flip(): Can only flip STRING and INTEGER values! entity.controller.inc:742                                                                                                                                           [warning]
array_flip(): Can only flip STRING and INTEGER values! entity.controller.inc:219                                                                                                                                           [warning]
All caches have been cleared with drush_registry_rebuild_cc_all.                                                                                                                                                           [success]
All registry rebuilds have been completed.                                                                                                                                                                                 [success]

[jbickar]

I can reproduce the Undefined offset notice on the mimi homepage.

Clearing the cache from the GUI gives:
Notice: Undefined property: CAPx\Drupal\Entities\CFEntityType::$machine_name in capx_filters_entity_presave() (line 343 of /Users/jbickar/Sites/mimisbrunnr/sites/default/modules/stanford/stanford_capx/modules/capx_filters/capx_filters.module).

[zchandler]

Thanks Guys!

[sherakama]

@jbickar
You should no longer get the PHP errors. Thanks for finding those.

[jbickar]

Works great; nice work.

[jbickar]

This is a fix to bad logic, unrelated to the memory issue, yeah?

[sherakama]

Yeah, Was acting on too many types.

[jbickar]

Fun times:
stanford_capx_entity_load_limit = 1000, everything is wunderbar
stanford_capx_entity_load_limit = 100000, things start to get wonky
stanford_capx_entity_load_limit = 1000000, WSOD

That's pretty neat.

[sherakama]

Thanks for reviewing.