Improve performance of oiksc_create_api

[bobbingwide]

In a largish site, as the number of records in wp_posts and wp_postmeta increases so the performance of SQL queries degrades. When parsing an API the time taken to find the post IDs of related APIs, classes, hooks etc causes server transactions to take an inordinate amount of time.

On my PC, with 47K posts and 230K post meta each query to find oik_api posts based on post meta data takes up to 500 milliseconds. Note: There are 15K oik_api posts with the same number of post meta rows with meta_key _oik_api_name

This is 3 times the elapsed time when there are a third of the number of records.

Note also that in the larger system the tables are InnoDB, in the other they're MyISAM. This may make a difference; I don't know. I'm using MySQL 5.6.15. InnoDB buffer pool size is 291MiB.

Server transaction times range considerably, but can easily take up to 30 seconds with SQL processing being 80% of the processing.

Proposed solution

We know that performance of queries against wp_postmeta is slow due to the lack of an index on the meta_value column ( one reason why /scribu/wp-posts2posts ), but we aren't yet ready to move away from using wp_postmeta.

We believe some performance gain will be made by pre-loading known information related to the API in question and using in-memory lookups for the relevant IDs and the like.

[bobbingwide]

We can also improve performance by not recreating the parsed source record if the source we're parsing hasn't changed since we last parsed it, even though the file in which the code exists has changed. For some source files, with many functions or methods, this should offer a significant performance improvement.

[bobbingwide]

We can also improve performance by not recreating the parsed source record if the source we're parsing hasn't changed since we last parsed it

I think this can be closed now, though I’d like to confirm I implemented this.

[bobbingwide]

I think this confirms I have

Processing API: rss2_site_icon, wp-includes/feed.php, wordpress, wordpress
Nothing to do for: rss2_site_icon