Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Converting post metadata to a better system #185

Open
sybrew opened this issue Aug 28, 2017 · 4 comments
Open

Converting post metadata to a better system #185

sybrew opened this issue Aug 28, 2017 · 4 comments
Assignees
Labels
[Impact] Performance [Resolution] Maybe later [Type] Code debt [Type] Enhancement

Comments

@sybrew
Copy link
Owner

sybrew commented Aug 28, 2017

Currently, post metadata is stored through individual keys.

This means there are a lot of entries stored in the database, and key collision is prominent.

Current issues:

  1. Prone to key collisions.
  2. There are loads of entries, increasing exponentially as we add more options.
  3. Entries that don't start with an underscore, are shown as separated custom fields. Offenders:
    • exclude_from_archive
    • exclude_local_search
    • redirect
  4. The data isn't grouped and therefore, requires more effort to find, debug, handle, etc.

To resolve this, we must upgrade the metadata from single entries to an array with a common (namespaced) key.

References:

How we did the term meta upgrade:

In WordPress 4.4, new term metadata functionality was added. In TSF 2.7, the Term Metadata upgrade was handled and annotated.

This upgrade was done in one single request.

/**
* Upgrades term metadata for version 2701.
*
* @since 2.7.0
*/
function the_seo_framework_do_upgrade_2701() {
$term_meta = get_option( 'autodescription-term-meta' );
foreach ( (array) $term_meta as $term_id => $meta ) {
add_term_meta( $term_id, THE_SEO_FRAMEWORK_TERM_OPTIONS, $meta, true );
}
update_option( 'the_seo_framework_upgraded_db_version', '2701' );
}

This can't be done for posts.

Obstacles:

  1. Term data was converted from a single database entry. Post data has to be converted from thousands to even millions of database entries.
  2. TSF's post-meta API has to be prominently adopted, a must-use for developers so they won't see a change of functionality. This already exists: get_custom_field().
  3. Old and new data both must be checked for; i.e. either through flags or through plain deletion if new data is inserted.

Propositions

  1. Convert on post save, while maintaining two versions. This means:
    • All posts saves will convert data to the newer version, discarding the old.
  2. Convert on-the-fly on post visit, while maintaining two versions. This means:
    • All posts visits will convert data to the newer version, discarding the old.
  3. Upgrade loop. This means:
    • After plugin update, the user is prompted to upgrade posts.
    • When activated, it will loop through all N posts every single admin request until done.

Now, all propositions can work alongside. In all cases, new data will be tested for existence, and will then fall back to old.

Footnotes

When done follow-up: #48
Related functionality: #84

@sybrew sybrew added [Type] Code debt [Type] Enhancement [Impact] Performance labels Aug 28, 2017
@sybrew sybrew added this to the 3.2.0 milestone Aug 28, 2017
@sybrew sybrew self-assigned this Aug 28, 2017
@sybrew
Copy link
Owner Author

sybrew commented Dec 12, 2017

Candidate for 3.1.0. But, I don't want to rush this as it's a vital component and it needs to be done perfectly.

@sybrew sybrew modified the milestones: 3.2.0, 3.1.0 Dec 12, 2017
@sybrew sybrew removed this from the 3.1.0 milestone Jul 22, 2018
@sybrew
Copy link
Owner Author

sybrew commented Jul 22, 2018

Maybe later, it's not pressing.

@sybrew
Copy link
Owner Author

sybrew commented Feb 20, 2020

In TSF 4.0, we revised how post-metadata is handled, with this issue in mind.
See the post-data.class.php file.

@sybrew
Copy link
Owner Author

sybrew commented Jul 13, 2020

When we move to this method of storing data, plugins like "Better Search and Replace", and possibly "WP-Optimize", will annihilate the metadata as they pass over it.

Here's one recent finding, where "Better Search and Replace" replaces Extension Manager's neatly-packed metadata with (string) "1".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Impact] Performance [Resolution] Maybe later [Type] Code debt [Type] Enhancement
Projects
None yet
Development

No branches or pull requests

1 participant