Plugin support #1075

merged 123 commits into from Oct 5, 2016


None yet

4 participants

borekb commented Jun 1, 2016 edited

Resolves most of #1036.

Opening this PR early, it will be a long-running one, probably.

🚧 Progress: #1075 (comment)

Update 4th Oct 2016: We're mostly done with this, some smaller things have been moved to separate issues and are linked from #1075 (comment). If you want to provide feedback, please read and leave a comment.

@JanVoracek JanVoracek self-assigned this Jun 3, 2016
@JanVoracek JanVoracek added this to the 4.0 milestone Jun 3, 2016
@vasek17 vasek17 closed this Jul 11, 2016
@vasek17 vasek17 deleted the 1036-plugin-support branch Jul 11, 2016
@JanVoracek JanVoracek restored the 1036-plugin-support branch Jul 11, 2016

Closed by mistake; reopening.

@JanVoracek JanVoracek reopened this Jul 11, 2016
JanVoracek added some commits Jul 12, 2016
@JanVoracek JanVoracek [#1036] Removed all custom entity-related *ChangeInfo classes and all…
… custom Bulk*ChangeInfo classes

Note: This commit will not pass the tests. I haven't found any easy way how to do this change in small steps. There remains a lot of related changes but is already too big; thus, I will commit it in multiple commits.
@JanVoracek JanVoracek [#1036] Updated docs of actions 796ee17
@JanVoracek JanVoracek [#1036] Updated ChangeInfoEnvelopeTest to use new "ChangeInfo style" b959a98
@JanVoracek JanVoracek [#1036] All entity-specific StorageTests replaced with two generic te…
…st classes
@JanVoracek JanVoracek [#1036] `actions.yaml` renamed to `actions.yml` db8afba
@JanVoracek JanVoracek [#1036] Restored accidentally removed line from `wordpress-schema.yml` c92d090
@JanVoracek JanVoracek [#1036] Most of ChangeInfo preprocessors now work with general Entity…
@JanVoracek JanVoracek [#1036] Fixed number of parameters at filters da43f26
@JanVoracek JanVoracek [#1036] Plugin-specific VP hooks are now automatically registered 1becb2c
@JanVoracek JanVoracek [#1036] Entity-specific hooks in `setup-hooks.php` now work with gene…
…ral EntityChangeInfo
@JanVoracek JanVoracek [#1036] All `edit` actions have lower priorities than any other action 169014d
@JanVoracek JanVoracek [#1036] Restored VP-User-Login tag on usermeta actions c635cc9
@JanVoracek JanVoracek [#1036] Temporarily returned method `EntityInfo::buildFromCommitMessage` 2772d21
@JanVoracek JanVoracek [#1036] `ActionsInfo` is now registered in DI and injected as a depen…
@JanVoracek JanVoracek [#1036] ActionsInfo now parses action files (for now it uses only the…
… last one)
@JanVoracek JanVoracek [#1036] Removed tag VP-Post-UpdatedProperties 12f8ca5
@JanVoracek JanVoracek [#1036] Added mocks of ActionsInfo 89e4abf
@JanVoracek JanVoracek [#1036] ChangeInfo objects are now parsed from commit messages using …
…`ChangeInfoFactory` instead of many `buildFromCommitMessage` methods
@JanVoracek JanVoracek [#1036] Methods common for `EntityChangeInfo` and `TrackedChangeInfo`…
… moved to `TrackedChangeInfo`
@JanVoracek JanVoracek [#1036] `versionpress-functions.php` renamed to `internal-functions.php` ebab9af
@JanVoracek JanVoracek [#1036] Added `public-functions.php` containing functions for develop…
…ers of 3rd party plugins
@JanVoracek JanVoracek [#1036] `entityName` in `ChangeInfo` objects renamed to `scope` 8c59696
@JanVoracek JanVoracek [#1036] Changes related to Composer packages, plugins, themes, transl…
…ations, WP update and VP activation / deactivation no longer use ChangeInfo classes. They are now represented by a generic TrackedChangeInfo and definitions in actions.yml
@JanVoracek JanVoracek [#1036] RevertChangeInfo replaced with generic TrackedChangeInfo + re…
@JanVoracek JanVoracek [#1036] Removed unnecessary use statements 5f17ab4
@JanVoracek JanVoracek [#1036] Initial support new generic ChangeInfo objects in Reverter db4cff1
@JanVoracek JanVoracek [#1036] Fixed storage tests and unit tests 6b9cc22
@JanVoracek JanVoracek [#1036] Hooks are now registered on 'init' action e7f9439
@JanVoracek JanVoracek [#1036] HookMock now supports actions 0e0acd1
@JanVoracek JanVoracek [#1036] Old GUI now uses new method of parsing ChangeInfo objects fro…
…m commits
@JanVoracek JanVoracek [#1036] Temporary fix of ChangeInfo preprocessors a706075
@JanVoracek JanVoracek [#1036] Fixed mentions of no longer existing class `ChangeInfoMatcher` db2b337
@JanVoracek JanVoracek [#1036] Fixed some tests to work with new ChangeInfo classes f752402
@JanVoracek JanVoracek [#1036] Refactoring - removed unnecessary polyfill for `array_column`…
…, variadic function uses the `...` operator, renaming
@JanVoracek JanVoracek [#1036] Action piping now supports multiple arguments 165793d
@JanVoracek JanVoracek [#1036] Fixed displaying actions for undo and revert d9107fd
@JanVoracek JanVoracek [#1036] OptionsTest is no longer prone to "randomly" created WPLANG o…
@JanVoracek JanVoracek [#1036] Test covering renaming of media file is now skipped - it's im…
…possible to submit the edit form using Selenium
@JanVoracek JanVoracek [#1036] Increased priority of `term/create` 64c94eb
@JanVoracek JanVoracek [#1036] Editing postmeta of attachment now commits the upload file (u…
…seful e.g. for rotations)
@JanVoracek JanVoracek [#1036] Widgets are no longer saved in two requests (probably since W…
…P 4.5)
@JanVoracek JanVoracek [#1036] Increased priority of `translation/activate` (switching to no…
…n-installed language does `translate/install` and `translate/activate` - the activation is more important here)
@JanVoracek JanVoracek [#1036] Small tweaks of tests – Firefox is always maximized before ru…
…nning tests in wp-admin and CommentTestSeleniumWorker doesn't create post every time it needs to create a comment
JanVoracek commented Aug 12, 2016 edited

Current status

  • Storages
    • Replace classes specific for entities with generic ones
    • Extract code from shouldBeSaved methods to filters
    • Extract logic for creating ChangeInfo objects to filters
  • ChangeInfo
    • Replace classes specific for entities with generic ones
    • Extract logic for the detection of an action to filters
    • Implement automatic filling of tags from entities
    • Add filter for custom tags
    • Add filter for custom paths
    • Introduce priorities for actions and sort the ChangeInfo objects by them
    • Support using values of tags in commit messages
    • Add filter for customizing the commit message
    • Replace Committer::forceChangeInfo with a simple function
    • 👉 #1121 Support relative priorities (term/create has higher priority than option/create)
    • 👉 #1121 Support priorities for whole scopes (postmeta has lower priority than post)
    • 👉 #1121 Edit action (*/edit) should probably have lower priority than other (create, delete, draft)
  • schema.yml
    • Support loading multiple schema.yml files
    • Support defining new relations on existing entities
  • shortcodes.yml
    • Support loading multiple shortcodes.yml files
  • actions.yml
    • Support loading multiple actions.yml files
    • Support defining new actions in existing scopes
    • Restore actions.yml files from git after clone / pull / push / undo / rollback etc.
  • Synchronizers
    • Replace classes specific for entities with generic ones
    • Remove hardcoded synchronizationSequence
    • Move code from doEntitySpecificActions in filters
    • Support custom actions before and after sync
    • Introduce better syntax for cache invalidation
    • Replace transformEntities with a filter
    • 👉 #1122 Refactor tests
    • Create DB tables before sync
  • Committer
    • 👉 #1123 Replace pre-processing mechanism of ChangeInfo classes with WP filters (we need some simple format instead of ChangeInfo here)
    • 👉 #1123 Use multiple sources for ChangeInfo objects instead of forceChangeInfo method
    • 👉 #1124 Introduce some markup language for formatting of commit messages
    • 👉 #1124 Update GUI to support the markup language
  • Initialization
    • Remove "entity-specific actions" from Initializer
    • 👉 #1125 Generate all VPIDs before saving (to handle circular references)
  • Other
    • Support saving M:N relationships (e.g. term_relationship)
    • Remove "entity-specific" code from WpdbMirrorBrige (postmeta)
JanVoracek and others added some commits Aug 18, 2016
@JanVoracek JanVoracek [#1036] All `actions.yml` files are now archived in a Git note and th…
…e displaying logic uses them.
@JanVoracek JanVoracek [#1036] DbSchemaInfo now supports multiple schema files 0d5bb52
@JanVoracek JanVoracek [#1036] `wordpress-schema.yml` renamed to `.schema.yml` and moved fro…
…m `src/Database` to `.versionpress`
@JanVoracek JanVoracek [#1036] Shortcodes are now also loaded from all `shortcodes.yml` file…
…s defined by active plugins
@JanVoracek JanVoracek [#1036] Slashes in plugin names has to be sanitized a962fa4
@JanVoracek JanVoracek [#1036] SqlQueryParser is not needed for SELECT queries – it caused c…
…ircular dependency wpdb::query() -> SqlQueryParser -> DbSchema -> ActivePluginsVPFilesIterator -> wp_get_active_and_valid_plugins() -> wpdb::query()
@JanVoracek JanVoracek [#1036] Minor fix – we need to set dbVersion to the property before i…
…t's used
@JanVoracek JanVoracek [#1036] Removed simple synchronizers. Cache invalidation is now based…
… on `schema.yml`
@JanVoracek JanVoracek [#1036] TermSynchronizer and TermTaxonomySynchronizer replaced with t…
…he generic one
@JanVoracek JanVoracek [#1036] OptionsSynchronizer replaced with the generic one c703a07
@JanVoracek JanVoracek [#1036] Class SynchronizerBase merged with interface Synchronizer as …
…it is the only class implementing it
@JanVoracek JanVoracek [#1036] Ignored entities with natural IDs are now not deleted on `syn…
@JanVoracek JanVoracek [#1036] Refactoring of Synchronizer 75597f4
@JanVoracek JanVoracek [#1036] Synchronizer now handles circular dependencies fdb6992
@JanVoracek JanVoracek [#1036] SynchronizerFactory now always returns an instance of Synchro…
@JanVoracek JanVoracek [#1036] Fixed `restore-site` with dynamic schema loading 17b737e
@JanVoracek JanVoracek [#1036] Synchronization sequence is now calculated dynamically based …
…on references
@JanVoracek JanVoracek [#1036] Removed unnecessary `storageClass` from EntityInfo 6f6c9dc
@JanVoracek JanVoracek [#1036] Fixed parsing VP-Action without `id` part 858c508
@JanVoracek JanVoracek [#1036] Fixed activating using wp-cli command 50f5924
@JanVoracek JanVoracek [#1036] SqlQueryParser now supports tables with composite PK be8a7c4
@JanVoracek JanVoracek [#1036] M:N references (`term_relationships` table) is no longer save…
…d using custom hook and direct manipulation with Mirror. MnReferenceStorage based on `schema.yml` is used instead.
@JanVoracek JanVoracek [#1036] Removed disabled TermTaxonomiesSynchronizerTest (synchronizin…
…g term taxonomies depends on hooks in `hooks.php` that cannot be used in tests)
@JanVoracek JanVoracek [#1036] Removed unused code 169e051
@JanVoracek JanVoracek [#1036] Added doc comments for new classes and unclear parts 3b0b242
@JanVoracek JanVoracek [#1036] VP now saves DDLs for all tracked tables bea0431
@JanVoracek JanVoracek [#1036] DDLs are automatically updated on every plugin/theme/wp updat…
…e, activation, deactivation etc.
@JanVoracek JanVoracek [#1036] "Entity-specific" code from Initializer replaced by a generic…
… solution
@JanVoracek JanVoracek [#1036] Fixed parsing tags from commit message e506f3b
@JanVoracek JanVoracek [#1036] DbSchemaInfo is now refreshed after any change of plugin or t…
…heme. It is necessary for saving plugin's tables in the activation commit / deleting in the deactivation commit.
@JanVoracek JanVoracek [#1036] Plugin may perform some changes in the upload directory (WooC…
…ommerce does)
@JanVoracek JanVoracek [#1036] Update of WP also refreshes DB schema 0feb98b
@JanVoracek JanVoracek [#1036] The `restore-site` now doesn't invoke WP or plugins while res…
@JanVoracek JanVoracek [#1036] action.yml files for all plugins can be now restored from the…
… history
@JanVoracek JanVoracek [#1036] All action.yml files are now restored on clone, restore-site,…
… pull, push, vp update
@JanVoracek JanVoracek [#1036] Plugin's action.yml is now persisted on each action except de…
@JanVoracek JanVoracek [#1036] Fixed persisting actions.yml files on VP activation 47c14d4
@JanVoracek JanVoracek [#1036] Fixed handling table prefixes in options and usermeta 49fdc0b
@JanVoracek JanVoracek [#1036] Fixed code style violations ef6cf31
@JanVoracek JanVoracek [#1036] INSERT ON DUPLICATE KEY UPDATE doesn't support composite PK yet 3c2f020
@JanVoracek JanVoracek [#1036] Updated Plugin-Support docs 376c2bd
@JanVoracek JanVoracek [#1036] Added public filters, actions and functions to Plugin-Support…
… docs
@borekb borekb [#1036] Added `.versionpress` as a PhpStorm source folder 01dfc44
@borekb borekb [#1036] First pass of review. Notable changes:
- Actions moved as the top section
- ... and quite heavily edited
- `.versionpress` folder mentioned more explicitly
- Trying out a new term "plugin descriptor".
@borekb borekb [#1036] Updated example of the expected format for TrackedChangeInfo:…
@borekb borekb [#1036] Made heading structure one level flatter – actions, DB schema…
… etc. are now H2 instead of H3. This allows easier structuring inside them.
@borekb borekb [#1036] Moved content from `` to `` 1538c0e
@borekb borekb [#1036] The schema section edited (relatively smaller changes) 5ae03f8
@borekb borekb [#1036] content moved to 9e6fb98
@borekb borekb [#1036] Shortcodes section edited (relatively minor changes) 0cb4182
@borekb borekb [#1036] Described the discovery mechanism (rough draft) 045c6a7
@borekb borekb [#1036] Added a section about hooks ad8a9c9
@borekb borekb [#1036] Smaller stylistic updates 4de31b7
borekb commented Oct 3, 2016 edited

Here are some notes as I reviewed

  • It would be good to have a clear name for the set of files in the .versionpress folder. "Plugin definition" is the best I can currently think of.
    • Yes let's go with "plugin definition", that's fine.
  • We have been using the edit action since VersionPress 1.0 (e.g., option/edit), however, the more natural English verb is update. 4.0 would be a chance to fix this. (Feedback from native speakers welcome.)
    • We agreed that update is more natural. #1120
  • Do we need the separate _meta_ entity filters, e.g., vp_entity_tags_ vs. vp_meta_entity_tags_? I get that they are slightly different, however, if a single filter name could be used for both cases, it would make the API smaller.
  • Is "schema" the right word? Our schema.yml contains quite a bit more. On the other hand, "schema" is quite unique and intuitively connected to a database so it might be OK to call it that.
    • We could not come up with a better alternative.
  • .versionpress makes sense in the plugin folder but for the planned online repo, we need a convention to separate the plugins. See e.g. DefinitelyTyped.
    • Something like plugin's slug will be used.
  • Our term "actions" conflicts with WP actions. Not be a big deal for us but WP developers unfamiliar with VersionPress might be confused. If needs be, we can consider "events" instead.
    • As nouns the difference between action and event is that action is something done so as to accomplish a purpose while event is an occurrence; something that happens. --

    • In the team, we prefer actions over events (quite obviously as we've used that in the past). Hopefully it's ok for WP devs.
  • Document the slash in Actions
+- Hooks are defined in `hooks.php`
+All files are optional so for example, if a plugin doesn't define any new shortcodes it can omit the `shortcodes.yml` file. Simple plugins like _Hello Dolly_ might even omit everything; they just need to have the `.versionpress` folder so that VersionPress knows the plugin is supported.
octopuss Oct 3, 2016 Contributor

How will VP versioning mechanism handle this Empty folder ? Empty folders are not versioned by default.

JanVoracek Oct 3, 2016 Member

I don't really know. Originally, I didn't intend to use this directory to let VP know if the plugin is supported. It's maybe question to @borekb.

borekb Oct 3, 2016 Member

.gitkeep is an option but it's true that we didn't discuss this yet. The empty .versionpress folder seemed to me like the easiest way for a plugin author to declare explicit support for VersionPress even if no definitions are necessary.

Or maybe it could be a .versionpress file?

JanVoracek Oct 4, 2016 Member

I think it should be some service. The main reason is that you don't have the files before you install the plugin; therefore, you cannot say whether the plugin is supported or not on the /plugin-install.php page.

borekb Oct 4, 2016 Member

Good point. Let's talk about this later, seems like a relative detail.

+### Non-database actions
+Some actions are not directly related to the database entities, e.g. plugin installation, WP update, etc. VersionPress provides function `vp_force_action` for these actions. VersionPress will use only action specified by parameters of this function and ignore all automatically catched. For example:
octopuss Oct 3, 2016 Contributor

Add note that priority is ignored when action is forced.

JanVoracek Oct 3, 2016 Member

Not only the priority – the actions are ignored entirely. Do you have any suggestion how to reword the sentence?

borekb Oct 3, 2016 Member

I wanted to look at this section anyway ("catched" etc.), will try to reword it.

JanVoracek Oct 4, 2016 edited Member

I need an emoji for :facepalm: ... catched 🙄

+ table: posts
+ id: ID
octopuss Oct 3, 2016 Contributor

What about multi column id ?

JanVoracek Oct 3, 2016 Member

VersionPress doesn't support it yet.

- return $this->commitMessage->getUnprefixedSubject();
+ return 0;
octopuss Oct 3, 2016 Contributor

When no better is found, why priority is not high instead ?

JanVoracek Oct 3, 2016 Member

UntrackedChangeInfo represents manual commit; thus, there are no other ChangeInfo objects to compare with. Plus, ignoring the fact it will not compare with anything, zero is the highest priority.

borekb and others added some commits Oct 3, 2016
@borekb borekb [#1036] I've simplified and rearranged the section on actions. Details:
- Removed section about how VersionPress stores actions in commit messages (it didn't bring enough value for its length)
- More clearly described what an action is and what it consists of
- There is now clearer distinction between database and non-database actions
- Filters are just mentioned but their signature is not described
- Some other smaller edits
@JanVoracek JanVoracek Merge branch 'master' into 1036-plugin-support
# Conflicts:
#	plugins/versionpress/composer.lock
@JanVoracek JanVoracek merged commit 0ecf1d2 into master Oct 5, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
@JanVoracek JanVoracek deleted the 1036-plugin-support branch Oct 5, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment