New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom order tables + new orders UI #10071

Closed
mikejolley opened this Issue Jan 14, 2016 · 73 comments

Comments

Projects
None yet
@mikejolley
Member

mikejolley commented Jan 14, 2016

These are the steps I think we need to cover (in sequence) in order to migrate orders to a custom DB table (#9735).

Phase 1: Data Abstraction - 2.x.x

Phase 1 will be to remove any reliance for core and 3rd party plugins on WP functions, namely post and post meta functions. Also (@claudiosmweb) we really need to start consolidating code used in the API/CLI and WP Backend to update orders. Our current codebase is not DRY at all.

  • Introduce wc_get_orders() function to replace get_posts().
  • Introduce meta setter and getter functions to avoid any kind of get_post_meta usage, where needed.
  • Expand CRUD operations for order classes.
  • Replace core usage of get_posts, get_post_meta and direct SQL queries where possible (reports cannot be changed however due to the nature of the complex queries currently required).
  • API CRUD usage for order endpoints.
  • CLI CRUD usage for order endpoints.

Phase 2: UI Refresh - 3.x.x

Next we need to redo the UI to again remove the reliance on WP core. Since we'll have a decent CRUD by this point, we can make use of React/Backbone to make something that performs well and gives a better experience for the store owner.

This will have the side affect of breaking any plugin which adds custom meta boxes, so new hooks and ways of adding custom content will be needed to ensure things are still extensible.

@jameskoster will design new screens to show order data. Without the constraints of WP admin + meta boxes we can do pretty much anything we like here.

  • Replace posts list with custom list table.
  • Replace view/edit order screen with custom UI. This will include new panels for downloadable products to get around the performance issues originality reported in #8589

Phase 3: Data Migration - 3.x.x

  • Create new tables; woocommerce_orders and woocommerce_ordermeta
  • Separate payment status from shipped status. Just throwing this on the list but needs more consideration ( @maxrice @justinstern ) to make statuses more flexible,
  • Create migration script (dedicated page with progress bar, or background operation) to migrate posts to orders.
  • Switch CRUD operations to new tables.
  • Rewrite reports to use new tables. Also a good opportunity to do a refresh of the entire reports section once data is easier to access.

I will create issues for the above points in the relevant milestones, however, I wanted to get feedback prior and ensure I have not missed anything. @pmgarman @thenbrent @maxrice @claudiosmweb @mattyza @allendav @justinshreve @jameskoster .

@mikejolley mikejolley added this to the 3.0 milestone Jan 14, 2016

@rahul286

This comment has been minimized.

Show comment
Hide comment
@rahul286

rahul286 Jan 14, 2016

Wow! :-) 👍

rahul286 commented Jan 14, 2016

Wow! :-) 👍

@claudiosanches

This comment has been minimized.

Show comment
Hide comment
@claudiosanches
Member

claudiosanches commented Jan 14, 2016

😃

@lukecav

This comment has been minimized.

Show comment
Hide comment
@lukecav

lukecav commented Jan 14, 2016

😃

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley

mikejolley Jan 14, 2016

Member

Non-smiley feedback welcome :D

Member

mikejolley commented Jan 14, 2016

Non-smiley feedback welcome :D

@pmgarman

This comment has been minimized.

Show comment
Hide comment
@pmgarman

pmgarman Jan 14, 2016

Contributor

What kind of parameters will wc_get_orders take in an arguments array. My thinking is that what would be taken in while using get_posts would be pretty different from what is taken in on custom tables. Unless the initial version of it while using get_posts was pretty limited and the more advanced parameteters would be added later.

For example: status. If @ 3.0 there becomes a payment and a shipping status, will the wc_get_orders arguments taken a status or go straight to payment_status? Or will status just get deprecated pretty quick in order to move it to payment_status? Obviously it could just stay status but isn't not as obvious when looking at the array of parameters that it is the payment or shipping status.

Contributor

pmgarman commented Jan 14, 2016

What kind of parameters will wc_get_orders take in an arguments array. My thinking is that what would be taken in while using get_posts would be pretty different from what is taken in on custom tables. Unless the initial version of it while using get_posts was pretty limited and the more advanced parameteters would be added later.

For example: status. If @ 3.0 there becomes a payment and a shipping status, will the wc_get_orders arguments taken a status or go straight to payment_status? Or will status just get deprecated pretty quick in order to move it to payment_status? Obviously it could just stay status but isn't not as obvious when looking at the array of parameters that it is the payment or shipping status.

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley

mikejolley Jan 14, 2016

Member

We'd need some of the get_posts args, limit, per page etc + status (we can handle bw compat if we change this in the future), customer_id, etc API has 'status' https://woothemes.github.io/woocommerce-rest-api-docs/#view-list-of-orders

Member

mikejolley commented Jan 14, 2016

We'd need some of the get_posts args, limit, per page etc + status (we can handle bw compat if we change this in the future), customer_id, etc API has 'status' https://woothemes.github.io/woocommerce-rest-api-docs/#view-list-of-orders

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley

mikejolley Jan 14, 2016

Member

@pmgarman did you look at these functions at all - you said you were keen the other day?

Member

mikejolley commented Jan 14, 2016

@pmgarman did you look at these functions at all - you said you were keen the other day?

@pmgarman

This comment has been minimized.

Show comment
Hide comment
@pmgarman

pmgarman Jan 14, 2016

Contributor

I've not yet, I'm working through some unrelated reporting work today, tomorrow/Monday I'm planning to get into some abstraction. Was out of office Tuesday/Wednesday.

Contributor

pmgarman commented Jan 14, 2016

I've not yet, I'm working through some unrelated reporting work today, tomorrow/Monday I'm planning to get into some abstraction. Was out of office Tuesday/Wednesday.

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley

mikejolley Jan 14, 2016

Member

👍 I'll make a branch once I've taken care of 2.5 release

Member

mikejolley commented Jan 14, 2016

👍 I'll make a branch once I've taken care of 2.5 release

@jkudish

This comment has been minimized.

Show comment
Hide comment
@jkudish

jkudish Jan 15, 2016

Contributor

API CRUD usage for order endpoints.
CLI CRUD usage for order endpoints.

The CLI and API Cruds can probably be one and the same, with the CLI just being a wrapper on top of the API.

I'm keen to help out with API and UI/React parts of this.

Contributor

jkudish commented Jan 15, 2016

API CRUD usage for order endpoints.
CLI CRUD usage for order endpoints.

The CLI and API Cruds can probably be one and the same, with the CLI just being a wrapper on top of the API.

I'm keen to help out with API and UI/React parts of this.

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley
Member

mikejolley commented Jan 15, 2016

@thenbrent

This comment has been minimized.

Show comment
Hide comment
@thenbrent

thenbrent Jan 15, 2016

Member

Scary (but exciting) stuff.

The 3 phases sounds like a sane way to tackle it.

What kind of parameters will wc_get_orders take in an arguments array.

For reference, Subscriptions has a wcs_get_subscriptions() method which accepts the following args:

 * 'customer_id' The user ID of a customer on the site.
 * 'product_id' The post ID of a WC_Product object
 * 'subscription_status' Any valid subscription status. Can be 'any', 'active', 'cancelled', 'suspended', 'expired', 'pending' or 'trash'. Defaults to 'any'.
 * 'order_id' The post ID of a shop_order post/WC_Order object which was used to create the subscription
 * 'orderby' The field which the subscriptions should be ordered by. Can be 'start_date', 'trial_end_date', 'end_date', 'status' or 'order_id'. Defaults to 'start_date'.
 * 'order' The order of the values returned. Can be 'ASC' or 'DESC'. Defaults to 'DESC'
 * 'subscriptions_per_page' The number of subscriptions to return. Set to -1 for unlimited. Default 10.
 * 'offset' An optional number of subscription to displace or pass over. Default 0.

Those have been sufficient for our needs, which are obviously more limited than the use cases for wc_get_orders().

You'll also notice we opted for explicit subscription param keys for any subscription data instead of accepting the same param keys get_posts() uses (e.g. subscription_status instead of post_status). But we support the same param keys for query related args, like order and offset. We also extend some get_posts() params, like orderby to accept additional values that relate to a subscription (like ordering by trial_end_date).

I can't say whether this is the best approach for wc_get_orders(), though it works for us pretty well.

This will have the side affect of breaking any plugin which adds custom meta boxes

That doesn't necessarily have to be the case. It will be possible (albeit pretty hacky) to maintain backward compatibility even if using React components for displaying new meta boxes. Something we can look at in more detail when the time comes.

Member

thenbrent commented Jan 15, 2016

Scary (but exciting) stuff.

The 3 phases sounds like a sane way to tackle it.

What kind of parameters will wc_get_orders take in an arguments array.

For reference, Subscriptions has a wcs_get_subscriptions() method which accepts the following args:

 * 'customer_id' The user ID of a customer on the site.
 * 'product_id' The post ID of a WC_Product object
 * 'subscription_status' Any valid subscription status. Can be 'any', 'active', 'cancelled', 'suspended', 'expired', 'pending' or 'trash'. Defaults to 'any'.
 * 'order_id' The post ID of a shop_order post/WC_Order object which was used to create the subscription
 * 'orderby' The field which the subscriptions should be ordered by. Can be 'start_date', 'trial_end_date', 'end_date', 'status' or 'order_id'. Defaults to 'start_date'.
 * 'order' The order of the values returned. Can be 'ASC' or 'DESC'. Defaults to 'DESC'
 * 'subscriptions_per_page' The number of subscriptions to return. Set to -1 for unlimited. Default 10.
 * 'offset' An optional number of subscription to displace or pass over. Default 0.

Those have been sufficient for our needs, which are obviously more limited than the use cases for wc_get_orders().

You'll also notice we opted for explicit subscription param keys for any subscription data instead of accepting the same param keys get_posts() uses (e.g. subscription_status instead of post_status). But we support the same param keys for query related args, like order and offset. We also extend some get_posts() params, like orderby to accept additional values that relate to a subscription (like ordering by trial_end_date).

I can't say whether this is the best approach for wc_get_orders(), though it works for us pretty well.

This will have the side affect of breaking any plugin which adds custom meta boxes

That doesn't necessarily have to be the case. It will be possible (albeit pretty hacky) to maintain backward compatibility even if using React components for displaying new meta boxes. Something we can look at in more detail when the time comes.

@webdados

This comment has been minimized.

Show comment
Hide comment
@webdados

webdados Jan 18, 2016

Contributor

Is this only for orders for now, or are you planning on doing the same for products down the road?

I've got a HUGE website where the products (and the WPML translations of each product) are loaded/updated via a excel file with 40+ fields per product and I use wp_update_post and update_post_meta A LOT!

Contributor

webdados commented Jan 18, 2016

Is this only for orders for now, or are you planning on doing the same for products down the road?

I've got a HUGE website where the products (and the WPML translations of each product) are loaded/updated via a excel file with 40+ fields per product and I use wp_update_post and update_post_meta A LOT!

@claudiosanches

This comment has been minimized.

Show comment
Hide comment
@claudiosanches

claudiosanches Jan 18, 2016

Member

@webdados only orders for now.

Member

claudiosanches commented Jan 18, 2016

@webdados only orders for now.

@webdados

This comment has been minimized.

Show comment
Hide comment
@webdados

webdados Jan 18, 2016

Contributor

@claudiosmweb "for now" :-/

Contributor

webdados commented Jan 18, 2016

@claudiosmweb "for now" :-/

@BuggeringHell

This comment has been minimized.

Show comment
Hide comment
@BuggeringHell

BuggeringHell Jan 19, 2016

This is great for stores with large order volumes, it'd be nice to eventually have the same treatment for products as stores with a large, regularly updated catalog (e.g. CSV imports) would really benefit from it.

BuggeringHell commented Jan 19, 2016

This is great for stores with large order volumes, it'd be nice to eventually have the same treatment for products as stores with a large, regularly updated catalog (e.g. CSV imports) would really benefit from it.

@shivapoudel

This comment has been minimized.

Show comment
Hide comment
@shivapoudel

shivapoudel Jan 19, 2016

Contributor

👍

Contributor

shivapoudel commented Jan 19, 2016

👍

@webdados

This comment has been minimized.

Show comment
Hide comment
@webdados

webdados Jan 19, 2016

Contributor

@BuggeringHell yes, I see it coming along the way. Huge amounts of products is what makes WC websites very slow. Moving them to separate tables would speed things up. I just hope import/export features/API are created. (I know there are tools like WP All Import, but they don't play well with WC+WPML, for example)

Contributor

webdados commented Jan 19, 2016

@BuggeringHell yes, I see it coming along the way. Huge amounts of products is what makes WC websites very slow. Moving them to separate tables would speed things up. I just hope import/export features/API are created. (I know there are tools like WP All Import, but they don't play well with WC+WPML, for example)

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley

mikejolley Jan 19, 2016

Member

Products suit posts tables. We also don't need to run complex queries across products like we do orders. So my vote is to leave products where they are.

Member

mikejolley commented Jan 19, 2016

Products suit posts tables. We also don't need to run complex queries across products like we do orders. So my vote is to leave products where they are.

@BuggeringHell

This comment has been minimized.

Show comment
Hide comment
@BuggeringHell

BuggeringHell Jan 19, 2016

Possibly not the product/post rows itself, but the transients/cache? MySQL gets swamped with queries called from delete_version_transients() located in woocommerce/includes/class-wc-cache-helper.php. Unless I am mistaken, it is called every time a product is updated? With very large product catalogues, imports become impossibly slow.

BuggeringHell commented Jan 19, 2016

Possibly not the product/post rows itself, but the transients/cache? MySQL gets swamped with queries called from delete_version_transients() located in woocommerce/includes/class-wc-cache-helper.php. Unless I am mistaken, it is called every time a product is updated? With very large product catalogues, imports become impossibly slow.

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley

mikejolley Jan 19, 2016

Member

@BuggeringHell read the release post for 2.5. Transients have been optimised.

Member

mikejolley commented Jan 19, 2016

@BuggeringHell read the release post for 2.5. Transients have been optimised.

@BuggeringHell

This comment has been minimized.

Show comment
Hide comment
@BuggeringHell

BuggeringHell Jan 19, 2016

Oh! I recently updated and tried an import to the same result, maybe I should clear all WC transients? My wp_options table hasn't budged much from ~1,200,000 rows.

BuggeringHell commented Jan 19, 2016

Oh! I recently updated and tried an import to the same result, maybe I should clear all WC transients? My wp_options table hasn't budged much from ~1,200,000 rows.

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley

mikejolley Jan 19, 2016

Member

Yeah, flush them all (WP normally does after updates). Sessions are also no longer stored in options table.

Member

mikejolley commented Jan 19, 2016

Yeah, flush them all (WP normally does after updates). Sessions are also no longer stored in options table.

@webdados

This comment has been minimized.

Show comment
Hide comment
@webdados

webdados Jan 19, 2016

Contributor
Contributor

webdados commented Jan 19, 2016

@maxrice

This comment has been minimized.

Show comment
Hide comment
@maxrice

maxrice Jan 19, 2016

Contributor

👍, the sooner we can begin the data abstraction the better.

Separate payment status from shipped status

Absolutely agree with this. I think the way Shopify approaches order statuses is pretty rational and would make for good inspiration:

screen shot 2016-01-19 at 3 00 57 pm

(see Order API docs)

There's an overall order status, then one for financial status and one for fulfillment status.

Contributor

maxrice commented Jan 19, 2016

👍, the sooner we can begin the data abstraction the better.

Separate payment status from shipped status

Absolutely agree with this. I think the way Shopify approaches order statuses is pretty rational and would make for good inspiration:

screen shot 2016-01-19 at 3 00 57 pm

(see Order API docs)

There's an overall order status, then one for financial status and one for fulfillment status.

@toddlahman

This comment has been minimized.

Show comment
Hide comment
@toddlahman

toddlahman Jan 20, 2016

Member

Although wc_get_orders() will be required, it would be a good approach to call an abstract method that allows the WC custom core table to be queried, or third party table to be accessed with a similar structure, but a custom table name, so the same performance gain can be realized by extending WC in third party plugins when a custom table is needed. Anytime the wheel doesn't need to be reinvented in third party plugins, would mean more consistency throughout the ecosystem.

Member

toddlahman commented Jan 20, 2016

Although wc_get_orders() will be required, it would be a good approach to call an abstract method that allows the WC custom core table to be queried, or third party table to be accessed with a similar structure, but a custom table name, so the same performance gain can be realized by extending WC in third party plugins when a custom table is needed. Anytime the wheel doesn't need to be reinvented in third party plugins, would mean more consistency throughout the ecosystem.

@pmgarman

This comment has been minimized.

Show comment
Hide comment
@pmgarman

pmgarman Jan 21, 2016

Contributor

Thanks @maxrice for the notes from Shopify, I think all three of those order statuses would be relevant for WooCommerce orders. Adding a financial_status could mean too in a even further in the future update that partial payments could be supported in WooCommerce even.

Contributor

pmgarman commented Jan 21, 2016

Thanks @maxrice for the notes from Shopify, I think all three of those order statuses would be relevant for WooCommerce orders. Adding a financial_status could mean too in a even further in the future update that partial payments could be supported in WooCommerce even.

@toddlahman

This comment has been minimized.

Show comment
Hide comment
@toddlahman

toddlahman Jan 22, 2016

Member

Partial payments would be awesome! It should be possible with gateways such as Stripe.

Member

toddlahman commented Jan 22, 2016

Partial payments would be awesome! It should be possible with gateways such as Stripe.

@amjad

This comment has been minimized.

Show comment
Hide comment
@amjad

amjad Apr 7, 2016

Why don't we still have "Order Type" as a default feature in WooCommerce?

amjad commented Apr 7, 2016

Why don't we still have "Order Type" as a default feature in WooCommerce?

@thisisbbc

This comment has been minimized.

Show comment
Hide comment
@thisisbbc

thisisbbc May 9, 2016

Really looking forward to see this change. It's been really a pain to work with a staging platform and try to merge everything when new orders were being registered. I wish I wasn't such a noob with php and js to help you guys. 👶 👍

thisisbbc commented May 9, 2016

Really looking forward to see this change. It's been really a pain to work with a staging platform and try to merge everything when new orders were being registered. I wish I wasn't such a noob with php and js to help you guys. 👶 👍

@franticpsyx

This comment has been minimized.

Show comment
Hide comment
@franticpsyx

franticpsyx Jun 24, 2016

Member

Scary (but exciting) stuff.

+1

Member

franticpsyx commented Jun 24, 2016

Scary (but exciting) stuff.

+1

@mikejolley mikejolley modified the milestone: 4.0.0 Mar 17, 2017

@mikejolley mikejolley modified the milestone: 5.0.0 Jun 29, 2017

@mikejolley mikejolley modified the milestones: 5.0.0, 2018 projects Jul 11, 2017

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@javorszky

This comment has been minimized.

Show comment
Hide comment
@javorszky

javorszky Jul 15, 2017

Contributor

@mikejolley that card is on a private board it seems, it's not accessible to me (and assume others)

Contributor

javorszky commented Jul 15, 2017

@mikejolley that card is on a private board it seems, it's not accessible to me (and assume others)

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley

mikejolley Jul 15, 2017

Member

@javorszky It's private for now - just experimenting, I'm working on it :) Wasn't aware it was pinging issues.

Member

mikejolley commented Jul 15, 2017

@javorszky It's private for now - just experimenting, I'm working on it :) Wasn't aware it was pinging issues.

@lukecav

This comment has been minimized.

Show comment
Hide comment
@lukecav

lukecav Sep 20, 2017

@mikejolley Order reports should support CRUD API as well.

lukecav commented Sep 20, 2017

@mikejolley Order reports should support CRUD API as well.

@JJJ

This comment has been minimized.

Show comment
Hide comment
@JJJ

JJJ Oct 27, 2017

Contributor

Looks there are still some UI decisions to make. I’d like to suggest keeping the meta-box API, but being very intentional about the individual box designs.

It takes a bit of extra work to use meta-boxes outside of posts, but is worth it for the ease of use.

See: https://github.com/stuttter/wp-user-profiles

Extending into new sections/boxes basically makes the entire setup infinitely awesome, and still sticks to familiar UI conventions so developers and users continue to feel comfortable during the important transition. If y’all want to React’ify it later, you can, or wait until meta-boxes get their own treatment maybe eventually.

Contributor

JJJ commented Oct 27, 2017

Looks there are still some UI decisions to make. I’d like to suggest keeping the meta-box API, but being very intentional about the individual box designs.

It takes a bit of extra work to use meta-boxes outside of posts, but is worth it for the ease of use.

See: https://github.com/stuttter/wp-user-profiles

Extending into new sections/boxes basically makes the entire setup infinitely awesome, and still sticks to familiar UI conventions so developers and users continue to feel comfortable during the important transition. If y’all want to React’ify it later, you can, or wait until meta-boxes get their own treatment maybe eventually.

@tameemsafi

This comment has been minimized.

Show comment
Hide comment
@tameemsafi

tameemsafi Jan 6, 2018

UPDATE:

Please see: #18378

tameemsafi commented Jan 6, 2018

UPDATE:

Please see: #18378

@pmgarman

This comment has been minimized.

Show comment
Hide comment
@pmgarman

pmgarman Jan 6, 2018

Contributor

That probably would be better opened as a separate issue.

Contributor

pmgarman commented Jan 6, 2018

That probably would be better opened as a separate issue.

@tameemsafi

This comment has been minimized.

Show comment
Hide comment
@tameemsafi

tameemsafi Jan 6, 2018

I have updated my comment above to point to the new issue I created.

tameemsafi commented Jan 6, 2018

I have updated my comment above to point to the new issue I created.

@MTphilclothier

This comment has been minimized.

Show comment
Hide comment
@MTphilclothier

MTphilclothier Jan 19, 2018

I thought this may be useful to reference here for anyone which has not already seen - https://github.com/liquidweb/woocommerce-order-tables

MTphilclothier commented Jan 19, 2018

I thought this may be useful to reference here for anyone which has not already seen - https://github.com/liquidweb/woocommerce-order-tables

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley

mikejolley Jan 30, 2018

Member

So this issue is out of date now so I wanted to come back and give an update on the status of things.

Phase 1 (Data Abstraction) is complete. We have a robust CRUD system now and data stores to handle data read and writes. Adoption of this is growing and the majority of core now users it where possible. 🎆

Phase 2 (UI) is no longer needed. At least, it is no longer needed in order to move to custom tables. Whilst an eventual order screen redesign is inevitable, CRUD and data stores mean we can implement new tables without doing this first. Keeping the 'post' object around is the path of least resistence.

Phase 3 (Data Migration) is reliant on a feature plugin being shipped and tested first. Liquid Web have their own (https://github.com/liquidweb/woocommerce-custom-orders-table) and @pmgarman requested (https://github.com/woocommerce/woocommerce-order-tables-feature-plugin) but has not yet made movement on this.

@stevegrunwell Seems to be a committer on the Liquid Web one. Steve; if would be good if we could communicate on the plans around that plugin to discuss if a core inclusion is even viable. I'm not sure what you had planned for it.

We'd ideally like to get an 'official' feature plugin setup and distributed, and help whomever works on it get it to the point where it can be rolled into core. We've not had any input in the Liquidweb version because of how it was started. I personally don't mind who we work with but it needs to be open and transparent and we need all contributors to be on the same page, including @pmgarman who has been vocal about it for quite some time.

cc @warrendholmes

So thats the current state of play. Our team is working on product custom tables in the meantime and that will be public soon (and hopefully serve as a template for the path from feature plugin -> core inclusion).

Member

mikejolley commented Jan 30, 2018

So this issue is out of date now so I wanted to come back and give an update on the status of things.

Phase 1 (Data Abstraction) is complete. We have a robust CRUD system now and data stores to handle data read and writes. Adoption of this is growing and the majority of core now users it where possible. 🎆

Phase 2 (UI) is no longer needed. At least, it is no longer needed in order to move to custom tables. Whilst an eventual order screen redesign is inevitable, CRUD and data stores mean we can implement new tables without doing this first. Keeping the 'post' object around is the path of least resistence.

Phase 3 (Data Migration) is reliant on a feature plugin being shipped and tested first. Liquid Web have their own (https://github.com/liquidweb/woocommerce-custom-orders-table) and @pmgarman requested (https://github.com/woocommerce/woocommerce-order-tables-feature-plugin) but has not yet made movement on this.

@stevegrunwell Seems to be a committer on the Liquid Web one. Steve; if would be good if we could communicate on the plans around that plugin to discuss if a core inclusion is even viable. I'm not sure what you had planned for it.

We'd ideally like to get an 'official' feature plugin setup and distributed, and help whomever works on it get it to the point where it can be rolled into core. We've not had any input in the Liquidweb version because of how it was started. I personally don't mind who we work with but it needs to be open and transparent and we need all contributors to be on the same page, including @pmgarman who has been vocal about it for quite some time.

cc @warrendholmes

So thats the current state of play. Our team is working on product custom tables in the meantime and that will be public soon (and hopefully serve as a template for the path from feature plugin -> core inclusion).

@mikejolley mikejolley closed this Jan 30, 2018

@stevegrunwell

This comment has been minimized.

Show comment
Hide comment
@stevegrunwell

stevegrunwell Jan 30, 2018

Contributor

I may be able to clarify things a bit, here: @pmgarman and the good folks at Mindsize built the first iteration of the Liquid Web plugin, in cooperation with @bswatson, @boogah, @lukecav, and @chrislema on the Liquid Web side.

As that team got swept up in the preparation for Liquid Web's Managed WooCommerce Hosting launch earlier this month, Lema asked that I spend some time focusing on the custom orders table plugin, especially given my propensity for automated testing. As I dug in, I ended up loading the WooCommerce core test suite as a development dependency, ensuring that "if something works in WooCommerce without the plugin, it should still work when the plugin is active."

The maintainers of WooCommerce may have seen a number of PRs come through from me in the past few weeks; these were (mostly) around adding test coverage for areas that touched order data (and thus may need to interface with the custom order table), enabling me to keep that prime directive of "hey, don't break WooCommerce."

From the time I started working on the plugin, I was aware that this discussion was happening and that parts — if not the entire codebase — may be treated as an "official" feature plugin or even rolled into core. As a result, the plugin in its current state has been written to be compatible with PHP 5.2, adhere to WooCommerce's coding standards, and use the same GPL license as WooCommerce core; whatever our plugin can do to further the cause, we're for it :)

That being said, there are certainly portions of the plugin that would need adjustment before the codebase could be rolled into WooCommerce core. For example, the entire customer data store was introduced as a work-around for some hard-coded $wpdb->postmeta table joins in the WC_Customer_Data_Store class. There are also a number of filter callbacks that are performing surgery on WooCommerce-generated SQL queries, which are well-tested but less-than-ideal; rolling the plugin into core would permit some of these replacements to be handled more cleanly.

Short version: this represents @Mindsize and @liquidweb going "Yeah, a custom orders table is super useful, and here's a way to do it."

Contributor

stevegrunwell commented Jan 30, 2018

I may be able to clarify things a bit, here: @pmgarman and the good folks at Mindsize built the first iteration of the Liquid Web plugin, in cooperation with @bswatson, @boogah, @lukecav, and @chrislema on the Liquid Web side.

As that team got swept up in the preparation for Liquid Web's Managed WooCommerce Hosting launch earlier this month, Lema asked that I spend some time focusing on the custom orders table plugin, especially given my propensity for automated testing. As I dug in, I ended up loading the WooCommerce core test suite as a development dependency, ensuring that "if something works in WooCommerce without the plugin, it should still work when the plugin is active."

The maintainers of WooCommerce may have seen a number of PRs come through from me in the past few weeks; these were (mostly) around adding test coverage for areas that touched order data (and thus may need to interface with the custom order table), enabling me to keep that prime directive of "hey, don't break WooCommerce."

From the time I started working on the plugin, I was aware that this discussion was happening and that parts — if not the entire codebase — may be treated as an "official" feature plugin or even rolled into core. As a result, the plugin in its current state has been written to be compatible with PHP 5.2, adhere to WooCommerce's coding standards, and use the same GPL license as WooCommerce core; whatever our plugin can do to further the cause, we're for it :)

That being said, there are certainly portions of the plugin that would need adjustment before the codebase could be rolled into WooCommerce core. For example, the entire customer data store was introduced as a work-around for some hard-coded $wpdb->postmeta table joins in the WC_Customer_Data_Store class. There are also a number of filter callbacks that are performing surgery on WooCommerce-generated SQL queries, which are well-tested but less-than-ideal; rolling the plugin into core would permit some of these replacements to be handled more cleanly.

Short version: this represents @Mindsize and @liquidweb going "Yeah, a custom orders table is super useful, and here's a way to do it."

@webdados

This comment has been minimized.

Show comment
Hide comment
@webdados

webdados Jan 30, 2018

Contributor

@mikejolley Does this mean that the products will definitely stop being posts, or am I not understanding?
"Our team is working on product custom tables in the meantime and that will be public soon"

Contributor

webdados commented Jan 30, 2018

@mikejolley Does this mean that the products will definitely stop being posts, or am I not understanding?
"Our team is working on product custom tables in the meantime and that will be public soon"

@mikejolley

This comment has been minimized.

Show comment
Hide comment
@mikejolley

mikejolley Jan 30, 2018

Member

@webdados They will be posts - but the meta data and attributes will be in a custom table.

@stevegrunwell Thanks @stevegrunwell. Who'd be best to get together to chat about the best path forward? I know @warrendholmes tried reaching out to some folks.

Member

mikejolley commented Jan 30, 2018

@webdados They will be posts - but the meta data and attributes will be in a custom table.

@stevegrunwell Thanks @stevegrunwell. Who'd be best to get together to chat about the best path forward? I know @warrendholmes tried reaching out to some folks.

@stevegrunwell

This comment has been minimized.

Show comment
Hide comment
@stevegrunwell

stevegrunwell Jan 30, 2018

Contributor

@mikejolley I'm happy to volunteer from the Liquid Web side, since I've been driving development for the last month or so. I can also pull in others from this side as necessary (right now they're busy giving ❤️ s to my earlier comment).

Contributor

stevegrunwell commented Jan 30, 2018

@mikejolley I'm happy to volunteer from the Liquid Web side, since I've been driving development for the last month or so. I can also pull in others from this side as necessary (right now they're busy giving ❤️ s to my earlier comment).

@webdados

This comment has been minimized.

Show comment
Hide comment
@webdados

webdados Jan 30, 2018

Contributor

@mikejolley Nice!

Contributor

webdados commented Jan 30, 2018

@mikejolley Nice!

@surferking

This comment has been minimized.

Show comment
Hide comment
@surferking

surferking Feb 19, 2018

I think some people here and #11467 #11913 #12065 #17241 might find this site and related patch plugins interesting:

800,000 products in Woocommerce searched & filtered in ~0.5s on a $40 Digital Ocean droplet...

Should alleviate some misconceptions on the EAV data-model being an issue for performance too ;)

The only thing we want separate order tables for is dev-ops to make the dev > staging > live process easier.

Performance with correct indexes and queries is only as limited as the hardware with this tuned setup.

surferking commented Feb 19, 2018

I think some people here and #11467 #11913 #12065 #17241 might find this site and related patch plugins interesting:

800,000 products in Woocommerce searched & filtered in ~0.5s on a $40 Digital Ocean droplet...

Should alleviate some misconceptions on the EAV data-model being an issue for performance too ;)

The only thing we want separate order tables for is dev-ops to make the dev > staging > live process easier.

Performance with correct indexes and queries is only as limited as the hardware with this tuned setup.

@pabl0rg

This comment has been minimized.

Show comment
Hide comment
@pabl0rg

pabl0rg Mar 5, 2018

@surferking i've purchased your scalability pro plugin and it causes problems with wp_sessions

Having proper tables for products and orders would be better than hacking around the EAV data-model.

pabl0rg commented Mar 5, 2018

@surferking i've purchased your scalability pro plugin and it causes problems with wp_sessions

Having proper tables for products and orders would be better than hacking around the EAV data-model.

@kevin25

This comment has been minimized.

Show comment
Hide comment
@kevin25

kevin25 Mar 10, 2018

Anh ETA for this, @mikejolley ?

kevin25 commented Mar 10, 2018

Anh ETA for this, @mikejolley ?

@WPprodigy

This comment has been minimized.

Show comment
Hide comment
@WPprodigy
Member

WPprodigy commented Mar 11, 2018

@jonasskafte

This comment has been minimized.

Show comment
Hide comment
@jonasskafte

jonasskafte Mar 13, 2018

Thanks @WPprodigy – but @kevin25, did you mean an ETA for the implementation of a data migration solution, which @stevegrunwell was kindly offering to help with here and which @mikejolley is suggesting to move forward with here? -or am I missing something? Anyway, if so, then I'm also super interested! 😃

jonasskafte commented Mar 13, 2018

Thanks @WPprodigy – but @kevin25, did you mean an ETA for the implementation of a data migration solution, which @stevegrunwell was kindly offering to help with here and which @mikejolley is suggesting to move forward with here? -or am I missing something? Anyway, if so, then I'm also super interested! 😃

@kevin25

This comment has been minimized.

Show comment
Hide comment
@kevin25

kevin25 Mar 16, 2018

@jonasskafte If you don't know right now their plugin doesn't work. I can't even migrate any orders. I need it as the core so it can work stablely.

kevin25 commented Mar 16, 2018

@jonasskafte If you don't know right now their plugin doesn't work. I can't even migrate any orders. I need it as the core so it can work stablely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment