Store all order statuses

[joshuatf]

Fixes #1251

This PR:

• Stores all order statuses
• Adds an array of excluded statuses
• Filters all products, coupons, categories, taxes, variations, and orders by the default (non-excluded) statuses if no status query is provided

Detailed test instructions:

1. Make sure that you have regenerated lookup tables and that the status column is filled in wc_order_stats.
2. Create requests to each of the endpoints.
3. Check that stats returned included stats from all orders excluding the default excluded statuses (refunded, pending, failed, cancelled).
4. Make a request to the /wp-json/wc/v3/reports/orders/stats endpoint and check that status_is and status_is_not query params override default excluded statuses (e.g., you can still query by failed order statuses).
5. Check that all phpunit tests are passing.

[jeffstieler]

In testing the /reports/categories endpoint - I'm seeing 165 results with status_is = 'failed'. There are only 33 rows in my wc_order_stats table with a status of wc-failed.. is this correct?

[jeffstieler]

In testing the /reports/categories endpoint - I'm seeing 165 results with status_is = 'failed'. There are only 33 rows in my wc_order_stats table with a status of wc-failed.. is this correct?

[jeffstieler]

Why str_replace() here? I think it would be simpler to just define the column query here instead of modifying an incomplete/incorrect definition.

[joshuatf]

It would be simpler, but I think it would cause more confusion when debugging and means we are defining this value twice instead of filtering it.

For example, if we redefine this variable entirely and someone changes the value in $report_columns instead of the construct method, no errors occur, but the code does not work as intended.

On the other hand, if we filter the value and the user changes the $report_columns value from:
'orders_count' => 'COUNT(DISTINCT order_id) as orders_count' to
'orders_count' => 'COUNT(order_id) as orders_count'
then the function continues to work as expected. And if we filter a wrong value like 'orders_count' => 'COUNT(DISTINCT order_id) as order_id_alias', we at least get an error message.

Neither one of these are very ideal solutions though, so I've opened another issue to address how we deal with ambiguous columns here. If it's okay with you, we can deal with that in another PR.

[jeffstieler]

Same question about str_replace() here.

[jeffstieler]

Same question about str_replace() here.

[jeffstieler]

Optional: there's an opportunity to avoid some copy-pasta here and combine this with the if() statement above it.

[joshuatf]

Good idea. Updated in 930ab80

[jeffstieler]

Same question about str_replace().

[jeffstieler]

Same question about str_replace().

[jeffstieler]

Same question about str_replace().

[jeffstieler]

Same question about str_replace().

[jeffstieler]

Is there an opportunity to DRY this up? Perhaps a base "joins on wc_order_stats" class that handles the from_clause, where_clause, normalize_order_by(), and the orders_count column?

[joshuatf]

Hmm, I'm not sure another class would be the best solution here. Report data stores already extend from WC_Admin_Reports_Data_Store, but quite a few also override the normalize_order_by() and other methods.

The bigger issue here is that there are about 3-4 different patterns being used in these data stores to append query parameters which makes it difficult to create a universal class that does not result in overriding in most data stores. For example, here are a few different variations just for the "where" clause:

$sql_query_params['where_clause'] .= " AND ( {$order_status_filter} )";

$where_subclause = implode( " $operator ", array_filter( array( $where_subclause, $order_status_filter ) ) );

$products_where_clause .= " AND ( {$order_status_filter} )";

$where_filters[] = $order_status_filter;

$sql_query_params['where_clause'] .= " AND ( {$order_status_filter} )";
$sql_query_params['where_clause'] .= ' AND variation_id > 0';

It seems like we could make a call outside of and adjacent to the get_sql_query_params() that might work for most data stores, but that would also be semantically incorrect and we'd be introducing more patterns that may come back to haunt us later.

I do agree that it would be nice if this were dryer, but it seems like a fairly large refactor that might be outside the scope of this PR. If we decided to do something like that, it might be worth breaking into bite-size chunks (like just refactoring get_sql_query_params() to use the same patterns) to create more manageable PRs.

[joshuatf]

@jeffstieler Thanks for taking the time to review.

I'm seeing 165 results with status_is = 'failed'. There are only 33 rows in my wc_order_stats table with a status of wc-failed.. is this correct?

That's very possible if each of those failed 33 orders had a couple categories with a few products, but it could also be a bug. It's hard to say without seeing the data, but feel free to drop me a DB dump file and I can help investigate if you'd like.

[jeffstieler]

That's very possible if each of those failed 33 orders had a couple categories with a few products, but it could also be a bug. It's hard to say without seeing the data, but feel free to drop me a DB dump file and I can help investigate if you'd like.

I double checked the report data and it all looks good. 👍