New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build out API for creating report sections, tiles, and trend graphs #6236

Closed
pippinsplugins opened this Issue Dec 18, 2017 · 27 comments

Comments

@pippinsplugins
Member

pippinsplugins commented Dec 18, 2017

As part of the 3.0 development push, we're going to create a new API that allows core and extensions to easily create new report options.

Tracking PR Project
#6273 3DD™: Reporting API Reboot

This API will comprise three primary pieces:

Infrastructure

  • #6253 – API to register (any) stuff
  • #6253 – API to handle exceptions
  • #6260 – Concrete class to initialize the Reports API

Report (tabs)

  • #6254 – API to register report tabs (PR: #6263)
  • #6284Report object (PR: #6289)
  • #6284 – API to build and render reports (PRs: #6289, #6329)
  • #6334 – Integrate the Carbon library to simplify date filtering (PR: #6353)
  • #6257 – API to register filters against report tabs (PR: #6361)
  • #6360 – Deprecate old report view/tab display actions using a callback (PR: #6371)
  • #6212 – Audit test coverage for Reports API Reboot pre-merge (PR: #6520)

Endpoints

  • #6280 – API for Endpoints (PR: #6283)
  • #6285 – Add tile support to the Endpoints API (PR: #6313)
  • #6287 – Add chart support to the Endpoints API
  • #6286 – Add table support to the Endpoints API (PR: #6339)
  • #6511 – Add pie chart support to the chart endpoint (PR: #6515)

Nice to Have

  • #6281 – Expose endpoints via REST

The reports screen from AffiliateWP represents a fairly good view of what this API will ultimately provide:

AffiliateWP Reports

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Dec 27, 2017

Member

PR submitted to @mindctrl for review on #6253.

Member

DrewAPicture commented Dec 27, 2017

PR submitted to @mindctrl for review on #6253.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Dec 28, 2017

Member

The empty EDD_Reports class has been merged to issue/6236 via #6260. And so it goes.

Member

DrewAPicture commented Dec 28, 2017

The empty EDD_Reports class has been merged to issue/6236 via #6260. And so it goes.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Dec 29, 2017

Member

It's worth noting that while we're in the process of moving the Logs tab from Reports to Tools, this tab largely comprises graphs that will (likely) be directly integrated into new Reports views, so may actually be able to just completely deprecate the Logs tab altogether at the end of this thing.

Member

DrewAPicture commented Dec 29, 2017

It's worth noting that while we're in the process of moving the Logs tab from Reports to Tools, this tab largely comprises graphs that will (likely) be directly integrated into new Reports views, so may actually be able to just completely deprecate the Logs tab altogether at the end of this thing.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Dec 31, 2017

Member

EDD_Registry was merged via #6253.

Member

DrewAPicture commented Dec 31, 2017

EDD_Registry was merged via #6253.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Dec 31, 2017

Member

The Logs tab was moved to Tools in #6265.

Member

DrewAPicture commented Dec 31, 2017

The Logs tab was moved to Tools in #6265.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Jan 1, 2018

Member

Added PR for the master tracking issue (#6273). This is obviously going to be merged last, but all sub-issues are using issue/6236 as a "master" branch, so this PR can be used to get a 10,000 ft view of progress on the Reports API reboot.

Member

DrewAPicture commented Jan 1, 2018

Added PR for the master tracking issue (#6273). This is obviously going to be merged last, but all sub-issues are using issue/6236 as a "master" branch, so this PR can be used to get a 10,000 ft view of progress on the Reports API reboot.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Jan 1, 2018

Member

Decided to consolidate the effort between #6254 and #6255 into the PR for just #6254.

Member

DrewAPicture commented Jan 1, 2018

Decided to consolidate the effort between #6254 and #6255 into the PR for just #6254.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Jan 8, 2018

Member

Added to-do items for the following:

  • #6280 – Introduce an Endpoint API
  • #6281 – Expose reports endpoints via REST
  • #6284 – Create an API to build and render reports

The REST issue is still a maybe and lower priority than any of the others. But it would be nice to have.

Member

DrewAPicture commented Jan 8, 2018

Added to-do items for the following:

  • #6280 – Introduce an Endpoint API
  • #6281 – Expose reports endpoints via REST
  • #6284 – Create an API to build and render reports

The REST issue is still a maybe and lower priority than any of the others. But it would be nice to have.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Jan 29, 2018

Member

Merged #6329 containing the bulk of the reports rendering logic for #6284.

Member

DrewAPicture commented Jan 29, 2018

Merged #6329 containing the bulk of the reports rendering logic for #6284.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Jan 30, 2018

Member

Merged #6339 to add table endpoint support.

Member

DrewAPicture commented Jan 30, 2018

Merged #6339 to add table endpoint support.

DrewAPicture added a commit that referenced this issue Feb 5, 2018

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Feb 6, 2018

Member

Integrated the Carbon library and introduced a variety of date functions and helpers in #6334.

Member

DrewAPicture commented Feb 6, 2018

Integrated the Carbon library and introduced a variety of date functions and helpers in #6334.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Feb 14, 2018

Member

Merged #6361. to introduce an API for registering and using filters.

Member

DrewAPicture commented Feb 14, 2018

Merged #6361. to introduce an API for registering and using filters.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Apr 10, 2018

Member

Merged the PR for #6287 to introduce chart endpoint support.

Member

DrewAPicture commented Apr 10, 2018

Merged the PR for #6287 to introduce chart endpoint support.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Apr 21, 2018

Member

#6515 was merged for #6511 to add pie chart endpoint support.

Member

DrewAPicture commented Apr 21, 2018

#6515 was merged for #6511 to add pie chart endpoint support.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Apr 26, 2018

Member

#6520 was merged for #6512 bringing this up to pretty much full test coverage!

Member

DrewAPicture commented Apr 26, 2018

#6520 was merged for #6512 bringing this up to pretty much full test coverage!

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Apr 26, 2018

Member

The Reports API Reboot is now feature complete and ready for review! 🎉

Member

DrewAPicture commented Apr 26, 2018

The Reports API Reboot is now feature complete and ready for review! 🎉

@sunnyratilal

This comment has been minimized.

Show comment
Hide comment
@sunnyratilal

sunnyratilal Apr 27, 2018

Member

@easydigitaldownloads/core-team Could we please have some testing done on this PR as it is pivotal for the next phase of 3DD where this new API will be extensively used.

@DrewAPicture has done phenomenal work here.

The Reports page now separates reports by vertical tabs. Each report can consist of any combination of three endpoint types: tiles, tables, and charts.

Please test with this example plugin that Drew created demonstrating how each of the three aforementioned endpoints will work (this will also provide a good example of how all the new APIs are used to create reports): https://gist.github.com/DrewAPicture/69acc0a74af184f11443687802ac89a4

There are two ways reports can be created using the new API: one is the using the way above and the other is by directly the functions: https://gist.github.com/DrewAPicture/d0717133deb5c1c7b569dfdbdd730676. The first requires developers to wrap their code in a try/catch, the second does not. However, both will still throw exceptions if invalid data is passed. Exceptions will automatically be logged to the debug log if Debug Mode is enabled.

To Test

  1. Check Reports page loads all of the existing EDD reports and displays existing charts
  2. Install the Example Report plugin and ensure that the endpoints are displayed correctly.

Note:
Deprecated PHP notices are being thrown and this is intended. These notices will begin to disappear once we recreate the reports using the new APIs.

Member

sunnyratilal commented Apr 27, 2018

@easydigitaldownloads/core-team Could we please have some testing done on this PR as it is pivotal for the next phase of 3DD where this new API will be extensively used.

@DrewAPicture has done phenomenal work here.

The Reports page now separates reports by vertical tabs. Each report can consist of any combination of three endpoint types: tiles, tables, and charts.

Please test with this example plugin that Drew created demonstrating how each of the three aforementioned endpoints will work (this will also provide a good example of how all the new APIs are used to create reports): https://gist.github.com/DrewAPicture/69acc0a74af184f11443687802ac89a4

There are two ways reports can be created using the new API: one is the using the way above and the other is by directly the functions: https://gist.github.com/DrewAPicture/d0717133deb5c1c7b569dfdbdd730676. The first requires developers to wrap their code in a try/catch, the second does not. However, both will still throw exceptions if invalid data is passed. Exceptions will automatically be logged to the debug log if Debug Mode is enabled.

To Test

  1. Check Reports page loads all of the existing EDD reports and displays existing charts
  2. Install the Example Report plugin and ensure that the endpoints are displayed correctly.

Note:
Deprecated PHP notices are being thrown and this is intended. These notices will begin to disappear once we recreate the reports using the new APIs.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Apr 27, 2018

Member

It's worth noting also that this PR is largely a representation of the reports API internals only, that is, I fully expect the implementation team to build outer layers for the registering and configuring of reports, endpoints, and display mechanisms in EDD.

So please try to keep in mind that this really just represents the first part of the reports overhaul – the guts; none of the existing core reports have been converted to use any of the new APIs (the implementation team will do that).

A few functions and hooks have been deprecated, and as @sunnyratilal alluded, will throw expected notices. The styles and UIs and interactions of the elements will likely change once the implementation team gets in there.

Example of expected notices (in lieu of newly implemented core reports):

PHP Notice:  The edd_report_views hook is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use the edd_reports_get_tabs hook instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 735
PHP Notice:  The edd_report_views hook is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use the edd_reports_get_tabs hook instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 735
PHP Notice:  The edd_report_views hook is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use the edd_reports_get_tabs hook instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 735
PHP Notice:  The edd_report_views hook is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use the edd_reports_get_tabs hook instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 735
PHP Notice:  The edd_reports_view_earnings hook is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use the \EDD\Reports\add_report hook instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 735
PHP Notice:  edd_get_report_dates is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use \EDD\Reports\get_dates_filter instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 535
PHP Notice:   in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 536
PHP Notice:  edd_get_reporting_view is <strong>deprecated</strong> since Easy Digital Downloads version 3.0 with no alternative available. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 539
PHP Notice:   in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 540

The fact that the core reports still display is merely an indicator that backward compatibility has been taken into account for display purposes only.

Member

DrewAPicture commented Apr 27, 2018

It's worth noting also that this PR is largely a representation of the reports API internals only, that is, I fully expect the implementation team to build outer layers for the registering and configuring of reports, endpoints, and display mechanisms in EDD.

So please try to keep in mind that this really just represents the first part of the reports overhaul – the guts; none of the existing core reports have been converted to use any of the new APIs (the implementation team will do that).

A few functions and hooks have been deprecated, and as @sunnyratilal alluded, will throw expected notices. The styles and UIs and interactions of the elements will likely change once the implementation team gets in there.

Example of expected notices (in lieu of newly implemented core reports):

PHP Notice:  The edd_report_views hook is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use the edd_reports_get_tabs hook instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 735
PHP Notice:  The edd_report_views hook is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use the edd_reports_get_tabs hook instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 735
PHP Notice:  The edd_report_views hook is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use the edd_reports_get_tabs hook instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 735
PHP Notice:  The edd_report_views hook is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use the edd_reports_get_tabs hook instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 735
PHP Notice:  The edd_reports_view_earnings hook is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use the \EDD\Reports\add_report hook instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 735
PHP Notice:  edd_get_report_dates is <strong>deprecated</strong> since Easy Digital Downloads version 3.0! Use \EDD\Reports\get_dates_filter instead. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 535
PHP Notice:   in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 536
PHP Notice:  edd_get_reporting_view is <strong>deprecated</strong> since Easy Digital Downloads version 3.0 with no alternative available. in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 539
PHP Notice:   in /path/to/wp/wp-content/plugins/easy-digital-downloads/includes/misc-functions.php on line 540

The fact that the core reports still display is merely an indicator that backward compatibility has been taken into account for display purposes only.

DrewAPicture added a commit that referenced this issue Apr 27, 2018

Restore the Earnings report to its pre-3.0 glory. It was changed duri…
…ng the development process for #6236 for testing purposes.
@pippinsplugins

This comment has been minimized.

Show comment
Hide comment
@pippinsplugins

pippinsplugins Apr 27, 2018

Member

@DrewAPicture do I need to do anything specific to support date filters? I see the date filters in the existing and new reports (using the example report provided) but they do not function. Is the example missing a key piece or is that a bug?

Member

pippinsplugins commented Apr 27, 2018

@DrewAPicture do I need to do anything specific to support date filters? I see the date filters in the existing and new reports (using the example report provided) but they do not function. Is the example missing a key piece or is that a bug?

@spencerfinnell

This comment has been minimized.

Show comment
Hide comment
@spencerfinnell

spencerfinnell Apr 27, 2018

Member

Noticed some tabs are outputting multiple date selectors. This causes the following JS to fail:

// Show hide extended date options
$( '#edd-graphs-date-options' ).change( function() {
var $this = $(this),
date_range_options = $( '#edd-date-range-options' );
if ( 'other' === $this.val() ) {
date_range_options.removeClass( 'screen-reader-text' );
} else {
date_range_options.addClass( 'screen-reader-text' );
}
});

The multiple IDs on the page are invalid (technically) and causes only the first instance of the HTML to use the JS. The ID should be switched to a class selector if multiple will appear on the page.

Class selectors are slower but it would require looping through all instances of the ID otherwise, which is probably just as slow.

Also I think the previously used .show() and .hide() functions make more sense here as when the items are hidden they are not relevant to screen readers either.

Something like:

// toggle extended date options
$( '.edd-graphs-date-options' ).change( function() {
  var $this = $( this );
  var date_range_options = $this.closest( '.edd-date-range-options' );

  return date_range_options.toggle( 'other' === $this.val() );
} );
Member

spencerfinnell commented Apr 27, 2018

Noticed some tabs are outputting multiple date selectors. This causes the following JS to fail:

// Show hide extended date options
$( '#edd-graphs-date-options' ).change( function() {
var $this = $(this),
date_range_options = $( '#edd-date-range-options' );
if ( 'other' === $this.val() ) {
date_range_options.removeClass( 'screen-reader-text' );
} else {
date_range_options.addClass( 'screen-reader-text' );
}
});

The multiple IDs on the page are invalid (technically) and causes only the first instance of the HTML to use the JS. The ID should be switched to a class selector if multiple will appear on the page.

Class selectors are slower but it would require looping through all instances of the ID otherwise, which is probably just as slow.

Also I think the previously used .show() and .hide() functions make more sense here as when the items are hidden they are not relevant to screen readers either.

Something like:

// toggle extended date options
$( '.edd-graphs-date-options' ).change( function() {
  var $this = $( this );
  var date_range_options = $this.closest( '.edd-date-range-options' );

  return date_range_options.toggle( 'other' === $this.val() );
} );
@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Apr 27, 2018

Member

Just demonstrated the date filter saving state and adjusting a chart endpoint in person for @pippinsplugins so maybe something weird with his setup isn't working.

@spencerfinnell Noted. Probably can just switch to a class and each it.

Member

DrewAPicture commented Apr 27, 2018

Just demonstrated the date filter saving state and adjusting a chart endpoint in person for @pippinsplugins so maybe something weird with his setup isn't working.

@spencerfinnell Noted. Probably can just switch to a class and each it.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture Apr 29, 2018

Member

Also I think the previously used .show() and .hide() functions make more sense here as when the items are hidden they are not relevant to screen readers either.

@spencerfinnell If they're UI elements, they're relevant. display:none completely hides the elements and unless you notify screen readers when you're showing and hiding elements with JS through something like wp.a11y.speak() every time, then those visual interactions are pretty much a complete mystery to screen reader users. Describing those fields via an aria-describedby or legend would probably be useful, regardless.

Member

DrewAPicture commented Apr 29, 2018

Also I think the previously used .show() and .hide() functions make more sense here as when the items are hidden they are not relevant to screen readers either.

@spencerfinnell If they're UI elements, they're relevant. display:none completely hides the elements and unless you notify screen readers when you're showing and hiding elements with JS through something like wp.a11y.speak() every time, then those visual interactions are pretty much a complete mystery to screen reader users. Describing those fields via an aria-describedby or legend would probably be useful, regardless.

DrewAPicture added a commit that referenced this issue May 1, 2018

@DrewAPicture DrewAPicture removed the Has PR label May 1, 2018

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture May 1, 2018

Member

Merged #6273 to bring in the entirety of the new Reports API internals 🎉

New issues for anything that comes up in implementation.

Member

DrewAPicture commented May 1, 2018

Merged #6273 to bring in the entirety of the new Reports API internals 🎉

New issues for anything that comes up in implementation.

3DD™: Reporting API Reboot automation moved this from In Progress to Done May 1, 2018

Release 3.0 Sprint automation moved this from Reports API Reboot to Done May 1, 2018

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture May 1, 2018

Member

Example report:

Example report

Member

DrewAPicture commented May 1, 2018

Example report:

Example report

@spencerfinnell

This comment has been minimized.

Show comment
Hide comment
@spencerfinnell

spencerfinnell May 1, 2018

Member

@DrewAPicture

If they're UI elements, they're relevant. display:none completely hides the elements and unless you notify screen readers when you're showing and hiding elements with JS through something like wp.a11y.speak() every time, then those visual interactions are pretty much a complete mystery to screen reader users. Describing those fields via an aria-describedby or legend would probably be useful, regardless.

But the fields are being hidden because they are not relevant to the other filters (and cannot be modified by mouse, keyboard, or voice). Having them visible to screen readers when using a range other than "Custom" would cause readers to announce them when they are not needed and are not able to be modified.

I'm not sure the best way to announce when they have appeared -- probably wp.a11y.speak() like you said.

Member

spencerfinnell commented May 1, 2018

@DrewAPicture

If they're UI elements, they're relevant. display:none completely hides the elements and unless you notify screen readers when you're showing and hiding elements with JS through something like wp.a11y.speak() every time, then those visual interactions are pretty much a complete mystery to screen reader users. Describing those fields via an aria-describedby or legend would probably be useful, regardless.

But the fields are being hidden because they are not relevant to the other filters (and cannot be modified by mouse, keyboard, or voice). Having them visible to screen readers when using a range other than "Custom" would cause readers to announce them when they are not needed and are not able to be modified.

I'm not sure the best way to announce when they have appeared -- probably wp.a11y.speak() like you said.

@DrewAPicture

This comment has been minimized.

Show comment
Hide comment
@DrewAPicture

DrewAPicture May 8, 2018

Member

Overview

The Reports API Reboot represents the next higher level of modularization for reports. The entire idea of this reboot is to break the data represented in a "report" into logical chunks, make it possible to build new or rebuild old reports using those chunks, and refresh how those chunks can be visualized

Think of the idea of a new "report" as a way to containerize data points (endpoints) in a structured way. A report contains endpoints, which each contain view definitions, and sometimes those view definitions contain datasets and manifests. Basically speaking, the reports API reboot takes what is currently hard-coded, modularizes it, and sets up a mechanism through which data is retrieved just in time for display.

Getting Registered

EDD 3.0 introduces the concept of a "registry". Items get added to a registry (in memory) and then can be retrieved globally for various purposes.

And there can be any number of independent registries created. The new Reports API introduces two registries, one each for reports and endpoints that will be consumed by those registered reports, respectively.

Moving forward, it's important to draw a distinction between what we think of as a "reports tab" and a report, because a "reports tab" represents the visualization of a "report" which is now a structured data object. More on that below.

Reports and endpoint registration are handled separately for reasons that will be explained later, but the main thing to consider is that reports and endpoints are added to registries to kill two birds with one stone:

  1. Get all of the needed data together in one place for global access
  2. Handle error checking and validation on the way in

The reason error checking and validation happens during registration, is that the registry data needs to be in a very specific format when it is later pulled out and built into structured data objects. If there's a problem on the way in, there is a whole bunch of internal logic in place to not only check for errors, but report those exceptions to the debug log (yay making support easier!).

Great pains have been taken to ensure that errors at any level of the API can be logged and tracked down if debug mode is turned on.

Reports and Endpoints and Views, oh my!

Let's talk a little bit more about terminology for a minute. For the purposes of the new reports API, a "report" is a structured data object representing all of the data that might be displayed with it. An "endpoint" is a structured data object leveraged by a report to display a piece of data. A view defines one or more ways an endpoint can be visualized.

To see this in action, check this sample plugin.

Here, two reports are registered in the register_report() method, each defining endpoints they would like to surface via several different "view groups", i.e. tiles, charts, or tables.

In the register_endpoints() method, endpoints are registered. Notice that each endpoint defines views through which that particular data point can be visualized. Each view definition contains its own display and data callbacks.

In comparing the report and endpoint registrations, notice the correlation between the view groups in the 'endpoints' attribute of the report and the view definitions in the 'views' attribute of the endpoint. tile:tiles as chart:charts as table:tables.

One Endpoint, Multiple Views

Basically, any one endpoint can register view definitions for any valid view type, making the data reusable in multiple contexts and even multiple different reports, thus why reports and endpoints are registered separately.

For example, earnings data could be expressed as as a tile (single piece of data), a table (data table), and/or a chart (pie/doughnut chart or line/bar graph). An endpoint simply needs to register at least one view definition to be valid.

Reports leverage view groups to determine whether registered endpoints contain compatible view definitions. If an endpoint registers 'tile' and 'table' views and a report calls for that endpoint using the 'charts' group, it's going to fail silently.

Structured Data

When data in the reports and endpoints registries is later retrieved, API functionality builds that data into structured data objects.

Just as you could conceivably hit a REST endpoint and get back JSON data in the form of a response, Report and Endpoint objects represent data, but in a much richer format than just raw data: the response is a fully-qualified data object decorated with helper methods and error checking.

The API doesn't currently contain actual REST endpoints, but it's built in a way that making that possible in the future would be fairly straightforward. See #6281.

Here is a basic rundown of what happens when a reports tab is loaded

  1. A report tab starts to display
  2. It calls for a specific, registered report
  3. The report is retrieved from the registry and built into a Report object
  4. At this time, all endpoints whitelisted for the report are looped through and Endpoint objects are built for each of them, as long as they contain valid view definitions and don't trigger any errors. If one or both conditions are not met, errors are logged to the Report object and that specific Endpoint object is not built.
  5. Any whitelisted, valid filters are displayed
  6. The report display callback loops through those rich endpoint objects and triggers their display methods
  7. At this time, the data callbacks are finally fired, just in time for display – more on this below
  8. The report is displayed with all of its endpoints.

If you note in step 7 above, the data callbacks don't get called until the Endpoint object display callbacks are called. This is what makes the new reports API so much faster: All of the data in the registry is static, and all of the data in the Endpoints is static right up until it's time to display it and then it's retrieved just in time.


This is part one of a series of abstracts explaining how the new reports API works. More to come.

Member

DrewAPicture commented May 8, 2018

Overview

The Reports API Reboot represents the next higher level of modularization for reports. The entire idea of this reboot is to break the data represented in a "report" into logical chunks, make it possible to build new or rebuild old reports using those chunks, and refresh how those chunks can be visualized

Think of the idea of a new "report" as a way to containerize data points (endpoints) in a structured way. A report contains endpoints, which each contain view definitions, and sometimes those view definitions contain datasets and manifests. Basically speaking, the reports API reboot takes what is currently hard-coded, modularizes it, and sets up a mechanism through which data is retrieved just in time for display.

Getting Registered

EDD 3.0 introduces the concept of a "registry". Items get added to a registry (in memory) and then can be retrieved globally for various purposes.

And there can be any number of independent registries created. The new Reports API introduces two registries, one each for reports and endpoints that will be consumed by those registered reports, respectively.

Moving forward, it's important to draw a distinction between what we think of as a "reports tab" and a report, because a "reports tab" represents the visualization of a "report" which is now a structured data object. More on that below.

Reports and endpoint registration are handled separately for reasons that will be explained later, but the main thing to consider is that reports and endpoints are added to registries to kill two birds with one stone:

  1. Get all of the needed data together in one place for global access
  2. Handle error checking and validation on the way in

The reason error checking and validation happens during registration, is that the registry data needs to be in a very specific format when it is later pulled out and built into structured data objects. If there's a problem on the way in, there is a whole bunch of internal logic in place to not only check for errors, but report those exceptions to the debug log (yay making support easier!).

Great pains have been taken to ensure that errors at any level of the API can be logged and tracked down if debug mode is turned on.

Reports and Endpoints and Views, oh my!

Let's talk a little bit more about terminology for a minute. For the purposes of the new reports API, a "report" is a structured data object representing all of the data that might be displayed with it. An "endpoint" is a structured data object leveraged by a report to display a piece of data. A view defines one or more ways an endpoint can be visualized.

To see this in action, check this sample plugin.

Here, two reports are registered in the register_report() method, each defining endpoints they would like to surface via several different "view groups", i.e. tiles, charts, or tables.

In the register_endpoints() method, endpoints are registered. Notice that each endpoint defines views through which that particular data point can be visualized. Each view definition contains its own display and data callbacks.

In comparing the report and endpoint registrations, notice the correlation between the view groups in the 'endpoints' attribute of the report and the view definitions in the 'views' attribute of the endpoint. tile:tiles as chart:charts as table:tables.

One Endpoint, Multiple Views

Basically, any one endpoint can register view definitions for any valid view type, making the data reusable in multiple contexts and even multiple different reports, thus why reports and endpoints are registered separately.

For example, earnings data could be expressed as as a tile (single piece of data), a table (data table), and/or a chart (pie/doughnut chart or line/bar graph). An endpoint simply needs to register at least one view definition to be valid.

Reports leverage view groups to determine whether registered endpoints contain compatible view definitions. If an endpoint registers 'tile' and 'table' views and a report calls for that endpoint using the 'charts' group, it's going to fail silently.

Structured Data

When data in the reports and endpoints registries is later retrieved, API functionality builds that data into structured data objects.

Just as you could conceivably hit a REST endpoint and get back JSON data in the form of a response, Report and Endpoint objects represent data, but in a much richer format than just raw data: the response is a fully-qualified data object decorated with helper methods and error checking.

The API doesn't currently contain actual REST endpoints, but it's built in a way that making that possible in the future would be fairly straightforward. See #6281.

Here is a basic rundown of what happens when a reports tab is loaded

  1. A report tab starts to display
  2. It calls for a specific, registered report
  3. The report is retrieved from the registry and built into a Report object
  4. At this time, all endpoints whitelisted for the report are looped through and Endpoint objects are built for each of them, as long as they contain valid view definitions and don't trigger any errors. If one or both conditions are not met, errors are logged to the Report object and that specific Endpoint object is not built.
  5. Any whitelisted, valid filters are displayed
  6. The report display callback loops through those rich endpoint objects and triggers their display methods
  7. At this time, the data callbacks are finally fired, just in time for display – more on this below
  8. The report is displayed with all of its endpoints.

If you note in step 7 above, the data callbacks don't get called until the Endpoint object display callbacks are called. This is what makes the new reports API so much faster: All of the data in the registry is static, and all of the data in the Endpoints is static right up until it's time to display it and then it's retrieved just in time.


This is part one of a series of abstracts explaining how the new reports API works. More to come.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment