Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

React Timeline Chart and Table Component #8562

Merged
merged 1 commit into from Feb 10, 2023

Conversation

MelsHyrule
Copy link
Member

@MelsHyrule MelsHyrule commented Dec 7, 2022

Continuation of the React conversion for the timelines page. Part 1 of the page conversion can be found here #8496

This PR depends on ManageIQ/manageiq#22304 (or it's replacement ManageIQ/manageiq#22305) for optimization of the Event Streams API call.

Removed @pf3/timeline and old D3 code (https://github.com/ManageIQ/manageiq-ui-classic/blob/master/app/javascript/oldjs/miq_timeline.js) part of the update to jQuery3 #5585 (comment)

BEFORE

Screen Shot 2023-01-27 at 11 24 02 AM

AFTER

Screen Shot 2023-01-13 at 1 05 22 PM

Post Notes:

Useful links for timeline-chart dispatch events
Removed `@pf3/timeline` in terms of `d3`
  • Relevant PR Move remaining components in react-ui-components to ui-classic #7670
  • we need d3, no getting around it, however the VERSION is what is interesting. It looks like patternfly and pf3/timeline depended on old versions of d3 (specifically 3.X) BUT the carbon-charts needed the newer version of d3 to work (5.X).
  • To resolve this issue it looks like kavya made the alias of d3-5.X (in its own commit which made it very convenient to find 7b6f233) so that carbon-charts had their dependencies imported at the right version
  • But now that pf3/timeline is gone so thats one less use of d3 v3.X
  • patternfly still imports d3 v3.X tho
"patternfly@npm:~3.31.1":
  version: 3.31.2
  resolution: "patternfly@npm:3.31.2"
  dependencies:
    ...
    d3: ~3.5.17
    ...

so we will see in the yarn.lock multiple imports which i dont see any problem with, this is just the natural behavior of package imports

"d3@npm:^7.8.2":
  version: 7.8.2
  resolution: "d3@npm:7.8.2"
  dependencies:
    ...
  checksum: e7bf5918f2a97d0c0cc489d64348b323446aa824c32b65f0888846c26f80ed05dbee18a4b7c9a487b3e60d9d404e4672d169573fe63439e3e53abd5812bcd766
  languageName: node
  linkType: hard

"d3@npm:~3.5.0, d3@npm:~3.5.17":
  version: 3.5.17
  resolution: "d3@npm:3.5.17"
  checksum: 79b59275c36fa35a0af55f326bb56d65e14d71d44e96cbd05af767000af63c3ab8b3f479e2c25281afff7ba3f5858e272ded081e5d33807b005f217c4bfbd82d
  languageName: node
  linkType: hard

Next Steps:

Improve API call #8642

  • Have page additionally filter content by (for example) the page's selected Host or VM/Template
  • To do this we need to be able to distinguish what page the timeline is being called from

Improve Table

  • Once we have the improved api call we can dynamically change the table content to better suit the page it's displaying from

Apply Reviews

@MelsHyrule MelsHyrule requested a review from a team as a code owner December 7, 2022 19:32
@MelsHyrule MelsHyrule mentioned this pull request Dec 16, 2022
@MelsHyrule MelsHyrule force-pushed the custom-timeline branch 2 times, most recently from e6bcec9 to f765753 Compare January 17, 2023 15:03
@MelsHyrule MelsHyrule force-pushed the custom-timeline branch 2 times, most recently from 0267fbe to 26e68d9 Compare January 27, 2023 17:07
@MelsHyrule MelsHyrule changed the title [WIP] React Timeline Component React Timeline Chart and Table Component Jan 27, 2023
@miq-bot miq-bot removed the wip label Jan 27, 2023
@MelsHyrule MelsHyrule force-pushed the custom-timeline branch 2 times, most recently from 1e21e88 to c5073e5 Compare January 27, 2023 17:53
@MelsHyrule
Copy link
Member Author

MelsHyrule commented Jan 27, 2023

@miq-bot add-reviewer @DavidResende0
@miq-bot add-reviewer @akhilkr128
@miq-bot add-reviewer @jeffibm
@miq-bot add-reviewer @Fryguy
@miq-bot assign @jeffibm
@miq-bot add-label enhancement
cc @jrafanie @agrare

Opening this PR up for review. Any additional updates or changes I'd want to do on this i've listed under Next Steps in the description. This PR is already quite big so I think it would be good to split them off into their own PRs or make an issue and track them there.

@Fryguy Fryguy moved this from Petrosian to In progress in Roadmap Feb 10, 2023
@Fryguy
Copy link
Member

Fryguy commented Feb 10, 2023

@MelsHyrule Does this PR drop d3? I don't see the package.json nor yarn.lock changing, unless I'm misreading?

@MelsHyrule
Copy link
Member Author

@Fryguy Ah i removed the referenced in the code but not in package.json, will remove now, thanks for catching that

@jrafanie
Copy link
Member

jrafanie commented Feb 10, 2023

FIXED

6. After clicking on a timeline event, it brings you to a new screen with nothing on it and going back doesn't bring you back to the timeline...

screenshot

screencast 2023-02-10 13-11-49

@@ -43,7 +43,7 @@ const smartAndOrStatements = (group, array) => {
export const buildUrl = (values) => {
// TODO: Different timeline show different data, ensure all necesary data is pulled
let url = `/api/event_streams?limit=5000&offset=0&expand=resources`;
url += `&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host.name,source`;
url += `&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed you removed host.name - won't that make it a lot more expensive if it's returning the entire host object?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that was causing the issue 4) above. That was removed and I added issue 7) to add the proper ems_id/host_id filter[] as we're not filtering properly right now.

Copy link
Member

@jrafanie jrafanie Feb 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of either a virtual column or something that would make it more reliable as the host.name sometimes comes back and others it doesn't from the same API request. It makes the logic needlessly complex in the UI.

Example API response without host.name:

        {
            "href": "http://localhost:3000/api/event_streams/48428",
            "id": "48428",
            "event_type": "MODIFY_URI",
            "ems_id": "2",
            "type": "EmsEvent",
            "timestamp": "2022-10-05T04:00:31Z",
            "created_on": "2022-10-05T03:29:55Z",
            "source": "IBM_POWER_HMC",
            "host_id": "1",
            "group": "configuration",
            "group_level": "detail",
            "group_name": "Configuration/Reconfiguration"
        },

and with a host.name(from the same API request):


        {
            "href": "http://localhost:3000/api/event_streams/67912",
            "id": "67912",
            "event_type": "MODIFY_URI",
            "ems_id": "2",
            "type": "EmsEvent",
            "timestamp": "2022-10-06T09:58:54Z",
            "created_on": "2022-10-06T09:28:19Z",
            "source": "IBM_POWER_HMC",
            "host_id": "7",
            "group": "configuration",
            "group_level": "detail",
            "group_name": "Configuration/Reconfiguration",
            "host": {
                "name": "porthos"
            }
        },

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've opened this PR to add virtual columns to address this problem: ManageIQ/manageiq#22398


window.moment = require('moment');
require('moment-strftime');
require('moment-timezone');
require('moment-duration-format')(window.moment);

require('@pf3/timeline');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌

@@ -65,7 +64,7 @@
"codemirror": "~5.58.2",
"connected-react-router": "~6.7.0",
"create-react-context": "~0.3.0",
"d3-7.6.1": "npm:d3@^7.6.1",
"d3": "^7.8.2",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wish we could comment that this is only needed for @carbon/charts

@jrafanie
Copy link
Member

  1. Followup to issue 4). We're not specifying the host_id/ems_id as an API filter[] parameter based on the current screen we're on. In other words, we're showing all host and ems events for the other criteria.
Details
[----] I, [2023-02-10T17:31:22.457137 #37554:620ac]  INFO -- : Started GET "/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source&filter[]=type=EmsEvent&filter[]=group=other&filter[]=group_level=[critical,detail,warning]&filter[]=timestamp%3E2022-10-05T04:00:00.000Z&filter[]=timestamp%3C2022-10-10T04:00:00.000Z" for ::1 at 2023-02-10 17:31:22 -0500

@miq-bot
Copy link
Member

miq-bot commented Feb 10, 2023

Checked commit MelsHyrule@bca12e3 with ruby 2.6.10, rubocop 1.28.2, haml-lint 0.35.0, and yamllint
2 files checked, 0 offenses detected
Everything looks fine. 🏆

@jrafanie
Copy link
Member

jrafanie commented Feb 10, 2023

Sent in chat:

ok, so the remaining TBD is:
1 - enable/disable timeline button for host/ems if they have no events
3 - select all instead of selecting events group/levels one by one
5 - date picker / auto adjust start/end date when the other is set (same month at least)
7 - add host_id, ems_id API filter[] based on the current screen… host_id for a host screen, ems_id for ems screen

See above for details on each. We can fix each separately in followup PRs.

EDIT:

It's ODD that @MelsHyrule only fixes even issues.

@jrafanie
Copy link
Member

image

@MelsHyrule
Copy link
Member Author

Well you see @jrafanie they're a PRIME opportunity

Copy link
Member

@jrafanie jrafanie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Fryguy
Copy link
Member

Fryguy commented Feb 10, 2023

What happens in other screens like VM timelines? Do we also get non-vm events there? Or does that fall into the same concern about the emas_id/host_id

@jrafanie
Copy link
Member

What happens in other screens like VM timelines? Do we also get non-vm events there? Or does that fall into the same concern about the emas_id/host_id

Yes, it's the same thing

@jrafanie jrafanie assigned jrafanie and unassigned jeffibm Feb 10, 2023
@jrafanie jrafanie merged commit 2c8a098 into ManageIQ:master Feb 10, 2023
@Fryguy Fryguy moved this from In progress to Petrosian in Roadmap Feb 28, 2023
jrafanie added a commit to jrafanie/manageiq that referenced this pull request Mar 9, 2023
See client UI code in:
ManageIQ/manageiq-ui-classic#8562
ManageIQ/manageiq-ui-classic#8642

Previously, the UI needed to run an API request including the vm, host, and ext
management system and all of their data because sometimes the event stream didn't
have a vm, host or ems so the UI code would need to add conditional logic as
vm.name, host.name and ext_mangement_system would be empty for some entries and
exist for others. See [1].

We can add virtual columns to always return a name or null if the relationship
doesn't exist or the name is null.  This saves us lots of data if we're only
using a single column from each relationship.

For a specific API call with proper timestamp filtering so we only return tens
of thousands of event_streams:

"name": "event_streams",
"count": 299925,
"subcount": 1000,
"subquery_count": 39487,
"pages": 40,

Before:

```
http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message,vm,ext_management_system&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z

20.07 s
2.59 MB
```

After:

```
http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host_name,source,message,vm_or_template_name,ext_management_system_name&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z

(replacing ext_management_system, host, and vm with ext_management_system_name,
host_name, and vm_or_template_name)

20.10 s
375.04 KB
```

[1] If you use association.method format: vm_or_template.name, ext_management_system.name, host.name, you are missing sections if they're nil, leading to client code needing to have conditional logic:

```
{
    "href": "http://localhost:3000/api/event_streams/20409",
    "id": "20409",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:31Z",
    "created_on": "2022-10-03T12:35:09Z",
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template": {
        "name": "vm1"
    },
    "ext_management_system": {
        "name": "ems1"
    }
},
{
    "href": "http://localhost:3000/api/event_streams/20410",
    "id": "20410",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:51Z",
    "created_on": "2022-10-03T12:35:24Z",
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "ext_management_system": {
        "name": "ems1"
    }
},
```

If you use virtual columns, it's flat and easy to use:

```
{
    "href": "http://localhost:3000/api/event_streams/20409",
    "id": "20409",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:31Z",
    "created_on": "2022-10-03T12:35:09Z",
    "host_name": null,
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template_name": "vm1",
    "ext_management_system_name": "ems1"
},
{
    "href": "http://localhost:3000/api/event_streams/20410",
    "id": "20410",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:51Z",
    "created_on": "2022-10-03T12:35:24Z",
    "host_name": null,
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template_name": null,
    "ext_management_system_name": "ems1"
},
```
jrafanie added a commit to jrafanie/manageiq that referenced this pull request Mar 9, 2023
See client UI code in:
ManageIQ/manageiq-ui-classic#8562
ManageIQ/manageiq-ui-classic#8642

Previously, the UI needed to run an API request including the `vm`, `host`, and
`ext management system` and all of their data because sometimes the event
stream didn't have a `vm`, `host` or `ems` so the UI code would need to add
conditional logic as `vm.name`, `host.name` and `ext_mangement_system.name`
would be empty for some entries and exist for others. See [1].

We can add virtual columns to always return a name or null if the relationship
doesn't exist or the name is null.  This saves us lots of data if we're only
using a single column from each relationship.

For a specific API call with proper timestamp filtering so we only return tens
of thousands of event_streams:

"name": "event_streams",
"count": 299925,
"subcount": 1000,
"subquery_count": 39487,
"pages": 40,

Before:

```
http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message,vm,ext_management_system&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z

20.07 s
2.59 MB
```

After:

```
http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host_name,source,message,vm_or_template_name,ext_management_system_name&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z

(replacing ext_management_system, host, and vm with ext_management_system_name,
host_name, and vm_or_template_name)

20.10 s
375.04 KB
```

[1] If you use association.method format: vm_or_template.name,
ext_management_system.name, host.name, you are missing sections if they're nil,
leading to client code needing to have conditional logic:

```
{
    "href": "http://localhost:3000/api/event_streams/20409",
    "id": "20409",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:31Z",
    "created_on": "2022-10-03T12:35:09Z",
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template": {
        "name": "vm1"
    },
    "ext_management_system": {
        "name": "ems1"
    }
},
{
    "href": "http://localhost:3000/api/event_streams/20410",
    "id": "20410",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:51Z",
    "created_on": "2022-10-03T12:35:24Z",
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "ext_management_system": {
        "name": "ems1"
    }
},
```

If you use virtual columns, it's flat and easy to use:

```
{
    "href": "http://localhost:3000/api/event_streams/20409",
    "id": "20409",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:31Z",
    "created_on": "2022-10-03T12:35:09Z",
    "host_name": null,
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template_name": "vm1",
    "ext_management_system_name": "ems1"
},
{
    "href": "http://localhost:3000/api/event_streams/20410",
    "id": "20410",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:51Z",
    "created_on": "2022-10-03T12:35:24Z",
    "host_name": null,
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template_name": null,
    "ext_management_system_name": "ems1"
},
```
jrafanie added a commit to jrafanie/manageiq that referenced this pull request Mar 9, 2023
See client UI code in:
ManageIQ/manageiq-ui-classic#8562
ManageIQ/manageiq-ui-classic#8642

This is a followup to ManageIQ#22361

Previously, the UI needed to run an API request including the `vm`, `host`, and
`ext management system` and all of their data because sometimes the event
stream didn't have a `vm`, `host` or `ems` so the UI code would need to add
conditional logic as `vm.name`, `host.name` and `ext_mangement_system.name`
would be empty for some entries and exist for others. See [1].

We can add virtual columns to always return a name or null if the relationship
doesn't exist or the name is null.  This saves us lots of data if we're only
using a single column from each relationship.

For a specific API call with proper timestamp filtering so we only return tens
of thousands of event_streams:

"name": "event_streams",
"count": 299925,
"subcount": 1000,
"subquery_count": 39487,
"pages": 40,

Before:

```
http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message,vm,ext_management_system&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z

20.07 s
2.59 MB
```

After:

```
http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host_name,source,message,vm_or_template_name,ext_management_system_name&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z

(replacing ext_management_system, host, and vm with ext_management_system_name,
host_name, and vm_or_template_name)

20.10 s
375.04 KB
```

[1] If you use association.method format: vm_or_template.name,
ext_management_system.name, host.name, you are missing sections if they're nil,
leading to client code needing to have conditional logic:

```
{
    "href": "http://localhost:3000/api/event_streams/20409",
    "id": "20409",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:31Z",
    "created_on": "2022-10-03T12:35:09Z",
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template": {
        "name": "vm1"
    },
    "ext_management_system": {
        "name": "ems1"
    }
},
{
    "href": "http://localhost:3000/api/event_streams/20410",
    "id": "20410",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:51Z",
    "created_on": "2022-10-03T12:35:24Z",
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "ext_management_system": {
        "name": "ems1"
    }
},
```

If you use virtual columns, it's flat and easy to use:

```
{
    "href": "http://localhost:3000/api/event_streams/20409",
    "id": "20409",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:31Z",
    "created_on": "2022-10-03T12:35:09Z",
    "host_name": null,
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template_name": "vm1",
    "ext_management_system_name": "ems1"
},
{
    "href": "http://localhost:3000/api/event_streams/20410",
    "id": "20410",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:51Z",
    "created_on": "2022-10-03T12:35:24Z",
    "host_name": null,
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template_name": null,
    "ext_management_system_name": "ems1"
},
```
jrafanie added a commit to jrafanie/manageiq that referenced this pull request Mar 13, 2023
See client UI code in:
ManageIQ/manageiq-ui-classic#8562
ManageIQ/manageiq-ui-classic#8642

This is a followup to ManageIQ#22361

Previously, the UI needed to run an API request including the `vm`, `host`, and
`ext management system` and all of their data because sometimes the event
stream didn't have a `vm`, `host` or `ems` so the UI code would need to add
conditional logic as `vm.name`, `host.name` and `ext_mangement_system.name`
would be empty for some entries and exist for others. See [1].

We can add virtual delegates to always return a name or null if the relationship
doesn't exist or the name is null.  This saves us lots of data if we're only
using a single column from each relationship.

For a specific API call with proper timestamp filtering so we only return tens
of thousands of event_streams:

"name": "event_streams",
"count": 299925,
"subcount": 1000,
"subquery_count": 39487,
"pages": 40,

Before:

```
http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message,vm,ext_management_system&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z

20.07 s
2.59 MB
```

After:

```
http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host_name,source,message,vm_or_template_name,ext_management_system_name&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z

(replacing ext_management_system, host, and vm with ext_management_system_name,
host_name, and vm_or_template_name)

15.89 s
391 KB
```

[1] If you use association.method format: vm_or_template.name,
ext_management_system.name, host.name, you are missing sections if they're nil,
leading to client code needing to have conditional logic:

```
{
    "href": "http://localhost:3000/api/event_streams/20409",
    "id": "20409",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:31Z",
    "created_on": "2022-10-03T12:35:09Z",
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template": {
        "name": "vm1"
    },
    "ext_management_system": {
        "name": "ems1"
    }
},
{
    "href": "http://localhost:3000/api/event_streams/20410",
    "id": "20410",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:51Z",
    "created_on": "2022-10-03T12:35:24Z",
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "ext_management_system": {
        "name": "ems1"
    }
},
```

If you use virtual columns, it's flat and easy to use:

```
{
    "href": "http://localhost:3000/api/event_streams/20409",
    "id": "20409",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:31Z",
    "created_on": "2022-10-03T12:35:09Z",
    "host_name": null,
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template_name": "vm1",
    "ext_management_system_name": "ems1"
},
{
    "href": "http://localhost:3000/api/event_streams/20410",
    "id": "20410",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:51Z",
    "created_on": "2022-10-03T12:35:24Z",
    "host_name": null,
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template_name": null,
    "ext_management_system_name": "ems1"
},
```
GilbertCherrie pushed a commit to GilbertCherrie/manageiq that referenced this pull request Jul 7, 2023
See client UI code in:
ManageIQ/manageiq-ui-classic#8562
ManageIQ/manageiq-ui-classic#8642

This is a followup to ManageIQ#22361

Previously, the UI needed to run an API request including the `vm`, `host`, and
`ext management system` and all of their data because sometimes the event
stream didn't have a `vm`, `host` or `ems` so the UI code would need to add
conditional logic as `vm.name`, `host.name` and `ext_mangement_system.name`
would be empty for some entries and exist for others. See [1].

We can add virtual delegates to always return a name or null if the relationship
doesn't exist or the name is null.  This saves us lots of data if we're only
using a single column from each relationship.

For a specific API call with proper timestamp filtering so we only return tens
of thousands of event_streams:

"name": "event_streams",
"count": 299925,
"subcount": 1000,
"subquery_count": 39487,
"pages": 40,

Before:

```
http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message,vm,ext_management_system&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z

20.07 s
2.59 MB
```

After:

```
http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host_name,source,message,vm_or_template_name,ext_management_system_name&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z

(replacing ext_management_system, host, and vm with ext_management_system_name,
host_name, and vm_or_template_name)

15.89 s
391 KB
```

[1] If you use association.method format: vm_or_template.name,
ext_management_system.name, host.name, you are missing sections if they're nil,
leading to client code needing to have conditional logic:

```
{
    "href": "http://localhost:3000/api/event_streams/20409",
    "id": "20409",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:31Z",
    "created_on": "2022-10-03T12:35:09Z",
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template": {
        "name": "vm1"
    },
    "ext_management_system": {
        "name": "ems1"
    }
},
{
    "href": "http://localhost:3000/api/event_streams/20410",
    "id": "20410",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:51Z",
    "created_on": "2022-10-03T12:35:24Z",
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "ext_management_system": {
        "name": "ems1"
    }
},
```

If you use virtual columns, it's flat and easy to use:

```
{
    "href": "http://localhost:3000/api/event_streams/20409",
    "id": "20409",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:31Z",
    "created_on": "2022-10-03T12:35:09Z",
    "host_name": null,
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template_name": "vm1",
    "ext_management_system_name": "ems1"
},
{
    "href": "http://localhost:3000/api/event_streams/20410",
    "id": "20410",
    "event_type": "INVALID_URI",
    "ems_id": "2",
    "type": "EmsEvent",
    "timestamp": "2022-10-03T13:05:51Z",
    "created_on": "2022-10-03T12:35:24Z",
    "host_name": null,
    "source": "IBM_POWER_HMC",
    "message": null,
    "group": "other",
    "group_level": "detail",
    "group_name": "Other",
    "vm_or_template_name": null,
    "ext_management_system_name": "ems1"
},
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Roadmap
  
Petrosian
Development

Successfully merging this pull request may close these issues.

None yet

5 participants