Additional timeline filters #8642
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MelsHyrule
force-pushed
the
timeline-filters
branch
2 times, most recently
from
32f1d13
to
449bd1e
Compare
MelsHyrule
force-pushed
the
timeline-filters
branch
from
03d91fa
to
a53fc60
Compare
MelsHyrule
force-pushed
the
timeline-filters
branch
from
8181799
to
585ad48
Compare
|
Checked commit MelsHyrule@585ad48 with ruby 2.6.10, rubocop 1.28.2, haml-lint 0.35.0, and yamllint app/views/layouts/_tl_show.html.haml |
jrafanie
reviewed
Feb 16, 2023
| // TODO: Different timeline show different data, ensure all necesary data is pulled | ||
| let url = `/api/event_streams?limit=5000&offset=0&expand=resources`; | ||
| url += `&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source`; | ||
| url += `&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message`; | ||
| url += `,vm,ext_management_system`; // TODO heavy stuff to call in, but needed to be displayed in table |
There was a problem hiding this comment.
I think we have host_id already, so I need to expose name as a virtual column so you don't have to check if it's there.
Same for ext_management_system and vm. We can do that in a followup.
3 tasks
MelsHyrule
changed the title
miq-bot
requested review from
DavidResende0,
akhilkr128,
jeffibm,
Fryguy and
jrafanie
jrafanie
approved these changes
Feb 17, 2023
jrafanie
approved these changes
Feb 23, 2023
|
Backported to |
Fryguy
pushed a commit
that referenced
this pull request
Feb 28, 2023
(cherry picked from commit aa9d032)
jrafanie
added a commit
to jrafanie/manageiq
that referenced
this pull request
Mar 9, 2023
See client UI code in: ManageIQ/manageiq-ui-classic#8562 ManageIQ/manageiq-ui-classic#8642 Previously, the UI needed to run an API request including the vm, host, and ext management system and all of their data because sometimes the event stream didn't have a vm, host or ems so the UI code would need to add conditional logic as vm.name, host.name and ext_mangement_system would be empty for some entries and exist for others. See [1]. We can add virtual columns to always return a name or null if the relationship doesn't exist or the name is null. This saves us lots of data if we're only using a single column from each relationship. For a specific API call with proper timestamp filtering so we only return tens of thousands of event_streams: "name": "event_streams", "count": 299925, "subcount": 1000, "subquery_count": 39487, "pages": 40, Before: ``` http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message,vm,ext_management_system&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z 20.07 s 2.59 MB ``` After: ``` http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host_name,source,message,vm_or_template_name,ext_management_system_name&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z (replacing ext_management_system, host, and vm with ext_management_system_name, host_name, and vm_or_template_name) 20.10 s 375.04 KB ``` [1] If you use association.method format: vm_or_template.name, ext_management_system.name, host.name, you are missing sections if they're nil, leading to client code needing to have conditional logic: ``` { "href": "http://localhost:3000/api/event_streams/20409", "id": "20409", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:31Z", "created_on": "2022-10-03T12:35:09Z", "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template": { "name": "vm1" }, "ext_management_system": { "name": "ems1" } }, { "href": "http://localhost:3000/api/event_streams/20410", "id": "20410", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:51Z", "created_on": "2022-10-03T12:35:24Z", "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "ext_management_system": { "name": "ems1" } }, ``` If you use virtual columns, it's flat and easy to use: ``` { "href": "http://localhost:3000/api/event_streams/20409", "id": "20409", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:31Z", "created_on": "2022-10-03T12:35:09Z", "host_name": null, "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template_name": "vm1", "ext_management_system_name": "ems1" }, { "href": "http://localhost:3000/api/event_streams/20410", "id": "20410", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:51Z", "created_on": "2022-10-03T12:35:24Z", "host_name": null, "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template_name": null, "ext_management_system_name": "ems1" }, ```
jrafanie
added a commit
to jrafanie/manageiq
that referenced
this pull request
Mar 9, 2023
See client UI code in: ManageIQ/manageiq-ui-classic#8562 ManageIQ/manageiq-ui-classic#8642 Previously, the UI needed to run an API request including the `vm`, `host`, and `ext management system` and all of their data because sometimes the event stream didn't have a `vm`, `host` or `ems` so the UI code would need to add conditional logic as `vm.name`, `host.name` and `ext_mangement_system.name` would be empty for some entries and exist for others. See [1]. We can add virtual columns to always return a name or null if the relationship doesn't exist or the name is null. This saves us lots of data if we're only using a single column from each relationship. For a specific API call with proper timestamp filtering so we only return tens of thousands of event_streams: "name": "event_streams", "count": 299925, "subcount": 1000, "subquery_count": 39487, "pages": 40, Before: ``` http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message,vm,ext_management_system&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z 20.07 s 2.59 MB ``` After: ``` http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host_name,source,message,vm_or_template_name,ext_management_system_name&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z (replacing ext_management_system, host, and vm with ext_management_system_name, host_name, and vm_or_template_name) 20.10 s 375.04 KB ``` [1] If you use association.method format: vm_or_template.name, ext_management_system.name, host.name, you are missing sections if they're nil, leading to client code needing to have conditional logic: ``` { "href": "http://localhost:3000/api/event_streams/20409", "id": "20409", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:31Z", "created_on": "2022-10-03T12:35:09Z", "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template": { "name": "vm1" }, "ext_management_system": { "name": "ems1" } }, { "href": "http://localhost:3000/api/event_streams/20410", "id": "20410", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:51Z", "created_on": "2022-10-03T12:35:24Z", "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "ext_management_system": { "name": "ems1" } }, ``` If you use virtual columns, it's flat and easy to use: ``` { "href": "http://localhost:3000/api/event_streams/20409", "id": "20409", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:31Z", "created_on": "2022-10-03T12:35:09Z", "host_name": null, "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template_name": "vm1", "ext_management_system_name": "ems1" }, { "href": "http://localhost:3000/api/event_streams/20410", "id": "20410", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:51Z", "created_on": "2022-10-03T12:35:24Z", "host_name": null, "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template_name": null, "ext_management_system_name": "ems1" }, ```
jrafanie
added a commit
to jrafanie/manageiq
that referenced
this pull request
Mar 9, 2023
See client UI code in: ManageIQ/manageiq-ui-classic#8562 ManageIQ/manageiq-ui-classic#8642 This is a followup to ManageIQ#22361 Previously, the UI needed to run an API request including the `vm`, `host`, and `ext management system` and all of their data because sometimes the event stream didn't have a `vm`, `host` or `ems` so the UI code would need to add conditional logic as `vm.name`, `host.name` and `ext_mangement_system.name` would be empty for some entries and exist for others. See [1]. We can add virtual columns to always return a name or null if the relationship doesn't exist or the name is null. This saves us lots of data if we're only using a single column from each relationship. For a specific API call with proper timestamp filtering so we only return tens of thousands of event_streams: "name": "event_streams", "count": 299925, "subcount": 1000, "subquery_count": 39487, "pages": 40, Before: ``` http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message,vm,ext_management_system&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z 20.07 s 2.59 MB ``` After: ``` http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host_name,source,message,vm_or_template_name,ext_management_system_name&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z (replacing ext_management_system, host, and vm with ext_management_system_name, host_name, and vm_or_template_name) 20.10 s 375.04 KB ``` [1] If you use association.method format: vm_or_template.name, ext_management_system.name, host.name, you are missing sections if they're nil, leading to client code needing to have conditional logic: ``` { "href": "http://localhost:3000/api/event_streams/20409", "id": "20409", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:31Z", "created_on": "2022-10-03T12:35:09Z", "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template": { "name": "vm1" }, "ext_management_system": { "name": "ems1" } }, { "href": "http://localhost:3000/api/event_streams/20410", "id": "20410", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:51Z", "created_on": "2022-10-03T12:35:24Z", "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "ext_management_system": { "name": "ems1" } }, ``` If you use virtual columns, it's flat and easy to use: ``` { "href": "http://localhost:3000/api/event_streams/20409", "id": "20409", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:31Z", "created_on": "2022-10-03T12:35:09Z", "host_name": null, "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template_name": "vm1", "ext_management_system_name": "ems1" }, { "href": "http://localhost:3000/api/event_streams/20410", "id": "20410", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:51Z", "created_on": "2022-10-03T12:35:24Z", "host_name": null, "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template_name": null, "ext_management_system_name": "ems1" }, ```
jrafanie
added a commit
to jrafanie/manageiq
that referenced
this pull request
Mar 13, 2023
See client UI code in: ManageIQ/manageiq-ui-classic#8562 ManageIQ/manageiq-ui-classic#8642 This is a followup to ManageIQ#22361 Previously, the UI needed to run an API request including the `vm`, `host`, and `ext management system` and all of their data because sometimes the event stream didn't have a `vm`, `host` or `ems` so the UI code would need to add conditional logic as `vm.name`, `host.name` and `ext_mangement_system.name` would be empty for some entries and exist for others. See [1]. We can add virtual delegates to always return a name or null if the relationship doesn't exist or the name is null. This saves us lots of data if we're only using a single column from each relationship. For a specific API call with proper timestamp filtering so we only return tens of thousands of event_streams: "name": "event_streams", "count": 299925, "subcount": 1000, "subquery_count": 39487, "pages": 40, Before: ``` http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message,vm,ext_management_system&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z 20.07 s 2.59 MB ``` After: ``` http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host_name,source,message,vm_or_template_name,ext_management_system_name&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z (replacing ext_management_system, host, and vm with ext_management_system_name, host_name, and vm_or_template_name) 15.89 s 391 KB ``` [1] If you use association.method format: vm_or_template.name, ext_management_system.name, host.name, you are missing sections if they're nil, leading to client code needing to have conditional logic: ``` { "href": "http://localhost:3000/api/event_streams/20409", "id": "20409", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:31Z", "created_on": "2022-10-03T12:35:09Z", "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template": { "name": "vm1" }, "ext_management_system": { "name": "ems1" } }, { "href": "http://localhost:3000/api/event_streams/20410", "id": "20410", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:51Z", "created_on": "2022-10-03T12:35:24Z", "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "ext_management_system": { "name": "ems1" } }, ``` If you use virtual columns, it's flat and easy to use: ``` { "href": "http://localhost:3000/api/event_streams/20409", "id": "20409", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:31Z", "created_on": "2022-10-03T12:35:09Z", "host_name": null, "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template_name": "vm1", "ext_management_system_name": "ems1" }, { "href": "http://localhost:3000/api/event_streams/20410", "id": "20410", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:51Z", "created_on": "2022-10-03T12:35:24Z", "host_name": null, "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template_name": null, "ext_management_system_name": "ems1" }, ```
GilbertCherrie
pushed a commit
to GilbertCherrie/manageiq
that referenced
this pull request
Jul 7, 2023
See client UI code in: ManageIQ/manageiq-ui-classic#8562 ManageIQ/manageiq-ui-classic#8642 This is a followup to ManageIQ#22361 Previously, the UI needed to run an API request including the `vm`, `host`, and `ext management system` and all of their data because sometimes the event stream didn't have a `vm`, `host` or `ems` so the UI code would need to add conditional logic as `vm.name`, `host.name` and `ext_mangement_system.name` would be empty for some entries and exist for others. See [1]. We can add virtual delegates to always return a name or null if the relationship doesn't exist or the name is null. This saves us lots of data if we're only using a single column from each relationship. For a specific API call with proper timestamp filtering so we only return tens of thousands of event_streams: "name": "event_streams", "count": 299925, "subcount": 1000, "subquery_count": 39487, "pages": 40, Before: ``` http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host,source,message,vm,ext_management_system&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z 20.07 s 2.59 MB ``` After: ``` http://localhost:3000/api/event_streams?limit=5000&offset=0&expand=resources&attributes=group,group_level,group_name,id,event_type,ems_id,type,timestamp,created_on,host_name,source,message,vm_or_template_name,ext_management_system_name&filter[]=ems_id=2&filter[]=type=EmsEvent&filter[]=group=[configuration,other]&filter[]=group_level=[warning,detail,critical]&filter[]=timestamp%3E2022-10-01T22:19:39.148Z&filter[]=timestamp%3C2022-10-5T22:19:39.148Z (replacing ext_management_system, host, and vm with ext_management_system_name, host_name, and vm_or_template_name) 15.89 s 391 KB ``` [1] If you use association.method format: vm_or_template.name, ext_management_system.name, host.name, you are missing sections if they're nil, leading to client code needing to have conditional logic: ``` { "href": "http://localhost:3000/api/event_streams/20409", "id": "20409", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:31Z", "created_on": "2022-10-03T12:35:09Z", "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template": { "name": "vm1" }, "ext_management_system": { "name": "ems1" } }, { "href": "http://localhost:3000/api/event_streams/20410", "id": "20410", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:51Z", "created_on": "2022-10-03T12:35:24Z", "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "ext_management_system": { "name": "ems1" } }, ``` If you use virtual columns, it's flat and easy to use: ``` { "href": "http://localhost:3000/api/event_streams/20409", "id": "20409", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:31Z", "created_on": "2022-10-03T12:35:09Z", "host_name": null, "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template_name": "vm1", "ext_management_system_name": "ems1" }, { "href": "http://localhost:3000/api/event_streams/20410", "id": "20410", "event_type": "INVALID_URI", "ems_id": "2", "type": "EmsEvent", "timestamp": "2022-10-03T13:05:51Z", "created_on": "2022-10-03T12:35:24Z", "host_name": null, "source": "IBM_POWER_HMC", "message": null, "group": "other", "group_level": "detail", "group_name": "Other", "vm_or_template_name": null, "ext_management_system_name": "ems1" }, ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Below we list a series of quality of life improvements made to the Timelines page. This PR is dependent on ManageIQ/manageiq#22361
BEFORE
AFTER
Links in the table to their respective providers / source hosts
Date range validation
Policy Events Result Levels changed
Implements
event_stream_filtersfrom Fix event stream filtering so we can filter by ems event or miq event (policy) manageiq#22361 for more in depth filteringRemoves the "Abandon Changes" pop-up