METRON-1790: Unsubscribe from every observable in the pcap panel UI component #1208
Conversation
|
Checked and tested. Looks good to me. +1 |
There was a problem hiding this comment.
Thanks @ruffle1986, just a few questions to help me understand what are perhaps corner cases.
| this.startPolling(id); | ||
| } | ||
| }, (error: any) => { | ||
| this.errorMsg = `Response message: ${error.message}. Something went wrong with your query submission!`; |
There was a problem hiding this comment.
Why do we not need to unsubscribe here like we do for 'statusSubscription' and 'cancelSubscription'?
this.subscriptions['submitSubscription'].unsubscribe();
There was a problem hiding this comment.
In case of statusSubscription, we have to unsubscribe because we want to stop polling the server if an error occurs.
But the other observables are one-time observables which means once they complete (or throw an error) we won't get any additional event until we subscribe again. So in theory, if would be very precise, we unsubscribed from the observables in case of fulfilling not just in case of errors. But it would lead to code that is hard to reason about.
So yes, it's a nice catch. It's a bit more verbose to unsubscribe from the cancelSubscription because it's a one-time observable too. It's not bad, but it's imperative and the goal of using observables to write less code following a declarative approach.
Changing the way how you think in order to get the most out of observables is hard. It's hard even for me. Here we perform a lot of http requests using observables because we need it to meet the business requirements (feature) and that's angular way right? But we're doing it wrong. When it comes to multiple observables in one component, we should compose them somehow together in order the take advantage of reactive programming (RxJs). But you have to be very careful with composing observables because it's really hard to get it done or even maintain it later or read and understand what the code intends to do.
Long story short, when an http observable fulfils, it's not necessary to unsubscribe. There I can remove the unsubscribe from the cancel event if an error occurs in order to avoid confusion.
The goal of this PR was to cancel the actual http requests when the user navigates somewhere else. If users are not interested in the pcap panel anymore, we don't need to wait for the server's answer. It's silly to wait for the answer for a pcap request when you're on the alerts page.
I'm going to remove the unsubscribe from the cancel observable to avoid confusion.
I highly recommend to read this article by Ben Lesh (lead dev of the RxJs team) about it:
https://medium.com/@benlesh/rxjs-dont-unsubscribe-6753ed4fda87
There was a problem hiding this comment.
The goal of this PR was to cancel the actual http requests when the user navigates somewhere else. If users are not interested in the pcap panel anymore, we don't need to wait for the server's answer. It's silly to wait for the answer for a pcap request when you're on the alerts page.
What happens if I submit a pcap request, get bored and navigate back to Alerts, then navigate back to Pcap to check on my job status? Will I be able to 'see' my job finish and view the pcap?
There was a problem hiding this comment.
Yes because it's already implemented. Every time the pcap panel is rendered (ngOnInit ) we ask the server whether we have a running job in the background. If so, we reattach the job on the UI and keep pooling the server until the job gets done. But it has nothing to do with what we're doing here in this PR.
| this.queryRunning = false; | ||
| this.pcapService.getPackets(this.queryId, this.pagination.selectedPage).toPromise().then(pdml => { | ||
| this.subscriptions['packetSubscription'] = this.pcapService.getPackets(this.queryId, this.pagination.selectedPage).subscribe(pdml => { | ||
| this.pdml = pdml; | ||
| }, (error: RestError) => { |
There was a problem hiding this comment.
Why do we not need to unsubscribe here like we do for 'statusSubscription' and 'cancelSubscription'?
this.subscriptions['packetSubscription'].unsubscribe();
|
|
||
| constructor(private pcapService: PcapService) { } | ||
|
|
||
| ngOnInit() { | ||
| this.pcapRequest = new PcapRequest(); | ||
| this.pcapService.getRunningJob().subscribe((statusResponses: PcapStatusResponse[]) => { | ||
| this.subscriptions['runningJobSubscription'] = this.pcapService.getRunningJob().subscribe((statusResponses: PcapStatusResponse[]) => { |
There was a problem hiding this comment.
Why do we never see an unsubscribe for 'runningJobSubscription'?
There was a problem hiding this comment.
I collect every subscription in an object and I go through all of them in the unsubscribeAll method.
ruffle1986
force-pushed
the
METRON-1790
branch
from
e1bfcab
to
b2fd1b9
Compare
|
@nickwallen The concept here is observable is a stream of events. The basic rule is to unsubscribe when you no longer want to know about new values from a particular stream. (RxJs/Angular handling observables like streams, even if we know that there will be just one value. It's highly recommended to ignore that.) Actually, you are right with your concerns. The original implementation here gives no chance to handle events in a reactive way. In my opinion, our real streams here are the following:
And we only want to unsubscribe from these streams when the user navigates away to the alert tab. Unfortunately, the implementation creates new observables on every click instead of observing the stream of clicks. This leads to the fact that we unsubscribing and resubscribing all the time. Also makes this code hard to understand and explain any change or small improvement. However, IMHO unsubscribe implemented in this PR is an improvement, still not the perfect state of this code part. |
|
Thanks for the explanation @tiborm . @ruffle1986 - Can you create an Apache JIRA describing the next phase of this work as described by @tiborm? Pending that JIRA, I am +1 on this. Many thanks. |
|
Thanks, Nick. I added the two tickets to our Jira to track the refactorings described above. PCAP UI: Implementing proper reactive event handling in PCAP |
|
Looks good. Many thanks for the effort. Will merge in now. |
Contributor Comments
Original Jira ticket: https://issues.apache.org/jira/browse/METRON-1790
Motivation:
There are a lot of http requests performed in the pcap panel ui component and we just unsubscribe from some of them when the component is no longer rendered on the screen. It could cause memory consumption issues. Because of the active subscriptions, the garbage collector is not able to remove these objects from the memory, however they're not needed anymore.
There's another benefit of unsubscribing from these http calls. If the user leaves the pcap tab but there are pending requests, the unsubscribe method cancels the active xhrs immediately so it won't wait for fulfilment unnecessarily.
I would also refactor that part when we convert an observable to a promise. I would keep it as an observable. By doing this, we would be able to unsubscribe from it as well in the destructor method. Promises are not cancelable.
https://github.com/apache/metron/blob/master/metron-interface/metron-alerts/src/app/pcap/pcap-panel/pcap-panel.component.ts#L70
Resources:
https://angular.io/guide/lifecycle-hooks#ondestroy
Changes
getPacketsobservable as is. RemovetoPromise()Testing
Go to the PCAP tab and start querying or just let the component figure if there are any running jobs (The latter starts an http request immediately when the component is rendered). But before making any http requests, open up the network tab and make sure you throttled the traffic by setting the connection to "Slow 3G" (top right corner) to let the request be in pending state for a while before it fulfils. By this, you get some extra time to change between the PCAP and the Alerts tab. If you change between the tabs and there are any http requests in pending state, they're going to be cancelled and highlighted in red color.
Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:
For code changes:
Have you included steps to reproduce the behavior or problem that is being changed or addressed?
Have you included steps or a guide to how the change may be verified and tested manually?
Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
Have you written or updated unit tests and or integration tests to verify your changes?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
For documentation related changes:
Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via
site-book/target/site/index.html:Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.