Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

add user role filter for monitor on top of ad result #275

Merged
merged 5 commits into from
Oct 23, 2020

Conversation

ylwu-amzn
Copy link
Contributor

@ylwu-amzn ylwu-amzn commented Oct 19, 2020

Issue #opendistro-for-elasticsearch/anomaly-detection#195

Description of changes:

AD result index will be system indices, so create AD monitor will fail as common users have no read permission on anomaly result index.

And AD and Alerting will provide document level access control with user backend role user. This PR adds detector backend role checking when create and run monitor.

Backend role checking logic for AD monitor creation:
1.If monitor's user is null, then check if there is detector with null user. For old monitor and detector, they have no user field. With this we can provide backward compatibility.
2.If monitor's user is not null, but has no backend roles, then will check if there is detector with empty backend role.
3.If monitor's user has backend roles, will check if there is detector which backend role has any of the monitor's user backend roles.

The same logic for AD monitor's schedule job run, just change detector existence checking to user role filter query.

Tradeoff
We have no way to verify if user has AD read permission in alerting plugin now. So we have to check user's backend role no matter the backend role filter open or not. If we don't add backend role checking when the filter setting disabled, that will be a security leak: if user has permission to create monitor, they can get any detector's results even they have no AD read permission. We make a balance on security and user experience and choose to prioritize security. In next phase, when secure rest client ready or security plugin provides permission check action, we will add AD read permission check and provide complete user experience.

Test

Add UT and IT cases for AD monitor creation and run.
Have tested by running ./gradlew :alerting:run.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@codecov
Copy link

codecov bot commented Oct 19, 2020

Codecov Report

Merging #275 into master will decrease coverage by 0.14%.
The diff coverage is 88.88%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #275      +/-   ##
============================================
- Coverage     79.85%   79.71%   -0.15%     
- Complexity      194      196       +2     
============================================
  Files           150      151       +1     
  Lines          5132     5195      +63     
  Branches        661      672      +11     
============================================
+ Hits           4098     4141      +43     
- Misses          668      699      +31     
+ Partials        366      355      -11     
Impacted Files Coverage Δ Complexity Δ
...ticsearch/alerting/elasticapi/ElasticExtensions.kt 51.92% <0.00%> (-1.93%) 0.00 <0.00> (ø)
...ting/destination/response/DestinationResponse.java 84.61% <0.00%> (ø) 2.00 <0.00> (ø)
.../alerting/transport/TransportIndexMonitorAction.kt 61.76% <85.71%> (+1.89%) 0.00 <0.00> (ø)
...endistroforelasticsearch/alerting/MonitorRunner.kt 78.00% <87.50%> (+1.68%) 0.00 <0.00> (ø)
...opendistroforelasticsearch/alerting/model/Alert.kt 87.50% <100.00%> (ø) 0.00 <0.00> (ø)
...endistroforelasticsearch/alerting/model/Monitor.kt 88.70% <100.00%> (ø) 0.00 <0.00> (ø)
...icsearch/alerting/model/destination/Destination.kt 75.00% <100.00%> (ø) 0.00 <0.00> (ø)
...asticsearch/alerting/util/AnomalyDetectionUtils.kt 100.00% <100.00%> (ø) 0.00 <0.00> (?)
...alerting/transport/TransportSearchMonitorAction.kt 28.57% <0.00%> (-22.86%) 0.00% <0.00%> (ø%)
...rch/alerting/transport/TransportGetAlertsAction.kt 61.64% <0.00%> (-10.96%) 0.00% <0.00%> (ø%)
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f1b7375...308812d. Read the comment docs.

@@ -323,7 +324,19 @@ class MonitorRunner(
XContentType.JSON.xContent().createParser(xContentRegistry, LoggingDeprecationHandler.INSTANCE, searchSource).use {
searchRequest.source(SearchSourceBuilder.fromXContent(it))
}
val searchResponse: SearchResponse = client.suspendUntil { client.search(searchRequest, it) }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you think of implementing a separate collectInputResults for AD.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change breaks the functionality, needs rework.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I add a separate method to collect anomaly result input.

Copy link

@saratvemulapalli saratvemulapalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me.
Will wait for the tests.

Comment on lines +67 to +68
val nestedQueryBuilder = NestedQueryBuilder(userFieldName, userRolesFilterQuery, ScoreMode.None)
boolQueryBuilder.must(nestedQueryBuilder)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These 2 lines are repeated in all of the different if else cases

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not all branches. line 68 and line 63 looks like duplicate, but if I remove one line of them, I need to add another 2 lines:
Sample code

var mustQuery = null
if (...) {
   mustQuery = userExistsNestedQueryBuilder
} else {
   mustQuery = nestedQueryBuilder
}
boolQueryBuilder.must(mustQuery)

Prefer to keep as is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah ok, didnt realize that. Thanks

Comment on lines 151 to 155
fun addUserRolesFilter(userRoles: List<String>, searchSourceBuilder: SearchSourceBuilder, fieldName: String) {
val filterBackendRoles = QueryBuilders.termsQuery(fieldName, userRoles)
val queryBuilder = searchSourceBuilder.query() as BoolQueryBuilder
searchSourceBuilder.query(queryBuilder.filter(filterBackendRoles))
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do u need to create a new function? If it is because of the function name, I would then rather rename.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is from my previous commit, in that commit, I need this function. Will remove this one to make the change minimal .

Copy link

@saratvemulapalli saratvemulapalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes overall look good to me.
Thanks for the changes.

@@ -323,7 +324,19 @@ class MonitorRunner(
XContentType.JSON.xContent().createParser(xContentRegistry, LoggingDeprecationHandler.INSTANCE, searchSource).use {
searchRequest.source(SearchSourceBuilder.fromXContent(it))
}
val searchResponse: SearchResponse = client.suspendUntil { client.search(searchRequest, it) }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you think of implementing a separate collectInputResults for AD.

@@ -323,7 +324,19 @@ class MonitorRunner(
XContentType.JSON.xContent().createParser(xContentRegistry, LoggingDeprecationHandler.INSTANCE, searchSource).use {
searchRequest.source(SearchSourceBuilder.fromXContent(it))
}
val searchResponse: SearchResponse = client.suspendUntil { client.search(searchRequest, it) }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change breaks the functionality, needs rework.

@ylwu-amzn ylwu-amzn merged commit 866f9c3 into opendistro-for-elasticsearch:master Oct 23, 2020
tlfeng pushed a commit that referenced this pull request Feb 6, 2021
* add user role filter for monitor on top of ad result

* add separate method to collect anomaly result input

* check if user have available detectors when create monitor;add unit test and integration test

* set size 0 when checking detector existence

* rollback changes to add user role filter
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants