Fix ioc store config mappings #1087
Conversation
Signed-off-by: Joanne Wang <jowg@amazon.com>
jowg-amazon
requested review from
amsiglan,
AWSHurneyt,
getsaurabh02,
lezzago,
praveensameneni,
sbcd90,
eirsep,
engechas,
goyamegh and
riysaxen-amzn
as code owners
| @@ -115,8 +115,11 @@ public void indexIocs(List<STIX2IOC> iocs) throws IOException { | |||
| String feedIndexName = initFeedIndex(saTifSourceConfig.getId()); | |||
|
|
|||
| // Add the created index to the IocStoreConfig | |||
| ((DefaultIocStoreConfig) saTifSourceConfig.getIocStoreConfig()).getIocMapStore().putIfAbsent(saTifSourceConfig.getId(), new ArrayList<>()); | |||
| ((DefaultIocStoreConfig) saTifSourceConfig.getIocStoreConfig()).getIocMapStore().get(saTifSourceConfig.getId()).add(feedIndexName); | |||
| saTifSourceConfig.getIocTypes().forEach(type -> { | |||
There was a problem hiding this comment.
Not blocking, but we should probably considering refactoring the IocTypes list in the config data model to store a list of the IOCType enums instead of Strings.
https://github.com/opensearch-project/security-analytics-commons/blob/main/tif/src/main/java/org/opensearch/securityanalytics/commons/model/IOCType.java#L10
| @@ -25,7 +25,7 @@ public class DefaultIocStoreConfig extends IocStoreConfig implements Writeable, | |||
| public static final String DEFAULT_FIELD = "default"; | |||
| public static final String IOC_MAP = "ioc_map"; | |||
|
|
|||
| // Maps the SATIFSourceConfig ID to the list of index/alias names | |||
| // Maps the IOCs to the list of index/alias names | |||
There was a problem hiding this comment.
Would this be "IOC types" to the index/alias names?
There was a problem hiding this comment.
yes fixing comment to include ioc types
Signed-off-by: Joanne Wang <jowg@amazon.com>
| // // wait for job runner to run | ||
| // waitUntil(() -> { | ||
| // try { | ||
| // return verifyJobRan(createdId, firstUpdatedTime); |
There was a problem hiding this comment.
Not blocking, but could you add a javadoc comment to the verifyJobRan function to help clarify its use?
Signed-off-by: Joanne Wang <jowg@amazon.com>
| @@ -115,8 +115,11 @@ public void indexIocs(List<STIX2IOC> iocs) throws IOException { | |||
| String feedIndexName = initFeedIndex(saTifSourceConfig.getId()); | |||
|
|
|||
| // Add the created index to the IocStoreConfig | |||
| ((DefaultIocStoreConfig) saTifSourceConfig.getIocStoreConfig()).getIocMapStore().putIfAbsent(saTifSourceConfig.getId(), new ArrayList<>()); | |||
| ((DefaultIocStoreConfig) saTifSourceConfig.getIocStoreConfig()).getIocMapStore().get(saTifSourceConfig.getId()).add(feedIndexName); | |||
| saTifSourceConfig.getIocTypes().forEach(type -> { | |||
There was a problem hiding this comment.
@jowg-amazon A quick way to resolve the bug where the index name is being added each time the feed is refreshed would be to add the following add this following check around the forEach loop.
if (!feedIndexExists(feedIndexName)) {
...
}
There was a problem hiding this comment.
Would this always be true since we are doing initFeedIndex first?
There was a problem hiding this comment.
Actually, it would be better to refactor the initFeedIndex function to take in the index name, and a boolean "exists" variable; and replace the call to feedIndexExists in the init function with the boolean variable.
There was a problem hiding this comment.
Refactored method so that it checks if the index exists first, if it does not exist then it will initialize the index and add it to the ioc map
Signed-off-by: Joanne Wang <jowg@amazon.com>
jowg-amazon
merged commit c28f84f
into
opensearch-project:feature/threat_intel
* fix mappings Signed-off-by: Joanne Wang <jowg@amazon.com> * comment Signed-off-by: Joanne Wang <jowg@amazon.com> * fix comment Signed-off-by: Joanne Wang <jowg@amazon.com> * added java doc and todo Signed-off-by: Joanne Wang <jowg@amazon.com> * remove duplicate index names from mapping Signed-off-by: Joanne Wang <jowg@amazon.com> --------- Signed-off-by: Joanne Wang <jowg@amazon.com>
Description
[Describe what this change achieves]
Issues Resolved
[List any issues this PR will resolve]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.