Create authenticated enrichment service

[mlodic]

We could provide a service that could be queried via API key. In this way, it would be possibile to understand if an IOC is in the database of Greedybear without having to download and manage all the feeds from Greedybear.

It would be a simple enrichment service.

We would need:

• a basic GUI (Create basic GUI #11) to allow people register and get an API key.
• limit API usage to avoid abuse.
• allow different kind of API usage limits
• create new API endpoint (create new API endpoint for enrichment (related to #12) #17)
• Integrate it in IntelOwl ([analyzer] Integrate Greedybear from The Honeynet Project IntelOwl#817)

[uzaxirr]

Hey, I wanna take up this one.
I'll first start by creating the API Endpoint. From what I've understood is that you wanna have a API endpoint which serves the same result as this https://<greedybear_site>/api/feeds/<feed_type>/<attack_type>/<age>.<format> feed URL.
Below is my approach from the same based on my understanding.
I'll be using Django REST Framework and will be leveraging the api.views.feeds function for this endpoint. and the endpoint would be api/feeds and it will take the following fields in Request Body.

• feed_type
• attack_type
• age
• format
  Please Correct me if I'm wrong

[mlodic]

Hey, thanks for your interest in the project.

Well, no that was not my intention. Even if not built with the DRF, this does not mean that it is not already an API endpoint. That change would not benefit in any way.

I think I need to add more context to let you understand the issue. Right now, this project provides feeds that are regularly updated (every 10 minutes). This means that these indicators of compromise, to be used, must be downloaded by the consumer periodically and must be managed and saved in some way from there. This is useful for some use cases. If you have a Threat Intel Platform of your own, you can just download these data and feed the platform periodically. If you have a firewall, you could just add this feed and the firewall would manage the download. But if you do not need to retrieve all of these data periodically and manage them, but you just want to ask whether a specific observable (IP address or domain) was seen by Greedybear or not to know if you should flag it as malicious or no, well... you cannot. For this common use case, we would need to add the chance to query a new API endpoint (for ex, /api/enrichment?query=<observable>). The server would look for the queried observable in the database and would provide to the requester the data that it knows about. Pretty simple.

To give you either more context, IntelOwl is an aggregator of these kind of "enrichment" services. Most of the "analyzers" work in that way. For instance, this new service could be integrated in IntelOwl as a new analyzer. This is the direction that we want to follow.

[uzaxirr]

Hey, thanks for your interest in the project.

Well, no that was not my intention. Even if not built with the DRF, this does not mean that it is not already an API endpoint. That change would not benefit in any way.

I think I need to add more context to let you understand the issue. Right now, this project provides feeds that are regularly updated (every 10 minutes). This means that these indicators of compromise, to be used, must be downloaded by the consumer periodically and must be managed and saved in some way from there. This is useful for some use cases. If you have a Threat Intel Platform of your own, you can just download these data and feed the platform periodically. If you have a firewall, you could just add this feed and the firewall would manage the download. But if you do not need to retrieve all of these data periodically and manage them, but you just want to ask whether a specific observable (IP address or domain) was seen by Greedybear or not to know if you should flag it as malicious or no, well... you cannot. For this common use case, we would need to add the chance to query a new API endpoint (for ex, /api/enrichment?query=<observable>). The server would look for the queried observable in the database and would provide to the requester the data that it knows about. Pretty simple.

To give you either more context, IntelOwl is an aggregator of these kind of "enrichment" services. Most of the "analyzers" work in that way. For instance, this new service could be integrated in IntelOwl as a new analyzer. This is the direction that we want to follow.

Thanks a lot for such a detailed explanation. Got a Much clear idea of the task now.
Here's what I understood for eg: I hit /api/enrichment?query=199.127.60.104 i would search all the records in DB from IOC model's table against 199.127.60.104 and return all the details I get from that record.
please correct me again if I'm wrong.
and one more question. the observable (199.127.60.104 in the above case) would be present in name field of IOC model ryt?
so I have to query all records where name=<observable> (name=199.127.60.104 in the above case) correct?

[mlodic]

Exactly. Well, it is missing the unique value for the name attribute but it should be unique so no more than a single record should be found.
We would also need to add an index for that value

[uzaxirr]

Cool!!
If you wish you can create a child issue for Each of the tasks (create new API endpoint, allow different kind of API usage limits, etc) and assign the Create API one to me. This would make the feeds clutter-free.
Or if you do not wanna create a separate issue you can assign this to me as well. I'll be raising a PR soon

[yogesh-sirsat]

Hey, @uzaxirr are you still working on this?

[uzaxirr]

Hey, @uzaxirr are you still working on this?

Yep, I've created the API via #25 and currently working on #21 If you want you can pick up #11 because i don't intend to work on it

[yogesh-sirsat]

Okay.

[mlodic]

this has been completed with the major release