New Playbook: run only "free" analyzers

[eshaan7]

updated: 15th March 2022

Idea / Motivation

An option that allows the user to only run free analyzers, the ones that don't require API Keys or login/password.

The main use-case for this is the following example:

• A user has configured some paid analyzers (with keys/credentials) but maybe for some analysis they want to save on API/service credits by checking the "run only free analyzers" option.

Pre-requisite(s)

• Playbooks (automated scripts) #628

Implementation Idea

This can be solved by creating a playbook that contains all the free (without api key/login) analyzers.

[0x0elliot]

Can I take this issue up? From the looks of it, I might need to add minor changes to the backend and IntelOwl-ng (And probably PyIntelOwl)

Any guidance before I proceed?

[mlodic]

yep you are right, this involves all the projects.

I would suggest starting from the backend then the python client and lastly the frontend.

This would probably be another flag the application should receive once it receives analysis requests from the user. (https://github.com/intelowlproject/IntelOwl/blob/master/api_app/api.py#L36)

[0x0elliot]

great! I think I should make a draft PR for this after I am done with the first backend things.

[0x0elliot]

I was thinking that to implement this issue, we can make it so that there is another option in analyzer_config.json itself. This would make it easier for us to keep track of which requested analyzer is free and which isn't.

I can make it so that if nothing is provided, It counts the analyzer to be running on its free version/is free by default. We can add this part in the documentation as well later on. This just seems like the simpler way for me.

What do you say? If it would be required to be done this way, Which file should I refer to for making the edit? Preferably the place where analyzer_config.json is being read.

[mlodic]

I think it makes sense.

There is a serializer which manages the data extracted from that JSON file.

[0x0elliot]

Thanks! Looking into this.

[0x0elliot]

Hey @mlodic! I forgot to update you on this back then. I reached out to @eshaan7 separately regarding this issue when I was assigned to it and we came to the conclusion that marking the analyzers as "free" might have been a hassle and some more discussion was needed to be done on moving forward with keeping this feature in mind.

I would appreciate it if we can continue that discussion and see how we should proceed with solving this problem.

One of the solutions that I propose to solve this is that we can add a new option key to analyzer_config.json called "subscription_type" and have its value by default if nothing is provided for an analyzer as a free like the docker based option.

But I need confirmation on this if we can even move forward with this because there is some clarification needed to confirm if all our analyzers do actually have a free version that we have adapted them for. If not then it seems like we would need to go through them one by one which seems to be the part that would cause the hassle.

How should I proceed with this one? What do you folks say?

[mlodic]

I think that, in some way, this problem could also be solved by the "Playbook" feature #628 (we could pre-load a pre-configured playbook with only the free-analyzers) but that would certainly require more work to solve it.
Meanwhile, we could solve it the easier way we mentioned.

I think that every analyzer which needs an API key/login should not be considered as "free", while all the other yes.
I don't think it is too slow to go through all of them: it would be enough to check the type of configuration they already need and, based on that, understand if they are free or not.

[devmrfitz]

@0x0elliot Are you working on this?

If not, @mlodic do you recommend me to work on this issue?

[mlodic]

you can, but it would require you to work from the branch dev-v4 where there is the new GUI based on React

[0x0elliot]

@0x0elliot Are you working on this?

If not, @mlodic do you recommend me to work on this issue?

I am not working on it. You can take care of this.

[eshaan7]

After discussing more on this issue and the implementation that was tried in #927, we have come to the conclusion that we do not want to add extra parameters to the Job model or the /analyze API. These are not straight-forward to use and makes the API more complex than it already is. There are a lot of ways in which a user may wish to filter out the analyzers and we should not bloat the Job model with a boolean field for each such use-case.

Moreover, the idea behind #628 was to provide a solution to this exact problem. So this will be solved by creating a playbook (once we have those) that contains all the free (without api key/login) analyzers.

[mlodic]

closed with #1238