Open Source multi-vendor ICAP server
Scan files requested via a proxy server using ICAPeg ICAP server, ICAPeg is an ICAP server connecting web proxies with API based scanning services and more soon!. ICAPeg currently supports VirusTotal,VMRAY , MetaDefender & Clamav for scanning the files following the ICAP protocol. If you don't know about the ICAP protocol, here is a bit about it:
What is ICAP?
ICAP stands for Internet Content Adaptation Protocol. If a content (for example: file) you've requested over the internet to download or whatever, needs adaptation(some kind of modification or analysis), the proxy server sends the content to the ICAP server for adaptation and after performing the required tasks on the content, the ICAP server sends it back to the proxy server so that it may return the adapted content back to the destination. This can occur both during request and response.
To know more about the ICAP protocol, check this out.
Things to have
Before starting to play with ICAPeg, make sure you have the following things in your machine:
Golang(latest enough to be able to use go mod)
A sample installation of go version 1.14:
Prepare the apt packages
sudo apt update
sudo apt upgrade
Link of download of version 1.14 https://dl.google.com/go/go1.14.linux-amd64.tar.gz
Use the command
Untar in /usr/local
tar -C /usr/local -xzf go1.14.linux-amd64.tar.gz
Add /usr/local/go/bin to the PATH environment variable:
Note: this command adds the PATH temporarily to the environment variables, the path is removed if SSH is broken, or system reboots.
- A proxy server
squid is an example in this readme
- A scanner vendor.
Clamavas scanner vendors. Make sure that you setup your scanner vendor properly. You can setup your scanners for both RESPMOD & REQMOD. Although, not every scanner supports every mods. Check this out to know which scanner supports which mods.
VirusTotal as your scanner vendor in the config.toml file
resp_scanner_vendor = "virustotal"
req_scanner_vendor = "virustotal"
In that same file, add a VirusTotal API key in the
api_key field of the
[virustotal] section. Here is how you can get it.
MetaDefender as your scanner vendor in the config.toml file
resp_scanner_vendor = "metadefender"
req_scanner_vendor = "metadefender"
In that same file, add a MetaDefender API key in the
api_key field of the
[metadefender] section. Here is how you can get it.
vmray as your scanner vendor in the config.toml file
resp_scanner_vendor = "vmray"
req_scanner_vendor = "vmray"
In that same file, add a VMRay API key in the
api_key field of the
[vmray] section. Get your api key by requesting a free trial.
clamav as your scanner vendor in the config.toml file
resp_scanner_vendor = "clamav"
Next, provide the clamd socket file path(getting back to this in a bit) in the config.toml file inside the clamav section
socket_path = "<path to clamd socket file>"
NOTE: All the settings of ICAPeg is present in the config.toml file in the repo. Also before selecting your vendors as the scanners, keep in mind to check whether that certain vendor supports the modification mode or not. For example, when adding
virustotal as the
resp_scanner_vendor, check under the configuration of
virustotal if the
resp_supported flag is true or not. Likewise for
req_scanner_vendor and for any other vendors.
How do I turn this thing on!!
To turn on the ICAPeg server, proceed with the following steps (assuming you have golang installed in you system):
- Clone the ICAPeg repository
git clone https://github.com/egirna/icapeg.git
In case not using go version 1.14, you could discover your version
You should use the corresponding export command
1.14 ===> export GO114MODULE=on
1.13 ===> export GO113MODULE=on
- Change the directory to the repository
- Add the dependencies in the vendor file
go mod vendor
- Build the ICAPeg binary by
go build .
- Finally execute the file like you would for any other executable according to your OS, for Unix-based users though
You should see something like,
ICAP server is running on localhost:1344 .... This tells you the ICAP server is up and running
OR, you can do none of the above and simply execute the run.sh shell file provided, by
That should do the trick.
- Now that the server is up and running, the next thing to do is setup a proxy server which can send the request body to the ICAPeg server for adaptation. Squid looks like just the thing for the job, go to the site provided and set it up like you want. After setting up your proxy server for example squid, change its configuration file:
Open squid.conf file
sudo nano /etc/squid/squid.conf
Add the following lines at the bottom of your ACLs configurations
icap_enable on icap_service service_resp respmod_precache icap://127.0.0.1:1344/respmod-icapeg adaptation_access service_resp allow all
Add the following line at the end of the file
cache deny all
A sample conf file for squid exists in the repository in a file squid.conf
Save and close the file Press CTRL + x, then press Y, then Enter
systemctl restart squid
How do I know its working!
Now that you have squid running as well, you can test it out by trying to download/access a file from the Internet(through the proxy) and see the magic happens! You'll be able to download/access the file if its alright,
To test properly using malicious files , visit the Eicar Test File Site, and try to download a malicious file.
For example, open the following link
Oh, and do not forget to setup your Browser or Machine 's proxy settings according to the squid.
If you are still not getting any logs and the file is downloaded whatsoever, please check your proxy(squid) setup.
Things to keep in mind
You will have to restart the ICAP server each time you change anything in the config file.
You will have to restart squid whenever you restart the ICAP.
You need to configure your network(or your browser)'s proxy settings to go through squid.
This project is still a WIP. So you can contribute as well. See the contributions guide here.
ICAPeg is licensed under the Apache License 2.0.