Permalink
Browse files

Initial commit

  • Loading branch information...
Nicolas THIBAUT
Nicolas THIBAUT committed Jun 26, 2018
0 parents commit d64fc9cd81fe26204f5618dbf8ad14d47218ed5a
@@ -0,0 +1,2 @@
*~
venv
@@ -0,0 +1,80 @@
## *UPPERSAFE Open Source Firewall*

OSFW is a firewall, fully written in Python 3, that provides an IP / domain filtering based on a collection of public threat intelligence feeds.
It blocks in real time *incoming* / *outcoming* traffic considered as *malicious* (matching the filtering rules automatically set up for each threat).
It also provides a secure DNS service that blocks different kind of cyber threats (phishing websites, malware hosting, C&C servers, etc).

## Components

OSFW includes 3 main components:

- **osfw-sensor** (in charge of monitoring and logging the requests blocked by the firewall)
- **osfw-syncfw** (in charge of collecting and syncing the threat intelligence feeds)
- **osfw-webapp** (in charge of offering a management dashboard)

## Quick start

Setup the virtual environment:

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Start the firewall components:

bash run.sh

Attach a screen:

screen -r osfw-sensor
screen -r osfw-syncfw
screen -r osfw-webapp

## Configuration

To enable the secure DNS service, simply create a symbolic link of the *firewall.conf* file to the unbound configuration directory with the following command:

ln -s "$PWD/assets/unbound.conf" /etc/unbound/unbound.conf.d/firewall.conf

It is possible to customize the behaviour of the firewall by editing the default *config.yaml* file.

One of the reasons you would want to edit this file is to unblock specific websites.
It happens that some legit and top ranked websites got blocked because of different purposes, most of the time one of the following:

- Their users can upload files on the main domain (file transfer providers or cloud storage providers)
- Their users can upload files or even web pages on a subdomain (hosting providers)
- Their users can perform URL redirect (link shortener websites)

To prevent these websites from being blocked, you can specify them as a list in the configuration file.
In case you want to edit the default list, you can use a magic keyword ".tld" that will match any top level domain and the following second level domain names:

- .co.tld
- .com.tld
- .net.tld
- .org.tld
- .edu.tld
- .gov.tld

There is also a way to make a rule act as a subdomain wildcard, to do so you need to start the rule with a "." such as the ones already in the configuration file.

## Dependencies

- python3
- iptables
- unbound
- screen

## Python requirements

- pyyaml
- colorlog
- requests
- flask
- sqlalchemy
- pebble

## Support

Nicolas THIBAUT (nicolas[@]uppersafe[.]com)

https://www.patreon.com/dev2lead/memberships
No changes.
BIN +9.66 MB assets/sqlite.db
Binary file not shown.
@@ -0,0 +1,171 @@
---
db: assets/sqlite.db
debug: yes
threads: 32
flaskHost: 127.0.0.1
flaskPort: 8000
userAgent: UPPERSAFE (Open Source Firewall Project) https://github.com/dev2lead/uppersafe-osfw
queryTimeout: 10
refreshDelay: 3600
filterMode: classic
groupRange: yes
monitor: /var/log/messages
network:
eth: eth+
ppp: ppp+
tun: tun+
unbound:
verbosity: 1
hide-version: yes
interface: 0.0.0.0
access-control: 10.0.0.0/8 allow
feeds:
- alienvault
- blocklist
- ciarmy
- greensnow
- openphish
- phishtank
- ransomware
- spamhaus
# - uppersafe
exemptions:
# - 217.78.11.148
- 127.0.0.1
- .163.tld
- .360.tld
- .abuse.co
- adf.ly
- .adjust.tld
- .akamai.tld
- .akamaiedge.tld
- .akamaihd.tld
- .alibaba.tld
- .alienvault.tld
- .aliexpress.tld
- .amazon.tld
- amzn.to
- .aol.tld
- .aolcdn.tld
- .aryaka.tld
- .baidu.tld
- .bing.tld
- bit.ly
- .bitly.tld
- .bitbucket.tld
- .blocklist.tld
- .box.tld
- buff.ly
- .buffer.tld
- .cinsscore.tld
- .cloudflare.tld
- .discord.tld
- .discordapp.tld
- .doubleclick.tld
- .dropbox.tld
- .duckduckgo.tld
- .dyn.tld
- .dyndns.tld
- .emailretargeting.tld
- .facebook.tld
- .fb.tld
- .fbcdn.tld
- .free.tld
- .freecdn.tld
- g.co
- .gandi.tld
- .ggpht.tld
- .github.tld
- .githubusercontent.tld
- .gitlab.tld
- .gmail.tld
- goo.gl
- .google.tld
- .googleadservices.tld
- .googleadwords.tld
- .googleapis.tld
- .googleapps.tld
- .googlecache.tld
- .googlecode.tld
- .googledocs.tld
- .googledrive.tld
- .googlegroups.tld
- .googleimages.tld
- .googlemail.tld
- .googleplus.tld
- .googleproxy.tld
- .googlesearch.tld
- .googlesyndication.tld
- .googletranslate.tld
- .googleusercontent.tld
- .greensnow.tld
- .gstatic.tld
- .gsuite.tld
- .hotmail.tld
- .href.tld
- .icloud.tld
- .iconfinder.tld
- .imageshack.tld
- .imgur.tld
- .instagram.tld
- .line.tld
- .linkedin.tld
- .live.tld
- .lnkd.tld
- .mailchimp.tld
- .mailgun.tld
- .mailjet.tld
- .mandrill.tld
- .mandrillapp.tld
- .maxcdn.tld
- .messenger.tld
- .microsoft.tld
- .microsoftonline.tld
- .microsoftstore.tld
- .msdn.tld
- .msft.tld
- .msftncsi.tld
- .naver.tld
- .netflix.tld
- .office.tld
- .office365.tld
- .openphish.tld
- .outlook.tld
- .ovh.tld
- ow.ly
- .parastorage.tld
- .phishtank.tld
- .pinterest.tld
- .postimage.tld
- .postimg.tld
- .prestashop.tld
- .qq.tld
- .quora.tld
- .reddit.tld
- .shopify.tld
- .similarcdn.tld
- .similarweb.tld
- .sina.tld
- .sohu.tld
- .spamhaus.tld
- t.co
- .telegram.tld
- .tiny.tld
- .tinyurl.tld
- .tumblr.tld
- .twimg.tld
- .twitch.tld
- .twitter.tld
- .uppersafe.tld
- .vk.tld
- .weibo.tld
- .whatsapp.tld
- .wix.tld
- .yahoo.tld
- .ymail.tld
- .youku.tld
- youtu.be
- .youtube.tld
- .ytimg.tld
- .yy.tld
- .wetransfer.tld
@@ -0,0 +1,9 @@
#!/usr/bin/env python3
##
# Nicolas THIBAUT
# nicolas.thibaut@uppersafe.com
##
# -*- coding: utf-8 -*-

from .sensor import sensor
from .syncfw import syncfw
@@ -0,0 +1,62 @@
#!/usr/bin/env python3
##
# Nicolas THIBAUT
# nicolas.thibaut@uppersafe.com
##
# -*- coding: utf-8 -*-

from utils import configuration, database, logger
import sys, re, time, os

conf = configuration()
db = database(conf.get("db"))
log = logger(__name__, conf.get("debug"))

class sensor:
def __init__(self):
self.ino = 0
self.idx = 0
self.start()

def parse(self, buffer):
for source, destination, port in re.findall(".* SRC=([0-9a-f:.]+) DST=([0-9a-f:.]+) .* DPT=([0-9]+) .*", buffer, re.IGNORECASE):
log.info(str("Blocking '{}' -> '{}:{}'").format(source, destination, port))
db.session.add(db.models.events(source=source, destination=destination, port=port))
try:
db.session.commit()
except Exception as error:
db.session.rollback()
log.error(error)
return 0

def watch(self, file):
self.idx = 0
if not self.ino:
self.idx = os.stat(file).st_size
self.ino = os.stat(file).st_ino
log.info(str("[!] Subscribing to file '{}'").format(file))
with open(file) as fd:
fd.seek(self.idx)
try:
while self.ino == os.stat(file).st_ino:
for element in [x for x in fd.read().split("\n") if x.strip()]:
self.parse(element)
time.sleep(1)
except:
pass
log.warning(str("[!] Unsubscribing from file '{}' (probably because of log rotation)").format(file))
return 0

def start(self):
while self.idx >= 0:
try:
conf.reload()
except Exception as error:
log.critical(error)
return 1
if os.path.exists(conf.get("monitor")):
self.watch(conf.get("monitor"))
else:
log.error(str("[!] Could not find '{}' to monitor network events").format(conf.get("monitor")))
time.sleep(1)
return 0
Oops, something went wrong.

0 comments on commit d64fc9c

Please sign in to comment.