Skip to content

NLP processing of documents to extract potential threat intelligence data

License

Notifications You must be signed in to change notification settings

benhe119/act-scio

 
 

Repository files navigation

act-scio

Requirements

SCIO requires a working clojure environment to build and beanstalkd/java to run.

Beanstalkd

The scio platform uses the beanstalkd mq. This must be installed and running.

Installation

Clone repository

git clone https://github.com/mnemonic-no/act-scio.git
cd act-scio

Download vendor files to vendor/ (OpenNLP models, Geo names and TLDs)

This will populate the vendor/ directory.

scripts/get-vendor-files.sh

To run locally

In the repository root, run this command to create a local config (etc/scio.ini.local) where all directories points to our local repository.

This step is required to run the tests.

sed "s#/opt/scio#$(pwd)#g" etc/scio.ini > etc/scio.ini.local

Create directoy for storing documents.

mkdir documents

System wide installation

Copy required files to /opt/scio:

mkdir -p /opt/scio/documents
cp -r etc vendor /opt/scio

To build

lein uberjar
lein test

Testing

lein test

Usage

java -jar ./target/uberjar/scio-back-[VERSION]-standalone.jar --config etc/scio.ini.local

Config file defaults to /etc/scio.ini if not specified.

Running as a service

A systemd compatible service script can be found under examples/systemd.

To install (requires latest uberjar in /opt/scio):

cp examples/systemd/scio-back.service /usr/lib/systemd/system
systemctl enable scio-back.service
examples/systemd/upgrade-latest.sh

The upgrade script will create a symlink from the latest uberjar found in /opt/scio.

Bugs

License

Copyright © 2016-2019 by mnemonic AS opensource@mnemonic.no

Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT

About

NLP processing of documents to extract potential threat intelligence data

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Clojure 35.8%
  • HTML 32.5%
  • Python 26.8%
  • Shell 4.9%