Skip to content
Official repository for Crato: Easy log management for small applications
JavaScript Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
DB
bin
consumers
rsyslog-central-server update rsyslog .conf to handle failed delivery failure better. Update… Aug 18, 2019
.gitignore
Dockerfile
LICENSE
README.md Update README.md Aug 25, 2019
app.js
cli.js
config.js
crato.js es6 syntax for influx consumer Aug 11, 2019
docker-compose.yml
package-lock.json
package.json

README.md

Crato Banner

Welcome to Crato 👋

Crato version license

Crato is an open source framework for small applications to easily deploy centralized logging. Crato is built with Node.js, Rsyslog, Apache Kafka, InfluxDB, and AWS S3. This repo contains the core Crato project that receives, transforms, and stores log data for further analysis.

🏠 Homepage

Table of Contents

Crato Usage

Crato provides a CLI to make using the core system and tracking logs much easier. It requires installing both NPM and Node.js.

After installing the above dependencies, run npm install && npm link which will make the crato command available.

Here are a list of Crato commands:

Commands Description
services Provides a listing and description of all of Crato's services
deploy Starts up Crato system
shutdown Stops Crato system and all services
start <service> Starts a specific Crato service
stop <service> Stops a specific Crato service
install-kafka Installs Kafka cluster and textlogs & jsonlogs topics
container-logs | cl <service> Displays Docker container logs for a specific service
live-tail | lt See all external logs streaming into Crato. Press Ctrl-C to exit
shell | sh <service> Attaches a shell for a specific Crato service
status Displays the status of all of Crato's services

Environment Variables

Crato uses some environment variables for its Node.js consumer app mainly to archive logs to Amazon Web Services (AWS) Simple Storage Service (S3).

Crato uses AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_S3_BUCKET_NAME to authenticate for AWS S3. We also require the QUEUE_MAX_SIZE to determine the number of logs to queue, before being uploaded to AWS S3.

These environment variables should be written on your local machine and Docker will automatically read them if they are. See this great guide by Twilio for how to set environment variables on different operating systems.

Install

The Crato system is deployed and orchestrated through Docker and Docker-Compose with the docker-compose.yml file. Both of these must be installed.

To install Crato, clone this repo. Then run docker-compose pull to download the necessary Docker images to build the service containers. The install can be finished via Crato CLI commands or Docker or Docker-Compose CLI commands.

Deploying Crato

Using Crato CLI

  1. crato install-kafka
  2. crato deploy

Using Docker-Compose CLI

  1. docker-compose up -d zookeeper
  2. docker-compose up -d kafka
  3. docker-compose exec kafka kafka-topics --create --zookeeper zookeeper:8092 --replication-factor 3 --partitions 6 --topic textlogs
  4. docker-compose exec kafka kafka-topics --create --zookeeper zookeeper:8092 --replication-factor 3 --partitions 6 --topic jsonlogs
  5. docker-compose up -d

Use crato status or docker ps -a to check the status of the service containers that have been created.

Networking

Crato exposes ports 514 and 10514 for receiving log data into the system. On your own host machine where Crato is deployed, make sure these are available.

Show your support

Give a ⭐️ if you liked this project!


You can’t perform that action at this time.