Log Analytic Solution (AWS)

Overview

This project is a log analytic solution built on AWS services. It leverages various AWS services such as Amazon Kinesis Data Firehose, Amazon Managed Service for Apache Flink, Amazon S3, Amazon EC2, Amazon OpenSearch Service, and Kibana to provide a robust log analysis solution.

Features

• Real-time log ingestion using Amazon Kinesis Data Firehose.
• Stream processing and analytics with Amazon Managed Service for Apache Flink.
• Storage of processed logs in Amazon S3.
• Scalable compute resources with Amazon EC2.
• Log indexing and search capabilities with Amazon OpenSearch Service and Kibana.

Installation

1. Clone this repository to your local machine.

   git clone https://github.com/your_username/log-analytic-solution.git

2. Navigate to the project directory.

   cd log-analytic-solution

3. Follow the setup instructions for each individual AWS service used in the project. Refer to the documentation provided by AWS for detailed setup steps.

Usage

1. Configure Amazon Kinesis Data Firehose to ingest logs from your data sources.
2. Set up Amazon Managed Service for Apache Flink to process and analyze the ingested logs in real-time.
3. Use Amazon S3 for storing processed logs.
4. Deploy necessary compute resources using Amazon EC2 for running additional processing or analytics tasks.
5. Utilize Amazon OpenSearch Service and Kibana for log indexing, search, and visualization.

Contributors

• Saumya kumar
• Warshit poonia

License

This project is licensed under the MIT License.

---