-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Google Summer of Code 2015
The following page describe the project ideas that students can develop with our mentors and community in general. Keep in mind that for us the communication is a fundamental key and when applying you have to be very detailed about how do you plan to approach the project.
Difficulty: medium
Tags: C Ruby, Ruby, JSON, Web Services
Description
the Fluentd data collector program is a complete command line service, but there are some cases where a UI is required to handle multiple instances and perform complex configurations. Fluentd support many type of inputs and output data and recent version support specific filters to alter the data content. Having a UI aims to simplify the general overview of the Fluentd service instance.
This project requires a complete implementation of a Fluentd UI based on Ruby language with the following requirements:
- Handle Fluentd configuration schemas (sources and matches).
- Import Fluentd library and perform basic administration tasks such as start, stop, restart.
- Import and Export configuration to different formats (e.g: JSON)
- Templates support: templates aims to simplify the configuration when connecting with different services sush as Logstash, MongoDB, Elastic search. The number of templates and details will be defined later, the important idea is the engine to handle that capability.
- On the design part, it must use Twitter Bootstrap and be Mobile friendly.
- The complete implementation must use Ruby libraries and cannot use sources that are not part of the Ruby ecosystem
Fluentd is not just a data collector, it's also a complete ecosystem where different tools for different targets are provided. At the moment there is a missing component that we aim to implement: the C# Logger Library.
The student applying to this project, requires previous knowledge of C# language. The Library must be full documented and contain several examples of the usage to demonstrate it capabilities. The required features are the following:
-
The core library must support the MsgPack format, binary communication with Fluentd is fundamental for performance and network bandwidth reduction reasons.
-
Library must be capable to perform network operations with the main Fluentd service, it needs to support TCP_FASTOPEN options for Linux and offer KeepAlive persistent connection feature.
-
On the networking side, it must be capable to handle lost connection issues and detect latency problems, offering a set of callbacks to notify and let the caller take decisions over that.
-
Thread Mode Engine: this optional mode, aims to let the developer start a logger instance that runs in a service mode, it needs to be asynchronous and use lock-free features when handling global data structures.
-
Documentation: the library must be fully documented and with examples as stated before. This topic is fundamental for it adoption.
Despite Fluentd is a very fast log collector made in C Ruby, sometimes third party plugins for inputs or outputs, may affect it performance. We aim to implement a preventive mechanism where Fluentd uses can detect these easily and perform performance reports with real data.
The profiling stack project for Fluentd, needs to add the following features to the core product:
-
Be able to be activated on runtime: think about a service which is running in Production and is facing a low latency when processing thousands of events. On that case let the user to activate the profiling mode and gather some statistics for a fixed amount of time. It activation is on-demand, it would never need a restart.
-
Profiling for our context means gather metrics and timings associated to key calls of the stack. This cover from the Networking inputs, Routing, Filtering and Outputs. The software must be able to report where the bottleneck is.