-
Notifications
You must be signed in to change notification settings - Fork 464
GSoC 2018 Proposal: HTTP batch sending (NikitaDoroshkin)
HTTP batch sending project is HTTP destination module improvement which enables batch requests for HTTP destinations that support such. This project allows to group separate HTTP calls together into a single HTTP request, instead of sending each call separately. The result of this project is an improved HTTP destination solution (libcurl based) that will make possible to send batched HTTP requests.
Syslog-ng is used to collect logs, process them and deliver to different destinations including HTTP. The current problem is that syslog-ng now uses separate HTTP calls for sending single message, which results in a certain amount of overhead. It’s a common situation when there is a set of relevant data - in case of Splunk it can be a set of events - and, therefore, it is naturally to pass this data in single bunch than to do multiple requests (in case of Splunk batched events are sent to HTTP Event Collector on Splunk Enterprise or Splunk Cloud).
All in all, the main advantages this project will bring to syslog-ng are as follows: * HTTP batch sending decreases overheads by single request than by multiple. * It provides better semantic basis of this approach in case of sending relevant set of data. * Also it improves flexibility of sending HTTP requests in system, since you can choose most suitable approach.
I’m seriously passionate about applied computer science and this project is a great opportunity for me to apply my knowledge and skills, especially those that I’ve received during programming and computer network courses at the university and self-study. I think that it’s a great opportunity to apply theoretical knowledge and practical skills in real world full-scale project. Study is good in case that it gives you theoretical background and the opportunity to develop small educational projects, but I want to participate in something really significant and get the invaluable experience of working with a large, open-source codebase.
Except programming, one of my strenghts is computer networking including knowledge in network protocols, so I decided that this project best suits me. I believe that to participate in the development of such a large open-source project with the guidance of mentor is the best way to get experience and become a part of open-source community.
The goal of HTTP batch sending project is to create reliable, effective, well-tested and fully documented functionality, that will make it possible for syslog-ng HTTP destination solution to make batch HTTP requests.
Knowledge areas required for the success of the project are: * libcurl - Familiar * C - Familiar * syslog-ng (destination module, threading) - Beginner * Community - Beginner
-
Familiarize with community and mentor.
-
Understanding of the syslog-ng project related documentation and codebase.
-
Increase familiarity with current HTTP module.
-
Implementing a design of new module
-
Increasing knowledge in syslog-ng module implementing
-
Designing and prototyping functionality.
-
Discussing results with mentor.
-
Core functionality implementation - combine messages in batch and send in a single HTTP request.
-
Continue work on project:
-
Finish completing the functionality - add flexible configurations.
-
Integration with syslog-ng.
-
Testing, some improvments.
-
Work on project:
-
Finish integration into syslog-ng.
-
Integration and testing with Splunk’s HTTP API.
-
Minor improvments and tests.
-
Self evaluation.
-
Minor fixes, tests, improvments.
-
Documentation, examples, code review, user guide.
-
Evaluate with mentor.
-
HTTP destination module with ability to send multiple messages in single batch.
-
Integration with HTTP destinations with batch requests support.
-
Good tested solution.
-
Documentation and user guide with relevant examples.
-
Name: Nikita Doroshkin
-
Address: Belarus, Minsk, 10 Oktyabrskaya street, Apt. 907
-
University: Belarusian State University
-
Faculty: Faculty of Applied Mathematics and Computer Science
-
Speciality: Applied Computer Science
-
Education level: Second year undergraduate
-
Languages: C/C++, Java, C#, Python, JS, SQL
-
Contacts:
-
+375291232640