-
Notifications
You must be signed in to change notification settings - Fork 0
GSoC2014 Proposal: Integration with configuration management systems (ihrwein)
Syslog-ng is a feature rich and highly customizable log management infrastructure. As the Swiss army knife of logging, it is widely used in the industry and constantly gaining momentum in the cloud area. As the world moves towards to the Cloud and Big Data, system administrators need to manage hundreds of thousands of machines. The configurations of these computers are almost the same but the configuration maintenance requires a huge amount of time and work. The configuration management and deployment automation systems solve these problems, but the most widely known ones lack the proper plugin for syslog-ng configuration.
Syslog-ng uses its own configuration file format. Taking care of a lot of instances of the same configuration is boring and can be automated. With plugins for the best known CMS systems, syslog-ng could gain a lot of popularity in the cloud. The goal of this project to write syslog-ng plugins for Salt, Chef and Puppet applications.
With these plugins system administrators wil have new tools to configure their syslog-ng instances. They can create new sources and destinations, filters and templates, log paths on thousands of machines in a few seconds by typing a command in the CLI. It also gives the ability to automate these commands and use them in scripts.
The project's aim is to write code for CM's and not for syslog-ng, but there is no doubt that syslog-ng will benefit from it:
- It eases the configuration of syslog-ng on a vast amount of computers. The easier it is, the more likely system administrators are to use syslog-ng. The community will get more feedback, patches and the number of features will rise,
- Syslog-ng can gain ground in data warehouses,
- The generation of syslog-ng configurations can not only be used in CM’s but also during tests for syslog-ng.
The problem can be broken into smaller pieces:
- How can we represent the configuration in memory?
- How can we generate textual configuration from the in memory representation?
- How does the configuration generation process work?
- The configuration of syslog-ng is complex. An option can contain an other option, so it is worth to considering to use the Composite design pattern.
- Each option should know how it can be persisted as text. A source can contain a lot of options, persisting the source creates the textual definition of it. Then syslog-ng can be used to validate the generated config files with the --syntax-only command line option.
- As a user enters a CLI command, for example:
$ salt '*' syslog-ng.new_source file=’/var/log/input.log’ multi-line-mode=indented
The plugin creates the in-memory representation of the given configuration. Then it serializes the configuration to text form, which is syntax checked by a syslog-ng instance. If it is good, it will be sent to the machines. The plugins support a method to send pre-created config file parts to the computers as well. This enables sending of arbitrary configuration, which might be too complex for generation or is manually written and already used on some machines. It is also worth to mention, that Salt is written in Python, but Chef and Puppet in Ruby. The latter two can share a good amount of code, but the representation model can be very similar in all of them. Required deliverables plugin for Salt, Chef and Puppet to generate syslog-ng configuration the generated configuration should include sources, destinations, filters
Optimize the generated configuration, so adding a source S1 and destination D1, then S2 and D1 would create a junction, no two different log paths
I am not planning any holidays in summer, but I will have 3 - 4 exams in June.
- week 1 - 2: creating a general configuration model
- week 3 - 5: writing plugin for Salt with documentation
- week 6 : getting familiar with Ruby
- week 7 - 9: writing plugin for Chef and documenting it
- week 10 - 11: writing plugin for Puppet
- week 12: code cleaning, writing documentation
My name is Tibor Benke, I'm a first year MSc student at Budapest University of Technology and Economics, where I study Computer Science. I heard about GSoC and the most interesting organization was the syslog-ng project. I've professional experience in C/C++, Java, Python and worked with git.
Since my childhood I always wanted to be an engineer. After graduating I got a certificate of "IT network installer and operator". Then I went to university, where I specialize in Networks and services, but I concentrate on designing and writing networked applications. My BSc thesis was to write a web based network management application in Python.
During the academic years I participated in some competitions. What I'm most proud of is the national EuroSkills 2013 qualifying rounds, where my team earned the sixth place in which I was the Linux specialist. I also have work experience: one summer I patched a network driver in the Linux kernel, then did some research in the field of Hadoop and Big Data.
In 2013 I contributed to some open source projects, like Log4j2, Logback and syslog-ng. Along with my studies I lead a group of 60 volunteer students, who operate the network and server of my faculty's dormitory, write games, mobile- and other applications.
The project I'm most interested in is "Integration with config management systems". Since I wrote my thesis in Python, so the Salt part shouldn't be so hard. I like to learn new languages, so I'm looking forward to enhance my knowledge by learning Ruby.
Project’s Idea page: https://github.com/balabit/syslog-ng/wiki/GSoC2014-idea-&-project-list#integration-with-config-management-systems