-
Notifications
You must be signed in to change notification settings - Fork 0
GSoC2015 Proposal: syslog ng for Java (szpeter)
syslog-ng is powerful log management solution with many log processing features, but if it is not enough, it can be extended with new modules written in C. However C is not the language for rapid development,. To overcome this problem, a Java destination is already available in syslog-ng-incubator. The goal of this project is to extend this functionality, and make it possible to write filters, parsers, rewrite rules, template functions, and even sources in Java.
syslog-ng is a powerful tool, that gives solution for wide range of logging problems, but technology is evolving faster and faster, and new problems comes alive every day. syslog-ng has to keep pace with these claims, and the Java language binding can be extremely useful for this purpose, because Java is a very popular language with a rich set of existing libraries. It makes easier to extend the functionality of syslog-ng in a reduced time compared to C. And also it makes easier to integrate syslog-ng with other systems. Of course Java can be engaging to large number of developers who has the intent to contribute and has experience in Java, but cannot code in C.
It is not in the main scope now, but I think it can open a way to other languages too. Java code compiles to bytecode that runs on the Java Virtual Machine (JVM). The JVM was initially designed to support only the Java programming language, however, as time passed, more languages were adapted or designed to run on the JVM, such as Scala, Clojure, Groovy, JRuby, JPython and so on. Almost every language has a port to JVM, and these languages might be integrated with syslog-ng's JNI based connectors. To prove this I have done some experiments. Scala is a quick win, because of its seamless interoperability with Java. It worked without any problem, so this can be another good point if anyone would write a source/destination/parser/etc in a functional way. JRuby and JPython does not worked for the first try, however it does not seemed impossible to make it work.
The true expert can understand the operation of the whole system from the first bit in a cpu register to the high level architecture. I know a life is not enough to learn everything that involves, but I would like to learn as much as I can. I have experience in C, Java, and I have worked with JNI before, but there are a lot more to learn in this topic. I like to connect different little pieces to make something complete, and this project can make me deepen my knowledge in how modules written in different languages can interact with each other.
As I wrote in my introduction e-mail, I am using tons of open source software every day, including syslog-ng and this is a great opportunity to give something back to the community.
In syslog-ng the incoming log messages travel across the message processing pipeline. The goal is to make it possible to write the elements of this pipeline in Java. To reach this goal there should be pipeline elements which are actually connectors between the the C and Java layer. A Java destination is already exists in syslog-ng-incubator, and also integrated to syslog-ng master branch too. The implementation contains common parts, such as starting a JVM, loading of a Java class, calling its methods through the Java Native Interface (JNI), and of course the configuration related changes. In a nutshell, the existing Java destination consists of three parts now:
- A native part, which is an implementation of a destination driver. In this case this destination driver is just do basic task that has to be in the native code, and forward other calls to the proxy.
- The next part is the proxy. Proxies are the links between the C and Java code. Java can call native methods and C can call Java methods through this.
- Java classes following the C structure's hierarchy. These classes can be directly used in Java, or can be implemented in case of abstract base classes.
The task is to write connectors for the other elements of the message processing pipeline, following this concept.
Pipeline elements to be implemented:
- source
- filter
- rewrite rule
- parser
- template function
- Other deliverables:
Documentation, simple examples on using each connector.
- Before the official coding period beings I would like to give another try to languages which runs on JVM to uncover the possibilities. This undoubtedly leads me closer to the related parts of the codebase.
- week 1: Implement proof of concepts, and refine the plans according to the previous experiences.
- week 2-3: Implementing filter connector
- week 4: Implementing parser connector
- week 5: Implementing rewrite rule connector
- week 6: Implementing template function connector
- week 7-8: Implementing source
- week 9: Buffer period
- week 10: Performance tests, review
- week 11: Making improvements based on the review and test results
- week 12: Writing examples and documentations, deployment, summary
Each implementation period consists the following:
- design
- modification of config file
- implementation of C and Java parts
- unit testing on both the Java and C side if possible and necessary (should be done in parallel with the implementation)
If everything goes better than expected and I still have time, I would like to write examples in other languages if possible.
- patches should be clean and understandable
- follow the clean code principals
- compile via command line tools without warnings
My name is Peter Szabo and I am an undergraduate Computer Engineering student at Budapest University of Technology and Economics. I am a fan of free software and a fanatic geek with a compulsion to disassemble, understand then modify or extend almost everything that falls into my hands. I like to make/hack hardware, but I even more like to code. It requires just a computer, but gives me the freedom to create anything I can imagine. I have experience in C/C++, Java and Python, but I have worked with several other languages on various platforms. To show some examples, I wrote a parental control system in Java that ran on my router, cross platform games for mobile devices in Lua, a couple of C code on embedded systems, nokia ringtone parser and player in Python ran on ARM, other home projects in C#, and of course as a linux user, tons of bash scripts to make my life easier. Last, but not least I have experience in automated software testing too. I am a little bit maniac about quality, and besides code review, testing is an essential step to grant software quality.
https://github.com/balabit/syslog-ng/tree/master/modules/java
https://github.com/balabit/syslog-ng-incubator
http://en.wikipedia.org/wiki/List_of_JVM_languages
https://github.com/asdwsda/syslog-ng-incubator/tree/f/gsoc_task
There is also a Scala destination proof of concept on this branch, but please keep in mind that I have never wrote Scala code before. It is just to test if it works or not, so please ignore the coding style in this case.