-
Notifications
You must be signed in to change notification settings - Fork 0
GSoC2015 Proposal: Log Message flow visualization(mahmoodm2)
The syslog-ng daemon is a well-known highly scaleable logging solution which is widely used in creating the centralized logging solutions. It can process logs from different sources, apply different filters and rules on collected logs and send them to different destinations. It could be very useful, especially for debugging or educational purposes, to have a visualizer aiming to discover what's happening in the syslog-ng ecosystem.
The goal of this project is implementing a standalone visualizing application in order to visualize the details happening in syslog-ng pipe lines, sources, destinations and make it possible for the users to understand more details about timing, queues and messages life cycles.
I had an experience of contributing in developing a SIEM tool for collecting logs from IDSs and send them to a central processing unit for correlation using syslog and IDMEF. During that project I got familiar with syslog-ng and its capabilities, and know with this project I can significantly improve my knowledge and skills and contribute in a project which is used much more widely. This also can improve my skills in C/C++ and Qt programming by building a reliable and efficient tool.
To implement this project my first preferred IDE is Eclipse ( I currently use it for Java/JSP/Servlet and plug-in development for Eclipse).On of the challenging part of the project would be the unit testing phase to cover as much as different conditions a log status could be generated and also simulating the a real worl deployment of a syslog-ng ecosystem to test the whole process.
- May(2 weeks) :
* Reading and reviewing the syslog-ng documentations and architecture and configuration file.
* Getting familiar with syslog-ng code structure, main classes,data types, log pipelines, source/destination drivers ,log structure and route of a log message
* Compiling the source code and getting familiar with different modes of operation and different information which can be considered as a status
* Discussing with project mentors about the more detailed scope, deliverables and functionalities of the project
* Preparing a more detailed tasks, deliverables and scheduling of the project - May-June ( 3-4 weeks): * Designing a general statue reporting format with fields such as pipe line component ID ( if possible), send/receive time stamps( with considering the different time zones ), applied filter IDs(if possible), stop/restart actions and their time stamps, position in the queue ( if possible). It is required to know the structure of logs flow-control, fetch, fifo and control-windows parameters, different source/destination and parsing mechanisms. * Designing a new feature for different components of the syslog-ng which can support processing and sending the log statuses. More over, designing the structure of file/database required to store the log statuses ,how to deal with archiving/deleting the huge amount of log statuses during a long period and also the sending/receiving mechanism of log statuses to the visualizing application in addition to the deployment structure of an operational version of this new feature. * Implementing the the new status format and status reporting feature in components feasible for this purposes.It is requires to ensure that the log status reporting is functioning properly to store/send the log statuses in a central file/database. * Writing and running the test cases, unit tests and fixing bugs. Redefining the log status format based on the some limitation or new data discovered during the implementation. * Getting feedback from the mentor and redefine and modify the next phase details
- June-July( 2-3 weeks) * Designing the features of the standalone visualizing application such as db/file structure, main user interface components, some mockups and scenarios, message searching functionality, pipe lines components visualizing, message life cycle timings, send/receive times and delays.And finally getting and adjusting the design based on the feedback from the mentors. * Implementing the designed application using the Qt libraries and based on the mockups and visualizing features. It is required to have a prepared database of log statuses based on the developed new format to test the visualizing functionalities * Writing and running the test cases, unit tests and fixing the bugs * Getting feedback from the mentor and redefine and modify the remaining details
- July-August (3-4 weeks) * Integrating all the codes, the new features implemented in the different components with the visualizing application to cover whole log status life cycle. * Evaluating the test cases and some code modifications and fixing the bugs. * Revisit all the codes and make annotation on important or hard to understand code * Applying some code polishing * Deployment
Availability I will be able to commit 20-30 hours per week on May, and also 40( even more if required) hours per week from June to August.I don't have internship.
I am a graduate student pursuing the software and information systems major in the University of North Carolina at Charlotte(UNCC) who expect to graduate in 2017. I am also one the contributors of the OWSP ASIDE project which is an open source project about interactive static analysis to help the developers find and remove the security bugs during the development time.I had years of developing softwares in different areas in different languages as Java and C#. I mainly use Mac OS X and Ubuntu. I have experiences in software development as team member or coordinator. Currently I am programming in Java ( both for assignments and ASIDE project) and C# and some times C. I am also studying about using the unit testing for security purposes. I had an experience of working with log management systems to develop a SIEM system to collecting, normalizing and correlating the logs (and the IDMEF message exchange format) from different sources such as IDS or network firewalls and during this project I got familiar with syslog-ng and its capabilities.
Hopefully this project can improve my skills in software design, implementing and testing by contributing in real world and industry used open source project.