GSOC 2015 Application Goran Cetusic: Graph data backend API
| Goran Cetušić | Short bio |
|---|---|
 |
My academic and programming career started with my enrollment as a Bachelor student at the Faculty of Electrical Engineering and Computing in Zagreb. During that time I've been involved in various activities like lecturing student courses, organizing events and projects, leading several of them as the section leader of the Electrical Engineering student club. I got my first job as a network administrator at a small firm and where I started to learn some real programming in Python. Before finishing my Bachelor studies, I got an internship position as an Openlab summer student at CERN where I worked on process accounting software for their batch cluster. After a year working as a Linux administrator and contributing to several open source projects, I'm now a dedicated Python and Django developer, juggling my responsibilities as a Masters student in Telecommunications and Informatics, currently working on my thesis on Linux network simulators and various other interests like machine learning and bioinformatics, often "wasting" my time on Coursera. |
| Education |
 Faculty of Electrical Engineering and Computing, University of Zagreb Telecommunication and Informatics, Master studies |
| Contacts |
 
|
| Curiculum vitae | https://github.com/gcetusic/gcetusic.github.io/blob/master/docs/gcetusic_en.pdf |
A Linux enthusiast every since in 2006 someone installed Ubuntu on my laptop at a yearly Linux InstallFest. Since then I've taken over the organization of the same installfest for a couple of years and even organized and lectured two Linux courses at my college. After writing (mostly) bad C code in Gedit, I've switched to Python and Komodo Edit and it's been kind to me ever since. Except for that time when Komodo Edit deleted 300 lines of code...I'm using Sublime now, with all the benefits of its lint plugins.
After spending some time writing more bad code and making rookie mistakes, suddenly I realized I actually have something interesting to show to other developers. Python has served me well in this but I still have a lot to learn. For example, after doing web development for a couple of years, while trying to work around badly written Django code, I've implemented my own Python metaclass, probably the most abstract thing I've done in a programming language. Here are examples of some of the projects I've worked on (for a more complete list, please view my GitHub page, some are under NDA and closed sourced)
- https://github.com/stormpath/stormpath-sdk-python/ - a flexible user managagement API
- https://github.com/gcetusic/psacct-collector - process accounting software developed during CERN internship
- https://github.com/gcetusic/IMUNES - a powerful network simulator
The ProjectThe idea of the project is to make it possible to store graph data to separate backends. Until now, graph data has been represented as volatile Python dictionaries. It would make sense to provide a flexible backend interface to easily create modules for graph storage to efficiently access graphs at any time. This would usually include graph databases since their data representation is close what NetworkX does and such databases have efficient algorithms to access graph data but it shouldn't be restricted to such storages. Case in point would be document-store databases that can more or less directly save Python dictionaries as JSON data and load them. SQL databases are somewhat trickier because their data representation isn't directly compatible with graph.
Because this project would be implemented as part of GSOC there are a couple of ideas that are directly related to what the final implementation would look like because the other ideas proposed would be done in parallel. The most important one is the NetworkX 2.0 API. Right now, NetworkX has a lot of ways to access graph data because the data is represented as Python objects (dicts). A central data access location inside the code should make the job of creating a backend interface a lot easier. Since this largely depends on the redesign of the NetworkX API, people involved in these two projects should coordinate their efforts and keep track of the changes between these two branches. Also, there is talk about adjacency matrix representations of graphs. While the reasoning behind this are matrix related algorithms, it would make sense to also coordinate the development of this feature with backend storage and API 2.0 development. So the combined roles of these three projects would be:
- NetworkX 2.0 API
* create a consistent and centralized graph data accessor but with the usual dict-based storage - Backend API
* extend the 2.0 API to make the dict (memory) access modular * use the same interface to make a backend module - one for a graph db (possibly Neo4j) and one for a document db (Mongo or Couch) - Matrix graphs
* in accordance with the Backend API, create a backend module to store graphs as matrices
- Getting more familiar with the codebase
- Coordinating and defining specifications for related projects
- Researching graph-based databases and available graph algorithms
- Defining clean generic methods for possible backends
- Representative benchmark suite for performance evaluation
- Extracting the Python dict storage as a module
- Implementing the caching system
- Syncing designs with NetworkX 2.0 API
- Extensive cache related unit testing
- Midterm evaluation
- Resolving any design problems
- Maintenance and codebase improvements
- Implementing a graph backend
- Implementing a document store backend
- Refining the backend API - make it more flexible
- Researching SQL-based backend
- Creating stub methods for relational DB specific actions (e.g. migrations)
- Implementing some of them if there is time
- Buffer Period for completing any incomplete work
- Making sure there's good test coverage and documentation
- Final syncing of other GSOC projects (merges)
During the first 4-5 weeks I would have to work on my masters thesis parallel to GSOC (the network simulator mentioned in the NetworkX GUI proposal). It's my last year as a student and the deadline for turning it in is the end of June. Even though I would be able to devote at least 4 hrs a day during the period between the start of GSOC and the end of June I would start coding by the beginning of May (during the community bonding period) to get a head start.