As a part of GSoC 2018, I worked on vitrivr App which is an Android client for interacting with the vitrivr stack. This application is created from the scratch taking advantage of Android native features to provide an easy interface for constructing media queries. I also did some preliminary work on extending cineast functionality by adding support for 'Add Media API' which can be used to upload and extract media files by sending a HTTP request to the API.
vitrivr is an open source full-stack content-based multimedia retrieval system with focus on video. Its modular architecture makes it however easy to process different types of media as well. More information can be found on the vitrivr website or if you prefer, you can dive into getting started guide right away for setting up the vitrivr stack.
This repository itself is major part of the project created completely by me.
The 'Add Media API' can be found in gsoc-2018 branch of cineast
Link to commit can be found here
Cineast API Documentation can be found here.
- The main goal of the project was to provide better support for Android devices which vitrivr-ng (Web based UI) couldn't. This app provides all the features currently present in vitrivr-ng for Android.
- Spatial query interface is also implemented in the app.
- Documentation of Cineast API (both REST and WebSocket)
- Preliminary work on 'Add Media API' for cineast with UI for Android.
- Making 'Add Media API' more functional, robust and secure.
- 3D Model queries currently use WebView for using three.js library's geometry to JSON conversion algorithm. Implementing the required functionality in native platform to increase efficiency.
- Providing option for using cineast REST API.
- Adding neighboring segment query option.
- Adding support for Metadata & Tag lookup.
- In the beginning, I spent a lot of time in deciding and implementing the architecture and the project structure. This was mainly because I was used to MVP architecture but for this project, I decided to use the current popular and Android's recommended MVVM architecture. Turns out, the result is worth the time spent. Using MVVM, stream data flow and orientation handling turned out quite easy and robust.
- While implementing audio queries, I was getting a errors due to wrong sample rate and channel which I figured out after a while. In the process, I dived into various intricacies of digial audio (which I later realized were not needed) which helped me learn a lot about them.
- While imlementing 3d model queries, major challenge was poor 3D library support for android which was solved by using three.js Javascript library.
I tried to follow the best practices for Android and maintained high standards in terms of code quality and structure so that there is no compromise in app performance and stability:
- Model-View-ViewModel (MVVM) architecture
- Use of LiveData for communication between View & ViewModel, and RxJava for communication between Model & ViewModel.
- Maintained modular structure addressing seperation of concerns.
- Dependency Injection using Dagger2.
- A Service interacts with a Data source (Network, SharedPreferences etc)
- A Repository provides single source of entry to multiple services
The whole project is divided into five top level packages:
- components: Place for all shared activities (like DrawingActivity, MotionDrawingActivity), views (like SquareCardView, SquareImageView) and other stuff (like EqualSpacingItemDecoration).
- data: Contains packages and classes related to retrieving data from various sources. All models, repositories and services are contained in this package. The main WebSocket query handling is done in QueryResultsService. model contains all the model objects for queries & results.
- di: Contains classes used for Dependency Injection
- features: Each sub-package represents a single screen or group of related screens and contains Views and ViewModels for them. Specifically whole app functionality is divided into add media, query, results, result details & settings feature.
- utils: Utility classes (like Extensions containing extension functions)
 |
 |
 |
|---|---|---|
 |
 |
 |
 |
 |
 |