Design Document

Document Revision History

Date	Version	Details
2025/10/05	v1.0	Initial Documentation
2025/10/10	v1.1	Update API Specification
2025/10/14	v1.2	Remove API Specification
2025/10/19	v2.0	Separate PoC page, update diagrams
2025/10/19	v3.0	Update testing plan & diagrams
2025/11/02	v4.0	Update diagrams
2025/11/02	v4.1	Add search & recommendation algorithm
2025/11/16	v5.0	Update front diagram, testing result
2025/11/16	v5.1	Update diagrams

System Design

System Architecture

The figure above illustrates the overall architecture of MomenTag, which is composed of three layers: the Android frontend, the Django REST backend, and several external hosts used for vector search and embedding generation.

The frontend, built with Android and Jetpack Compose, communicates with the backend using Retrofit as the HTTP client. It handles user interactions such as displaying photos, uploading images, adding tags, and retrieving semantic search results.

On the backend host, the application runs on a containerized environment and uses the Django REST Framework as the main web server for processing client requests and managing user data. MariaDB is used as the relational database for storing user profiles, authentication information, and various metadata. MinIO provides bucket storage for managing original images. Redis functions both as a cache and as the message broker for Celery, which is responsible for distributing asynchronous tasks such as embedding generation and vector insertion.

External hosts are utilized for AI-powered features. When a user uploads an image or submits a text query, the backend calls an embedding service (using a Vision Transformer model) to generate embedding vectors. These vectors, along with image metadata, are stored in Qdrant, a high-performance vector database optimized for similarity search. The backend then queries Qdrant to efficiently retrieve semantically relevant images.

This modular and service-oriented architecture enables efficient communication between components, supports fast vector search, and maintains scalability for future feature expansion.

Class Diagrams and Data Models

Android Class Diagram

This is a class diagram of an Android application designed based on the MVVM (Model–View–ViewModel) architecture. It is structured as follows:

• View

  • MainActivity: Serves as the app’s entry point and sets up the navigation host.
  • Navigation (AppNavigation): A composable that manages navigation between screens using the Navigation component and NavHostController.
  • Screens (HomeScreen, LoginScreen, RegisterScreen, AlbumScreen, LocalGalleryScreen, LocalAlbumScreen, ImageDetailScreen, SearchResultScreen, AddTagScreen, SelectImageScreen, StoryScreen, MyTagsScreen):
    Composable functions that build each UI screen.
    Each screen observes one or more ViewModels and updates its UI reactively based on the corresponding UI state.

• ViewModel

  • AuthViewModel: Manages authentication-related state (login, register, token refresh, logout) using TokenRepository.
  • HomeViewModel: Manages the home screen state including home tags, grouped photos, all photos, selection mode, pagination, and tag deletion.
  • PhotoViewModel: Manages photo upload logic by coordinating between LocalRepository and RemoteRepository.
  • AlbumViewModel: Manages photos filtered by a specific tag (tag album view).
  • LocalViewModel: Manages local device data such as original user images, albums, and photo selection inside a local album via LocalRepository and ImageBrowserRepository.
  • SearchViewModel: Manages semantic search results, tag loading, search history, and photo selection for search results using SearchRepository and other repositories.
  • ImageDetailViewModel: Manages detailed information of a single image and its browsing context (ImageContext) via ImageBrowserRepository.
  • AddTagViewModel: Manages tag creation (with recommendations) and the mapping between a tag and selected photos, including validation and save state.
  • SelectImageViewModel: Manages selecting photos for a tag, loading all photos, pagination, and recommended photos for a given tag.
  • StoryViewModel: Manages the “story” (moment) feature, including loading stories by tag and handling user interactions such as custom tag.
  • MyTagsViewModel: Manages the user’s tag list screen, including edit mode, bulk selection, sorting, tag actions (rename, delete), and associated photo selection.

• Model

  • Repository Layer (RemoteRepository, LocalRepository, TokenRepository, SearchRepository, RecommendRepository, PhotoSelectionRepository, ImageBrowserRepository):
    Abstracts data sources (remote API, local storage, in-memory selection) and exposes high-level operations to the ViewModels. This separates data access logic from UI logic.
  • Data Source / Network Layer
    • ApiService: Retrofit interface that defines HTTP endpoints such as fetching tags, photos by tag, semantic search, uploading photos, authentication, recommendations, and stories.
    • RetrofitInstance: Provides a configured ApiService instance, wiring in AuthInterceptor and TokenAuthenticator.
    • AuthInterceptor & TokenAuthenticator: Handle attaching access tokens to requests and refreshing tokens when needed using SessionStore.
  • Session / Persistence
    • SessionStore (interface) & SessionManager (implementation): Manage access and refresh tokens using DataStore, providing a reactive token store for the rest of the app.
  • Data Classes (Tag, Photo, Album, PhotoMeta, ImageContext, StoryModel, etc.):
    Represent the core domain and network data models used throughout the View and ViewModel layers.

Backend Class Diagram

The backend system is organized into two primary applications, the gallery app and the search app, which operate on shared data models including User, Photos, Tags, Captions, RepVec, and the linking tables Photo_Tag and Photo_Caption.

The gallery app manages all photo related data flow. It handles photo uploads, generates captions through a language model, assigns or removes user created tags, and updates auxiliary structures such as representative vectors and graph metadata. Whenever photos, tags, or captions are modified, the gallery app refreshes embeddings and relational data to keep the search layer consistent and accurate.

The search app is responsible for both natural language search and personalized tag based search. It interprets user queries semantically, performs similarity search using embeddings, and applies user specific tag representations to return results that reflect each user’s tagging behavior and preferences.

Additional features include automatic caption creation for uploaded photos, tag name recommendations, and photo suggestions for existing tags.

Overall, these components form a unified backend architecture that integrates photo management, metadata generation, semantic modeling, and personalized retrieval to provide a scalable and efficient photo search experience.

ER Diagram

MomenTag stores the User, Tags, Photo_Tag, Captions, and Photo_Caption tables in MySQL as an RDB, while the Photos and RepVec collections are stored in Qdrant DB.

MomenTag stores the User, Tag, Photo, Photo_Tag, Caption, and Photo_Caption tables in MySQL, while the Photos and RepVec collections live in Qdrant.

The User table holds each account’s username, password, and email with id as the primary key. The Tag table records semantic labels per user to support flexible search and association.

The Photo table stores metadata for each uploaded image: photo_id (PK), the uploader’s user_id (FK), MediaStore photo_path_id for reconstructing the Android MediaStore URI, filename, creation timestamp, GPS coordinates, and an is_tagged flag. A per-user uniqueness constraint on (user_id, photo_path_id) prevents duplicate entries for the same physical image.

Photo_Tag is the junction table enabling the many-to-many relationship between photos and tags, and Photo_Caption plays the same role between photos and caption tokens. Both tables include user_id so the backend can track who attached each tag or caption.

Caption stores the LLM-generated photo captions in tokenized form, where each row represents one word-level token (CharField(50)).

In Qdrant, the Photos collection stores each photo’s embedding vector along with denormalized metadata required for vector search: photo_id as the primary key, user_id of the uploader, photo_path_id for content linkage, created_at, and location_lat/lng. This design allows the embedding store to remain self-contained while keeping MySQL as the source of truth for relational constraints.

The RepVec collection contains representative vectors per tag: for each tag, K-means clustering is applied over its associated photo embeddings, and the centroid vectors of the k clusters are stored. Additional representative vectors are also saved when an outlier group is detected.

Because data is split between the relational store and the vector store, MomenTag must coordinate both sides carefully. However, this hybrid architecture provides reliable relational integrity for user accounts, tags, and metadata, while enabling fast and adaptive semantic retrieval through vector similarity search.

Testing Plan

Unit Testing & Integration Testing

Method

• Who
  • Tests for the existing code are conducted by one coder
  • Tests for future code will be conducted by each coder
• When
  • At every PR

Android Frontend

• Unit Test: JUnit
• Integration Test: JUnit + Compose UI Test
• Architecture: MVVM
  • Model: test coverage for all business logic
  • View: test coverage for all UI behavior
  • ViewModel: test coverage for state and data handling

The Android application follows the MVVM architecture. Therefore the logic is concentrated in the Repository and ViewModel layers. For unit testing, we employ JUnit to validate core logic and synchronization between the local and remote repositories. Mock objects and MockWebServer are used for simulating network responses without depending on actual backend communication.

Since we use Jetpack Compose in our project, and composable functions are declarative without business logic, they are not a direct subject of unit test. Instead, integration tests using JUnit and Compose UI Test are performed to ensure that user interactions trigger the expected ViewModel updates and UI state changes correctly. The Compose UI Test enables simulation of user activities such as pressing the button and navigating through pages.

• Results

Backend

• Unit Test: unittest + DRF APITestCase
• Integration Test: DRF APIClient

For backend testing, we use Python’s unittest framework and Django REST Framework’s APITestCase to perform unit tests on models, serializers, and views. For integration test, we use the DRF APIClient to simulate API requests and responses. These tests simulates interactions between models, serializers, and views.

By combining these tests, we ensure data consistency between our relational database and Qdrant, and verify that both systems work properly under realistic client actions.

• Results

Search & Recommendation Algorithms

Preprocessing

To improve upload speed, the original photo's file size is reduced before uploading. Uploaded photos are embedded into vectors using an embedding model. After an initial, one-time scan for similar photos, any duplicate photos are removed, leaving only a single copy. Additionally, when a user adds a tag to a photo while using the app, this tag data is also stored. A caption is generated for each photo once at the time of upload, utilizing an LLM.

Search Algorithms

After distinguishing between tags and natural language within the user's input sentence, the RWR (Random Walk with Restart) algorithm is utilized, leveraging the embedded vectors, captions, and tag data.

Each weight is generated based on the Adamic/Adar index, reflecting factors such as the relationship with tags and vector similarity. This process is then used to conduct the photo search.

Recommendation Algorithms

Image recommendations for an image set are calculated using the RWR (Random Walk with Restart) algorithm, which leverages the embedded vectors, captions, and tag data.

Tag recommendations for an image are calculated using representative vectors.

Acceptance Testing

These are the five user stories for user acceptance testing.

Natural language photo search

As an user with a large photo collection,
I can search photos using natural language,
so that I can quickly find the images I want.

Personalized tag aware natural language photo search

As a user who has photos with personalized tags,
I can search photos based on my personalized tags,
so that I can easily perform searches related to my personalized tags.

Photo recommendation

As an user who wants to add tags to my photos,
I can receive recommendations of similar images even if I tag only a few photos,
so that I can easily make tags with many related photos.

Tag recommendation

As an user enjoying room escape,
I can accept automatically generated tag "room escape" for my new escape room photos,
so that I can add tag to images effortlessly.

Story feature

As a photo-heavy user,
I can skim through recommended images,
so that I can find forgotten photos and add tags to them.