Design Document

Document Revision History

Date	Version	Details
2025/10/05	v1.0	Initial Documentation
2025/10/10	v1.1	Update API Specification
2025/10/14	v1.2	Remove API Specification
2025/10/19	v2.0	Separate PoC page, update diagrams
2025/10/19	v3.0	Update testing plan & diagrams
2025/11/02	v4.0	Update diagrams
2025/11/02	v4.1	Add search & recommendation algorithm
2025/11/16	v5.0	Update front diagram, testing result
2025/11/16	v5.1	Update diagrams
2025/11/24	v6.0	Update system architecture & backend diagram & ER diagram description
2025/11/24	v6.1	Add tech stack

Architecture

MomenTag/
├── backend/      # Django REST API + Celery (Python)
└── android/      # Kotlin + Jetpack Compose

The project follows a client-server architecture with a mobile-first approach.

Server Infrastructure

Server	Components
CPU Server	Django REST API, MySQL Database, Redis
GPU Server	Celery Workers (gpu queue, interactive queue)
Cloud	Qdrant Cloud (Vector DB)

Tech Stack

Backend

Technology	Purpose
Django 5.2 + DRF	REST API Framework
Celery + Redis	Asynchronous task processing
MySQL	Primary database
Qdrant	Vector database for semantic search
Sentence Transformers	Image/text embedding generation
PyTorch	GPU-accelerated ML inference

Android

Technology	Purpose
Kotlin + Jetpack Compose	UI Framework
MVVM + Hilt	Architecture & Dependency Injection
Retrofit + OkHttp	Network layer
Room + DataStore	Local storage
WorkManager	Background processing
Coil	Image loading

API Endpoints

Authentication

Method	Endpoint	Description
POST	/api/accounts/signup/	User registration
POST	/api/accounts/signin/	Login
POST	/api/accounts/signout/	Logout
POST	/api/accounts/token/refresh/	Token refresh

Gallery

Method	Endpoint	Description
POST	/api/gallery/photo/	Upload photos (multipart)
GET	/api/gallery/photo/	Get user's photos
DELETE	/api/gallery/photo/<photo_id>/	Delete photo
POST	/api/gallery/tag/	Create tag
GET	/api/gallery/tag/	List tags
DELETE	/api/gallery/tag/<tag_id>/	Delete tag
POST	/api/gallery/tag/<tag_id>/recommend/	Get photo recommendations
GET	/api/gallery/tag-recommendations/<photo_id>/	Get tag suggestions
GET	/api/gallery/story/	Get story/moment data
POST	/api/gallery/story/	Generate story

Search

Method	Endpoint	Description
GET	/api/search/?query=<query>&offset=<n>	Semantic & hybrid search

Supports tag syntax: query {TagName} more text

System Design

System Architecture

The figure above illustrates the MomenTag architecture, comprising three distinct layers: the Android Frontend, a Distributed Django Backend (CPU/GPU), and Qdrant Cloud.

Android Frontend: Built with Jetpack Compose and Retrofit, it manages the user interface for photo display, uploads, tagging, and retrieving search results.

Distributed Backend:

• CPU Server: Hosts the Django REST Framework API, MySQL database (for storing metadata, tags, and user profiles), and Redis (acting as both cache and Celery message broker).

• GPU Server: Executes Celery workers across two specialized queues to optimize performance:

  • gpu queue: Handles compute-intensive tasks such as image embedding generation (Sentence Transformers) and auto-captioning (Vision-Language models).

  • interactive queue: Processes low-latency tasks like story generation and vector-based recommendations.

• Vector Search (Qdrant Cloud): Stores image embeddings generated by GPU workers, enabling fast and scalable semantic retrieval based on vector similarity.

This service-oriented architecture effectively decouples API handling from ML inference, ensuring high responsiveness and scalability for future expansion.

Class Diagrams and Data Models

Android Class Diagram

This is a class diagram of an Android application designed based on the MVVM (Model–View–ViewModel) architecture. It is structured as follows:

• View

  • MainActivity: Serves as the app’s entry point and sets up the navigation host.
  • Navigation (AppNavigation): A composable that manages navigation between screens using the Navigation component and NavHostController.
  • Screens (HomeScreen, LoginScreen, RegisterScreen, AlbumScreen, LocalGalleryScreen, LocalAlbumScreen, ImageDetailScreen, SearchResultScreen, AddTagScreen, SelectImageScreen, StoryScreen, MyTagsScreen):
    Composable functions that build each UI screen.
    Each screen observes one or more ViewModels and updates its UI reactively based on the corresponding UI state.

• ViewModel

  • AuthViewModel: Manages authentication-related state (login, register, token refresh, logout) using TokenRepository.
  • HomeViewModel: Manages the home screen state including home tags, grouped photos, all photos, selection mode, pagination, and tag deletion.
  • PhotoViewModel: Manages photo upload logic by coordinating between LocalRepository and RemoteRepository.
  • AlbumViewModel: Manages photos filtered by a specific tag (tag album view).
  • LocalViewModel: Manages local device data such as original user images, albums, and photo selection inside a local album via LocalRepository and ImageBrowserRepository.
  • SearchViewModel: Manages semantic search results, tag loading, search history, and photo selection for search results using SearchRepository and other repositories.
  • ImageDetailViewModel: Manages detailed information of a single image and its browsing context (ImageContext) via ImageBrowserRepository.
  • AddTagViewModel: Manages tag creation (with recommendations) and the mapping between a tag and selected photos, including validation and save state.
  • SelectImageViewModel: Manages selecting photos for a tag, loading all photos, pagination, and recommended photos for a given tag.
  • StoryViewModel: Manages the “story” (moment) feature, including loading stories by tag and handling user interactions such as custom tag.
  • MyTagsViewModel: Manages the user’s tag list screen, including edit mode, bulk selection, sorting, tag actions (rename, delete), and associated photo selection.

• Model

  • Repository Layer (RemoteRepository, LocalRepository, TokenRepository, SearchRepository, RecommendRepository, PhotoSelectionRepository, ImageBrowserRepository):
    Abstracts data sources (remote API, local storage, in-memory selection) and exposes high-level operations to the ViewModels. This separates data access logic from UI logic.
  • Data Source / Network Layer
    • ApiService: Retrofit interface that defines HTTP endpoints such as fetching tags, photos by tag, semantic search, uploading photos, authentication, recommendations, and stories.
    • RetrofitInstance: Provides a configured ApiService instance, wiring in AuthInterceptor and TokenAuthenticator.
    • AuthInterceptor & TokenAuthenticator: Handle attaching access tokens to requests and refreshing tokens when needed using SessionStore.
  • Session / Persistence
    • SessionStore (interface) & SessionManager (implementation): Manage access and refresh tokens using DataStore, providing a reactive token store for the rest of the app.
  • Data Classes (Tag, Photo, Album, PhotoMeta, ImageContext, StoryModel, etc.):
    Represent the core domain and network data models used throughout the View and ViewModel layers.

Backend Class Diagram

The backend architecture is organized into three core applications: accounts, gallery, and search. These components operate on shared data models, including User, Photos, Tags, Captions, and their respective linking tables (Photo_Tag, Photo_Caption).

The system runs on a distributed infrastructure where a CPU server hosts the Django REST API, MySQL database, and Redis broker, while a GPU server executes Celery workers across two specialized queues. The gpu queue handles compute-intensive tasks such as image embedding generation and caption creation, while the interactive queue processes latency-sensitive tasks, including story generation, vector computation, and recommendations. Vector embeddings are stored in Qdrant Cloud to ensure scalable semantic search.

• Accounts App: Manages user authentication via JWT, handling sign-up, log-in, log-out, and token refresh operations.

• Gallery App: Orchestrates the primary data flow. It handles batch photo uploads (8 images per batch), generates captions via Vision-Language models, and manages user-defined tags. It ensures search index consistency by refreshing embeddings and graph metadata whenever photos, tags, or captions are modified. Additionally, it powers the "Moments" feature—a scrollable feed with dynamic tag recommendations based on K-means clustering and graph analysis.

• Search App: Facilitates both natural language and personalized tag-based retrieval. It performs semantic similarity searches using Qdrant Cloud and supports hybrid search queries. The {TagName} syntax allows users to combine explicit tag filtering with semantic matching for precise results.

Additional features include automatic caption generation, context-aware tag recommendations, photo suggestions for existing tags, and story generation for rediscovering past memories. Together, these components form a unified architecture that leverages distributed CPU-GPU resources to deliver a scalable and efficient photo management and search experience.

ER Diagram

Data Schema & Storage

MomenTag employs a hybrid storage strategy, utilizing MySQL for relational data and Qdrant Cloud for vector embeddings.

• MySQL Tables

  • User: Manages account credentials (username, password, email).

  • Tag: Stores user-defined semantic labels with creation timestamps.

  • Photo: Records image metadata including MediaStore URIs, filenames, and GPS coordinates. Enforces uniqueness via a (user_id, photo_path_id) constraint.

  • Caption: Stores tokenized captions generated by Vision-Language models.

  • Photo_Tag / Photo_Caption: Junction tables enabling many-to-many relationships. Photo_Caption includes a weight score to quantify the relevance of specific caption tokens to an image.

• Qdrant Cloud Collections

  • Photos: Stores image embeddings (generated by Sentence Transformers) along with denormalized metadata (location, timestamps) to enable efficient, self-contained vector search.

  • RepVec: Maintains representative vectors for tags. These are derived using K-means clustering (for centroids) and Isolation Forest (for outlier detection) to accurately represent the semantic space of a tag.

Synchronization Data consistency between the relational database and the vector store is orchestrated via Celery workers on the GPU server, ensuring that metadata updates in MySQL are immediately reflected in the Qdrant vector index.

Testing Plan

Unit Testing & Integration Testing

Method

• Who
  • Tests for the existing code are conducted by one coder
  • Tests for future code will be conducted by each coder
• When
  • At every PR

Android Frontend

• Unit Test: JUnit
• Integration Test: JUnit + Compose UI Test
• Architecture: MVVM
  • Model: test coverage for all business logic
  • View: test coverage for all UI behavior
  • ViewModel: test coverage for state and data handling

The Android application follows the MVVM architecture. Therefore the logic is concentrated in the Repository and ViewModel layers. For unit testing, we employ JUnit to validate core logic and synchronization between the local and remote repositories. Mock objects and MockWebServer are used for simulating network responses without depending on actual backend communication.

Since we use Jetpack Compose in our project, and composable functions are declarative without business logic, they are not a direct subject of unit test. Instead, integration tests using JUnit and Compose UI Test are performed to ensure that user interactions trigger the expected ViewModel updates and UI state changes correctly. The Compose UI Test enables simulation of user activities such as pressing the button and navigating through pages.

• Results

Backend

• Unit Test: unittest + DRF APITestCase
• Integration Test: DRF APIClient

For backend testing, we use Python’s unittest framework and Django REST Framework’s APITestCase to perform unit tests on models, serializers, and views. For integration test, we use the DRF APIClient to simulate API requests and responses. These tests simulates interactions between models, serializers, and views.

By combining these tests, we ensure data consistency between our relational database and Qdrant, and verify that both systems work properly under realistic client actions.

• Results

Search & Recommendation Algorithms

Preprocessing

To improve upload speed, the original photo's file size is reduced before uploading. Uploaded photos are embedded into vectors using an embedding model. After an initial, one-time scan for similar photos, any duplicate photos are removed, leaving only a single copy. Additionally, when a user adds a tag to a photo while using the app, this tag data is also stored. A caption is generated for each photo once at the time of upload, utilizing an LLM.

Search Algorithms

After distinguishing between tags and natural language within the user's input sentence, the RWR (Random Walk with Restart) algorithm is utilized, leveraging the embedded vectors, captions, and tag data.

Each weight is generated based on the Adamic/Adar index, reflecting factors such as the relationship with tags and vector similarity. This process is then used to conduct the photo search.

Recommendation Algorithms

Image recommendations for an image set are calculated using the RWR (Random Walk with Restart) algorithm, which leverages the embedded vectors, captions, and tag data.

Tag recommendations for an image are calculated using representative vectors.

Acceptance Testing

These are the five user stories for user acceptance testing.

Natural language photo search

As an user with a large photo collection,
I can search photos using natural language,
so that I can quickly find the images I want.

Personalized tag aware natural language photo search

As a user who has photos with personalized tags,
I can search photos based on my personalized tags,
so that I can easily perform searches related to my personalized tags.

Photo recommendation

As an user who wants to add tags to my photos,
I can receive recommendations of similar images even if I tag only a few photos,
so that I can easily make tags with many related photos.

Tag recommendation

As an user enjoying room escape,
I can accept automatically generated tag "room escape" for my new escape room photos,
so that I can add tag to images effortlessly.

Story feature

As a photo-heavy user,
I can skim through recommended images,
so that I can find forgotten photos and add tags to them.