Design Document

Document Revision History

Date	Version	Details
2025/10/05	v1.0	Initial Documentation
2025/10/10	v1.1	Update API Specification
2025/10/14	v1.2	Remove API Specification
2025/10/19	v2.0	Separate PoC page, update diagrams
2025/10/19	v3.0	Update testing plan & diagrams
2025/11/02	v4.0	Update diagrams
2025/11/02	v4.1	Add search & recommendation algorithm
2025/11/16	v5.0	Update front diagram, testing result
2025/11/16	v5.1	Update diagrams
2025/11/24	v6.0	Update system architecture & backend diagram & ER diagram description
2025/11/24	v6.1	Add tech stack
2025/11/25	v6.2	Update ER diagram & terminology changes
2025/11/26	v6.3	Update UAT
2025/11/30	v6.4	Update frontend diagram
2025/11/30	v6.5	Update testing plan
2025/11/30	v6.6	Update search & recommendation algorithms
2025/11/30	v6.7	Update test result

Architecture

MomenTag/
├── backend/      # Django REST API + Celery (Python)
└── android/      # Kotlin + Jetpack Compose

My The project follows a client-server architecture with a mobile-first approach.

Server Infrastructure

Server	Components
CPU Server	Django REST API, MariaDB Database, Redis
GPU Server	Celery Workers (gpu queue, interactive queue)
Cloud	Qdrant Cloud (Vector DB)

Tech Stack

Backend

Technology	Purpose
Django 5.2 + DRF	REST API Framework
Celery + Redis	Asynchronous task processing
MariaDB	Primary database
Qdrant	Vector database for semantic search
Sentence Transformers	Image/text embedding generation
PyTorch	GPU-accelerated ML inference

Android

Technology	Purpose
Kotlin + Jetpack Compose	UI Framework
MVVM + Hilt	Architecture & Dependency Injection
Retrofit + OkHttp	Network layer
Room + DataStore	Local storage
WorkManager	Background processing
Coil	Image loading

API Endpoints

Authentication

Method	Endpoint	Description
POST	/api/accounts/signup/	User registration
POST	/api/accounts/signin/	Login
POST	/api/accounts/signout/	Logout
POST	/api/accounts/token/refresh/	Token refresh

Gallery

Method	Endpoint	Description
POST	/api/gallery/photo/	Upload photos (multipart)
GET	/api/gallery/photo/	Get user's photos
DELETE	/api/gallery/photo/<photo_id>/	Delete photo
POST	/api/gallery/tag/	Create tag
GET	/api/gallery/tag/	List tags
DELETE	/api/gallery/tag/<tag_id>/	Delete tag
POST	/api/gallery/tag/<tag_id>/recommend/	Get photo recommendations
GET	/api/gallery/tag-recommendations/<photo_id>/	Get tag suggestions
GET	/api/gallery/story/	Get story/moment data
POST	/api/gallery/story/	Generate story

Search

Method	Endpoint	Description
GET	/api/search/?query=<query>&offset=<n>	Semantic & hybrid search

Supports tag syntax: query {TagName} more text

System Design

System Architecture

The figure above illustrates the MomenTag architecture, comprising three distinct layers: the Android Frontend, a Distributed Django Backend (CPU/GPU), and Qdrant Cloud.

Android Frontend: Built with Jetpack Compose and Retrofit, it manages the user interface for photo display, uploads, tagging, and retrieving search results.

Distributed Backend:

• CPU Server: Hosts the Django REST Framework API, MariaDB database (for storing metadata, tags, and user profiles), and Redis (acting as both cache and Celery message broker).

• GPU Server: Executes Celery workers across two specialized queues to optimize performance:

  • gpu queue: Handles compute-intensive tasks such as image embedding generation (Sentence Transformers) and auto-captioning (Vision-Language models).

  • interactive queue: Processes low-latency tasks like story generation and vector-based recommendations.

• Vector Search (Qdrant Cloud): Stores image embeddings generated by GPU workers, enabling fast and scalable semantic retrieval based on vector similarity.

This service-oriented architecture effectively decouples API handling from ML inference, ensuring high responsiveness and scalability for future expansion.

Class Diagrams and Data Models

Android Class Diagram

This is a class diagram of an Android application designed based on the MVVM (Model–View–ViewModel) architecture. It is structured as follows:

• Repositories API calls are organized into dedicated repositories (RecommendRepository.kt, RemoteRepository.kt, SearchRepository.kt, TokenRepository.kt) to improve maintainability, facilitate team collaboration, and enhance testability. All the repositories are managed by Hilt as Singletons.

  • ImageBrowserRepository.kt: Manages the session for browsing image lists (e.g., search results, albums).

    • Stores image lists such as search result, tag album, local gallery etc. as a session.
    • Provide context for previous/next image browsing in ImageDetailScreen
    • Prevent data duplication over viewmodels and work as the single source of truth

  • LocalRepository.kt: Manages all interactions with the device's local storage.

    • Retrieves album and photo lists from MediaStore.
    • Reads metadata like photo creation date and location (EXIF).
    • Resizes and compresses images for upload efficiency.
    • Saves and manages recent search history on the device.

  • PhotoSelectionRepository.kt: Manages the state of user-selected photos across various screens.

    • Stores the state of photos selected during the "create new tag" process.

  • RecommendRepository.kt: Manages API calls related to recommendations.

    • Receives tag recommendations suitable for a photo.
    • Receives photo recommendations based on specific tags or other photos.
    • Requests and retrieves moments generation.

  • RemoteRepository.kt: Manages API calls related to Photo and Tag management.

    • Photo Management APIs
    • Tag Management APIs
    • Photo-Tag Relationship Management APIs

  • SearchRepository.kt: Manages semantic search API calls.

    • Performs semantic search API calls with the user's text query to find photos.

  • TokenRepository.kt: Manages API calls for authentication (Sign In, Sign Up, Sign Out) and session management.

    • Handles Access/Refresh tokens with SessionStore, automatically refreshing them when expired.
    • Provides the login status (isLoggedIn) via StateFlow for other parts of the app to observe.

• ViewModels

  • AddTagViewModel.kt: Viewmodel for AddTagScreen.

    • Uses PhotoSelectionRepository to manage the list of selected photos and the new tag's name.
    • Delegates state management to PhotoSelectionRepository
    • Communicates with RemoteRepository to save the new tag and link it to the selected photos.
    • Checks for duplicate tag names against existing tags.

  • AlbumViewModel.kt: Viewmodel for AlbumScreen.

    • Fetches photos associated with a tag from RemoteRepository.
    • Receives photo recommendations related to the current tag via RecommendRepository.
    • Handles actions within the album such as renaming a tag, deleting a tag, adding/removing photos from a tag, and sharing photos.
    • Uses ImageBrowserRepository to set up the image browsing session for photos within the album.

  • AuthViewModel.kt: Viewmodel that manages authentication states and events.

    • Delegates all authentication business logic (login, registration, etc.) to TokenRepository.
    • Observes the login status (isLoggedIn) from TokenRepository to update the UI.

  • HomeViewModel.kt: Viewmodel for HomeScreen.

    • Fetches and displays the user's tag album list and all photos from RemoteRepository.
    • Groups photos by date and implements infinite scrolling.
    • Manages tag sorting, deletion, and multi-selection mode for photos.
    • Triggers moment pre-generation via RecommendRepository.

  • ImageDetailViewModel.kt: Viewmodel for ImageDetailScreen.

    • Retrieves the current image and its browsing context from ImageBrowserRepository.
    • Fetches existing tags and recommended tags for the displayed photo from RemoteRepository and RecommendRepository.
    • Handles actions like adding new tags or removing existing tags from the photo.

  • LocalViewModel.kt: Viewmodel for LocalGalleryScreen and LocalAlbumScreen.

    • Uses LocalRepository to retrieve local albums and photos within them.
    • Manages the selection of local albums or individual photos for upload.

  • MyTagsViewModel.kt: Viewmodel for MyTagsScreen.

    • Fetches all tags and their respective photo counts from RemoteRepository.
    • Handles UI logic for sorting the tag list, editing mode for bulk tag deletion, and renaming/deleting individual tags.

  • PhotoViewModel.kt: A specialized ViewModel responsible for managing background photo uploads.

    • Uses WorkManager to schedule AlbumUploadWorker (for album uploads) or SelectedPhotoUploadWorker (for selected photo uploads). This ensures uploads continue even if the app is in the background.
    • Provides status messages about the upload process (e.g., "Background upload started.").

  • SearchViewModel.kt: Viewmodel for SearchScreen.

    • Manages the state of the search bar that can contain both text and tag "chips".
    • Performs semantic searches by calling SearchRepository.
    • Retrieves and manages user search history from LocalRepository.
    • Provides tag autocomplete suggestions when the user types '#'.

  • SelectImageViewModel.kt: Viewmodel for SelectImageScreen.

    • Primarily interacts with PhotoSelectionRepository to manage the list of selected photos.
    • Displays all of the user's remote photos (fetched from RemoteRepository) in a grid for selection.
    • Can recommend similar photos based on the current selection by calling RecommendRepository.

  • StoryViewModel.kt: Viewmodel for StoryScreen.

    • Fetches moments from backend using RecommendRepository.
    • Converts PhotoResponse to StoryModel with metadata
    • Polls the backend until moments are ready.
    • Manages user interaction with stories, such as selecting tags to add to a photo.
    • Handles the submission of selected tags to the backend.
    • Implements infinite scrolling by pre-fetching the next batch of stories.

• Data Source / Network Layer

  • ApiService: Retrofit interface that defines HTTP endpoints such as fetching tags, photos by tag, semantic search, uploading photos, authentication, recommendations, and stories.
  • RetrofitInstance: Provides a configured ApiService instance, wiring in AuthInterceptor and TokenAuthenticator.
  • AuthInterceptor & TokenAuthenticator: Handle attaching access tokens to requests and refreshing tokens when needed using SessionStore.

• Session / Persistence

  • SessionStore (interface) & SessionManager (implementation) & SessionExpirationManager: Manage access and refresh tokens using DataStore, providing a reactive token store for the rest of the app.

• Data Classes (Tag, Photo, Album, PhotoMeta, ImageContext, StoryModel, etc.): Represent the core domain and network data models used throughout the View and ViewModel layers.

Backend Class Diagram

The backend architecture is organized into three core applications: accounts, gallery, and search. These components operate on shared data models, including User, Photos, Tags, Captions, and their respective linking tables (Photo_Tag, Photo_Caption).

The system runs on a distributed infrastructure where a CPU server hosts the Django REST API, MariaDB database, and Redis broker, while a GPU server executes Celery workers across two specialized queues. The gpu queue handles compute-intensive tasks such as image embedding generation and caption creation, while the interactive queue processes latency-sensitive tasks, including story generation, vector computation, and recommendations. Vector embeddings are stored in Qdrant Cloud to ensure scalable semantic search.

• Accounts App: Manages user authentication via JWT, handling sign-up, log-in, log-out, and token refresh operations.

• Gallery App: Orchestrates the primary data flow. It handles batch photo uploads (8 images per batch), generates captions via Vision-Language models, and manages user-defined tags. It ensures search index consistency by refreshing embeddings and graph metadata whenever photos, tags, or captions are modified. Additionally, it powers the "Moments" feature—a scrollable feed with dynamic tag recommendations based on K-means clustering and graph analysis.

• Search App: Facilitates both natural language and personalized tag-based retrieval. It performs semantic similarity searches using Qdrant Cloud and supports hybrid search queries. The {TagName} syntax allows users to combine explicit tag filtering with semantic matching for precise results.

Additional features include automatic caption generation, context-aware tag recommendations, photo suggestions for existing tags, and story generation for rediscovering past memories. Together, these components form a unified architecture that leverages distributed CPU-GPU resources to deliver a scalable and efficient photo management and search experience.

ER Diagram

Data Schema & Storage

MomenTag employs a hybrid storage strategy, utilizing MariaDB for relational data and Qdrant Cloud for vector embeddings.

• MariaDB Tables

  • User: Manages account credentials (username, password, email).

  • Tag: Stores user-defined semantic labels with creation timestamps.

  • Photo: Records image metadata including MediaStore URIs, filenames, and GPS coordinates. Enforces uniqueness via a (user_id, photo_path_id) constraint.

  • Caption: Stores tokenized captions generated by Vision-Language models.

  • Photo_Tag / Photo_Caption: Junction tables enabling many-to-many relationships. Photo_Caption includes a weight score to quantify the relevance of specific caption tokens to an image.

• Qdrant Cloud Collections

  • Photos: Stores image embeddings (generated by Sentence Transformers) along with denormalized metadata (location, timestamps) to enable efficient, self-contained vector search.

  • RepVec: Maintains representative vectors for tags. These are derived using K-means clustering (for centroids) and Isolation Forest (for outlier detection) to accurately represent the semantic space of a tag.

Synchronization Data consistency between the relational database and the vector store is orchestrated via Celery workers on the GPU server, ensuring that metadata updates in MariaDB are immediately reflected in the Qdrant vector index.

Testing Plan

Unit Testing & Integration Testing

Method

• Who
  • Tests for the existing code are conducted by one coder
  • Tests for future code will be conducted by each coder
• When
  • At every PR

Android Frontend

• Unit Test: JUnit
• Integration Test: JUnit + Compose UI Test
• Architecture: MVVM
  • Model: test coverage for all business logic
  • View: test coverage for all UI behavior
  • ViewModel: test coverage for state and data handling

The Android application follows the MVVM architecture. Therefore the logic is concentrated in the Repository and ViewModel layers. For unit testing, we employ JUnit to validate core logic and synchronization between the local and remote repositories. Mock objects and MockWebServer are used for simulating network responses without depending on actual backend communication.

Since we use Jetpack Compose in our project, and composable functions are declarative without business logic, they are not a direct subject of unit test. Instead, integration tests using JUnit and Compose UI Test are performed to ensure that user interactions trigger the expected ViewModel updates and UI state changes correctly. The Compose UI Test enables simulation of user activities such as pressing the button and navigating through pages.

• Results
  • unit test

• integration test

Backend

• Unit Test: unittest + DRF APITestCase
• Integration Test: DRF APIClient

For backend testing, we use Python’s unittest framework and Django REST Framework’s APITestCase to perform unit tests on models, serializers, and views. For integration test, we use the DRF APIClient to simulate API requests and responses. These tests simulates interactions between models, serializers, and views.

By combining these tests, we ensure data consistency between our relational database and Qdrant, and verify that both systems work properly under realistic client actions.

• Test Files Structure: There is a separate test folder in each app directory containing test files.

backend/
├── accounts/
│   ├── test/
│   │   ├── __init__.py
│   │   ├── test_models.py
│   │   ├── test_serializers.py
│   │   └── test_views.py
│   └── ...
├── gallery/
│   ├── test/
│   │   ├── __init__.py
│   │   ├── test_delete_db_duplicate.py
│   │   ├── test_gpu_tasks.py
│   │   ├── test_models.py
│   │   ├── test_qdrant_utils.py
│   │   ├── test_serializers.py
│   │   ├── test_storage_service.py
│   │   ├── test_tasks.py
│   │   ├── test_views.py
│   │   └── test_vision_services.py
│   └── ...
└── search/
    ├── test/
    │   ├── __init__.py
    │   ├── test_embedding_service.py
    │   └── test_views.py
    └── ...

• Results

Search & Recommendation Algorithms

Preprocessing

To improve upload speed, the original photo's file size is reduced before uploading. The preprocessing stage prepares the data through two main paths:

1. Embedding: The image is passed through an embedding model to convert it into a vector.
2. Captioning: An LLM is utilized to generate a descriptive caption for the image. These vectors and captions serve as the foundational data for the search and recommendation systems.

Search Algorithms

The search mechanism distinguishes between Tags and Natural Language (NL) within the user's input. The final search results are presented by sorting images based on a Total Score, which is calculated by summing three components:

1. Tag Score: Images similar to the tags are selected via Qdrant with a similarity score (0–1). To handle multiple tags effectively, the similarities are multiplied, and a weight of 2^(n-1) (where n is the number of tags) is applied. This prevents score degradation from multiplying decimals and emphasizes multi-tag matches. A minimum similarity of 0.1 is enforced to prevent zero values.
2. NL Score: The non-tag portion of the input is embedded, and similarity scores (0–1) are assigned based on vector proximity to stored images.
3. Bonus: If the caption parsed from the NL input matches the image's stored caption, a 0.1 bonus is added to the score.

Recommendation Algorithms

Image recommendations are calculated using Qdrant's Images-to-Images Recommendation. This system analyzes a set of input images and identifies semantically similar images from the vector space to provide recommendations.

Tag recommendations for an image are calculated using representative vectors. These representative vectors are generated by performing k-means clustering on the photos associated with each tag, allowing the system to suggest the most relevant tags based on visual clusters.

Acceptance Testing

These are the five user stories for user acceptance testing.

Authentication (Sign Up/In)

User Stories

• As a first-time user, I see the Sign Up/In screen when the app launches so I can create an account or log in.
• As a first-time user, I can enter a username and password to register a new account.
• As a registered user, I can tap Sign In to access my MomenTag App.

Test Cases

Test Data	Precondition (Given)	User Action (When)	Pos / Neg	Acceptance Criteria (Then)
None	App launches	User views the screen	Positive	username and password fields and a Sign Up button are displayed
Valid username and password	Sign Up screen is displayed	Enter valid username and password and tap Sign Up	Positive	A new account is created and the app returns to the Login screen
Same username and password used for Sign Up	Completed Sign Up and on Login screen	Enter username and password and tap Sign In	Positive	User is authenticated and navigated to the Main Gallery Page
Invalid email or empty fields	Sign Up screen is displayed	Enter invalid data and tap Sign Up	Negative	Error message "Please check your input" is shown and remain on the Sign Up screen

---

Permissions & Upload

User Stories

• As an authorized user, I see a system permission request so I can allow the app to access my device photos.
• As an authorized user, I can tap Allow (or Select Photos) and the Home Screen displays a prompt guiding me to upload or select images (the app does not automatically start indexing).
• As an authorized user, I want to tap the Upload Photos button(or cloud upload icons) to select images or albums, So that I can store my pictures in the app and receive a confirmation notification indicating the upload is complete.

Test Data	Precondition (Given)	User Action (When)	Pos / Neg	Acceptance Criteria (Then)
None	Signed in for the first time	Main screen attempts to load	Positive	System pop-up requesting Storage/Gallery Access is shown
None	Permission pop-up is displayed	Tap Allow or Select Photos	Positive	Home Screen displays a prompt guiding the user to upload or select images
None	User is signed in and on the Home Screen	Tap Upload Photos button (or Upload Icon)	Positive	Navigate to the Local Albums screen listing device folders (e.g., Camera, Screenshot)
Album Name: "Camera"	User is on the Local Albums screen	Tap a specific album card (e.g., Camera)	Positive	Navigate to Local Album View displaying a grid of photos within that album
Selected Photo(s)	User is on Local Album View	Select photo(s) and tap the Upload/Confirm action	Positive	1. System processes the upload. 2. A System Notification appears in the status bar. 3. Expanding the notification shows: "All Photos uploaded successfully".
Network Error	User is on Local Album View with no internet connection	Select photo(s) and tap Upload	Negative	A error notification appears (e.g., "Upload failed. Please checks your connection").

---

HomeScreen ( Search , Photos View)

User Stories

• As a user, I can see a Search Bar at the top to search my photos.
• As a user, I can browse my uploaded images in a photo grid.
• As a user, I can tap the Search button to run a natural language search.

Test Cases

Test Data	Precondition (Given)	User Action (When)	Pos / Neg	Acceptance Criteria (Then)
None	On the Main Gallery Page	Look at the top of the screen	Positive	Text input with placeholder Search your moments (e.g., Cat sleeping) is visible
Natural language query (e.g., "Photos of my cat sleeping")	Search Bar is visible	Enter query and tap Search icon	Positive	Photo grid updates to show relevant images sorted by relevance
Photo tagged Exam study	Photo shows desk but tagged Exam study	Search for Exam study	Positive	Photo appears in results (manual tag prioritized)
Tag name (e.g., #Room Escape)	Search Bar is visible, user has tagged photos	Type #tagname, select tag chip, add additional query	Positive	Photos with the tag and matching additional query are shown

---

Image Detail & Tagging

User Stories

• As a user, I can tap a photo to open its Detail View.
• As a user, I can add a manual tag via a + button next to tags.
• As a user, I can accept AI-recommended tags shown in the tag list.

Test Cases

Test Data	Precondition (Given)	User Action (When)	Pos / Neg	Acceptance Criteria (Then)
None	On the Main Gallery Page	Tap a photo	Positive	App navigates to full-screen Detail View for that image
Keyword (e.g., Room Escape)	In Detail View	Tap +, enter keyword and confirm	Positive	Tag is attached to the image
AI-suggested exisiting tags (e.g., Hobby, Puzzle)	System has analyzed the image	Tap on a suggested tag	Positive	Recommended tag is added to the image

---

Image Recommendation

User Stories

• As a user, I see Similar Images when adding a tag so I can apply the tag to multiple photos at once.
• As a user, I can tap recommended images to include them in the selection.

Test Cases

Test Data	Precondition (Given)	User Action (When)	Pos / Neg	Acceptance Criteria (Then)
None	Adding a tag to a photo	Add Tag modal opens	Positive	List of visually similar or temporally related images is presented
None	Similar images are displayed	Select one or more and confirm tag addition	Positive	New tag is applied to all selected images simultaneously

---

Moment Feature

User Stories

• As a photo-heavy user, I can rediscover forgotten moments via a Moment card that features a recommended photo.
• As a user, I see four suggested tags to help quick categorization.
• To tag a Moment, I can select one or more recommended tags or create a custom tag, then tap Done to apply tags and advance to the next Moment.
• To edit a previous Moment, I can swipe down (top-to-bottom) to move to the previous Moment and tap Edit to modify attached tags.
• To skip tagging a Moment, I can swipe up (bottom-to-top) to immediately advance to the next Moment without adding tags.

Test Cases

Test Data	Precondition (Given)	User Action (When)	Pos / Neg	Acceptance Criteria (Then)
None	On the Main Page or MyTags tab	Open Moment Tab	Positive	Single recommended photo (Moment) with 4 suggested tags is shown
Selected tags or custom tag	Viewing a Moment card	Select tags and tap Done	Positive	1. Tags are saved to current photo. 2. Auto-advance to the next photo in the queue immediately.
None	Viewing Moments	Swipe down to previous Moment and tap Edit	Positive	Can modify attached tags and save changes
None	Viewing a Moment card	Swipe up	Positive	Current Moment is skipped and next Moment appears (no tags added)