2. Model and Engine
The User Frontend written with Kotlin manages the client-side experience of TripView. To view details about how TripView visually presents information and navigates, go to our View UI/UX section.
When a user views a trip and receives the list of stops for that trip from the LLM/backend, the frontend uses the Google Maps SDK to plot each stop as a point on the map to show to the user.
Our frontend communicates with other parts of the architecture such as the database and LLM by going through the Backend written with Python and Flask, which handles requests from the frontend and performs all business logic and interactions with other backend services before sending responses back to the frontend. This communication is encapsulated in ApiClient.kt. !TODO: LINK TO APIs and Controllers! Typically frontend data is serialized into JSON objects in requests as well as multipart forms for some image-based endpoints, while almost all backend data responses are serialized into JSON objects.
The backend uses a database for storing user and trip information. The backend communicates with this database using the psycopg2 package for Python.
LLM API refers to the API for our LLM of choice (built off gemini-2.5-flash) !TODO: LINK!. The backend is responsible for making requests to and handling responses from the LLM. The primary function for the LLM is to generate itineraries. To do this, a user fills out a form with their trip details on the frontend when they go to create a new trip. This information is then passed from the frontend to the backend in the ApiClient::submitForm !TODO: LINK! function to access the $BASE_URL/trips/send_form !TODO: LINK! endpoint, where the user form data is formatted by the backend for the LLM to use in the model.generate_content function which hits the /generateContent endpoint from Gemini !TODO: LINK!, alongside information the backend has about the user's preferences. The LLM then generates 3 itinerary choices into a predefined JSON schema for data consistency.
After a valid JSON object representing the trip itinerary choices is generated, the LLM is called again using the generated itineraries to provide high level trip information to help guide user choice, including total cost estimate, cost breakdown, transportation summary, and transportation breakdown, achieved through a second chained invocation of the model.generate_content function. Once again this data is returned into a valid JSON object. Finally the backend returns a json object that is used in the front end to construct the trip suggestions objects described in the UI/UX. !TODO: LINK!
The LLM is also used if the user wants to interactively ask the LLM questions about their trips dynamically. To do this the user enters a trip specific chat window facilitated via the ChatViewModel::ChatViewModelFactory ViewModel constructor and enters chats which are forward to the backend using the ApiClient:: in which case the backend formats a prompt with their trip information and question to provide context-informed responses.
The LLM also performs landmark identification by taking an image given by the user as well as location information from their phones location sensor. The image of the landmark is saved locally to the user's device and the text returned by the LLM as well as a uuid referring to the stored image is stored in the database for retrieval later.
Getting information about stops such as their rating, price range, hours, address, and Google Maps link is achieved by using the Google Places (New) API. Whenever a stop object is created in the database by a network request, the backend request handling queries this API using the stop's latitude and longitude and receives information about that stop.
In our skeletal product, the backend returns a polyline containing straight lines between all stops on the route to be visualized on the map. In the MVP, the backend calls the OSRM API to get a polyline consisting of the driving route between all points.