Missing knowledge and Independent Learning

Most of the team had never implemented a simulation. As a result, this project required a considerable amount of independent learning to resolve the missing knowledge the team had. The team completed rigorous researching of different tech stacks and examples of simple implementations to learn different methods and decide on the best options. The final decisions are documented in the Infrastructure and Tools Wiki page. Here follows a list of missing knowledge the team had to independently learn for this project:

Release 1

• How to run a simulation: Research of various technologies and solutions was conducted. After analyzing a few examples, a test repository was created to experiment with a simple simulation. Once a good result was implemented, it was presented to the rest of the team to ensure everyone had a good understanding on how it works.

• SimPy discrete-event simulation framework: None of the members of the simulation team had prior experience with SimPy. A lot of research was conducted before starting any simulation work, especially to understand SimPy's process-based approach and core concepts such as its environment-based time progression through reading documentation and experimenting.

• OSM data usage and routing: Some team members had indirectly worked previously with OSM data through Mapbox. However, none had directly used it. Since we wanted to minimize the amount of dependencies through APIs, we needed to learn to use OSM data directly for routing. Research was then done to determine the best tool for reading OSM data and abstracting it for the routing implementation through this OSM issue commit. To ensure functionality of both the OSM data and routing, multiple tests were executed in the terminal to validate returned values and functionalities.

• Usage of WebSockets: Similarly to how to run a simulation, not many members had worked with WebSockets previously. Hence, research was conducted to learn how to make proper use of them for our software. A test repository was then created to experiment with a simple WebSocket application which was presented to the team. Later during implementation, challenges arose in connecting the WebSocket subscriber to the simulation package, which uses SimPy on threads alongside the asyncio library for concurrency. An initial WebSocket streaming structure existed here and was later extended and adjusted in subsequent PRs. While time progression emitted correctly, resource position updates were missing due to improper handling of simulator instances and asynchronous frame emissions. The issue was resolved by assigning each simulation a dedicated Simulator instance in the active simulations registry and refining the async WebSocket loop. The .../simulation/stream/{sim_id} endpoint was also updated with WebSocket-compatible authentication to complete real-time data streaming. In short, this entire process provided valuable experience in managing asynchronous WebSocket communication with threaded simulation environments.

• Feature Toggles (aka Feature Flags): One team member initially faced challenges with how to disable or hide large portions of in-progress backend API code (hundreds of lines tied to endpoints not yet ready for deployment) without deleting or breaking existing tests. In exploring solutions, they came across feature toggles and helpful resources like Martin Fowler's 2017 blog post and YouTube videos by Web Dev Cody (1, 2). What made the resources so helpful was their demonstration of real-world use cases, like toggling API endpoints in a production environment if they begin to cause unexpected performance issues. In parallel, another team member introduced a frontend-facing feature toggle system. This implementation now made it possible for React components to check feature states and enabling local overrides via sessionStorage for testing different flag configurations. Together, these feature toggles established in Release 1 formed a solid foundation for controlled development/testing and handling system behavior in a production environment.

• Optimized Routing Algorithms: In COMP 352 Data Structures and Algorithms course, when the chapter of graph ADTs was introduced, the classical way to determine the shortest path was to use Dijkstra's algorithm. This is okay for small to medium graphs, however in the scope of the simulator, we have hundreds of thousands of nodes and edges to represent the island of Montreal. Hence, since it is not plausible to use an external API (due to API usage limitations), a local solution must be implemented. By utilizing Dijkstra, it would take a few seconds to calculate the path. However, this doesn't scale, since if the simulator is calculating many routes, it can take minutes to update anything. Hence, two forms of algorithms were needed to be implemented, which was contraction hierarchies (reference), which allows building transitive edges, which collapses the number of edges, speeding up Dijkstra's algorithm, as well as encoding the nodes themselves to be searchable very fast (when finding the nearest node when trying to locate a position on a map) using k-d trees (reference). Given both of these, performance has increased by 100x. More in the performance section. Implemented in #273.

Release 2

• FastAPI Endpoint Performance considerations: During backend development, the team initially implemented several FastAPI endpoints using async def while relying on synchronous SQLAlchemy database sessions. This led to a deeper investigation into how FastAPI handles concurrency. Through Release 1 "performance considerations" feedback, the team learned that using blocking database operations inside async endpoints can block the event loop, preventing other requests from being processed concurrently. Since FastAPI assumes async endpoints will only perform non-blocking operations, this design caused potential scalability and performance issues. To resolve this, the team learned two valid architectural approaches: either fully adopt asynchronous database drivers and async session handling, or convert endpoints to synchronous def functions so FastAPI can execute them safely in a threadpool. Given the existing synchronous database setup, the latter approach was chosen. Multiple endpoints were converted to synchronous handlers, ensuring proper concurrency without event loop blocking. This experience significantly improved the team’s understanding of asynchronous programming models and FastAPI’s execution behaviour.

• Usability Testing: While the team had prior exposure to usability testing through SOEN 390 (Software Engineering Team Design Project), that experience took place in a controlled academic environment where all teams evaluated the same application and gathered feedback from a general pool of users. In contrast, VeloSim is a domain-specific system designed primarily for BIXI dispatchers, requiring a more rigorous and context-aware usability testing approach. As a result, the team needed to independently learn how to design, conduct, and analyze usability testing for a specialized user group rather than a generalized audience. To guide this process, the team turned to established industry best practices, most notably those documented in Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests (Second Edition) by Jared Spool, Dana Chisnell, and Jeffrey Rubin. A key lesson learned was the distinction between testing with domain experts and non-expert participants. While friends and family could be used to evaluate general usability, clarity, and interface affordances, feedback from domain experts carried significantly higher weight for validating workflows, terminology, and mental models. This reinforced the need to carefully script test introductions and tasks so that non-expert participants could reasonably simulate dispatcher behavior without contaminating results through guesswork or misunderstanding of domain concepts. The team also gained experience applying moderated usability testing techniques, namely the think-aloud protocol. Rather than just relying on logs, having screen+audio recordings and observing 31 users attempt real tasks in the production environment revealed usability issues that would not have surfaced through internal testing alone (such as: missing affordances, unclear system feedback, mismatches between user expectations and system behaviour, and discoverability problems in the simulation view and scenario editor.) Finally, the team learned the importance of translating usability findings into concrete, actionable outcomes. Rather than treating usability testing as purely evaluative, results and feedback have been incorporated into the GitHub Projects work plan, spanning features, enhancements, and bug fixes. The full details can be found on our Usability Testing Wiki page.

• Research related to the routing, traffic engine, TAMM can be found here.