Risks

Problem Statement:

Organizations that have large scale network operations often struggle to find a real time simulation engine that allows them to test their system under realistic, dynamic conditions. Weather, traffic and a bunch of other factors play vital roles in properly simulating the real world and having accurate results. Existing tools often lack the realism organizations want, and other open source projects make it extremely difficult to represent geographic data accurately. This makes it harder to test their dispatching strategies as well as visualize how their network is operating. Velosim addresses all these problems by providing a real time simulation engine that that leverages Open Street Map Data (OSM) to accurately represent real life. However this comes with risk and challenges!

Risks:

The most significant risk the team will be facing is highly losing performance due to working with a considerable amount of data. The team will be working with large datasets to make the simulation realistic, such as a subset of BIXI's internal data and OpenStreetMap (OSM) data which includes all streets and intersections within the network of Montreal. As we will be heavily manipulating the OSM data to create graphs and routes within them, numerous analyses and iterations will need to be executed through it. This could lead to long computation times which could overload the CPU and buffer the application. In addition, since most of the data is stored in remote servers, performance will also depend on the network bandwidth. By requesting a huge load of data, the network could get congested as the data would need to be split and sent through multiple packets which could overload the buffer. This could then lead to loss of data and slow data retrieval. When performance becomes too slow, the application could seem unresponsive and could ruin the user experience. Users do not wish to use an application that continuously lags or has excessively long loading times. As a result, the loss of performance caused by the usage of a large amount of data could potentially kill our project if not mitigated.

To start attacking this risk, we decided to reduce the amount of network dependencies to manipulate the OSM data. As a result, the OSM data will be fetched once at initialization of the application and stored as a dump file to allow quick parsing of its values using the pyrosm library as discussed in issue #31. Although this improved the performance and reduced the risks of network congestion, it still took quite some time to create the graph and get a route which would be frequently requested. Hence, this was resolved by implementing a Contraction Hierarchy for quicker shortest path calculations and by introducing a KDTree algorithm for finding the nearest neighbor implemented in PR #273 within the OSMConnection.py file.

More details on how we improved and will improve in the future the performance of our application can be found in the Performance wiki page.