You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In an effort to improve access to financial services to the population in Uganda, HOT has been participating in a project attempting to map a different kind of financial service providers in Uganda. Financial service providers are entities that provide services like deposits, credit, transfers, withdrawals, foreign exchange etc, both in the formal and the informal sectors. In Uganda, examples are banks, ATMs, microfinance institutions, mobile money agents and Savings and credit co-operatives (SACCOs). Mapping them and viewing them through different thematic maps will allow different stakeholders such as the financial service providers themselves, the general public, regulators, donors and others to identify what areas are underserved or where there is potential for rolling out new coverage. The thematic analysis is built into the OSMA framework, and as such also provides an example of how thematic analysis can be added to OSMA.
Overview
HOT chose four thematic views which each answer a specific question that financial service providers might ask. The main goal was to create an analysis tool that can provide a basis for decision-making, mainly for financial service providers to see where there could be a potential market (population density and economic activity), where their competitors are active or where there are areas that do not have existing FSPs. The same approach can be chosen for other thematic analysis; however, it is crucial that the information showed on the maps is already providing a certain degree of analysis when it loads. The analysis can then be refined or adjusted using the controls and parameters available. It was somehow challenging to find a good balance between providing sufficient analysis for the maps to be good decision-making tools, but at the same time not be too complex for the users to understand.
Thematic analysis and context data
There is great potential for thematic analysis using the data from OSM. Our technical proposal outlines how we made improvements in OSMA to allow for easier thematic analysis. In addition, good knowledge of the OSM keys is necessary in order to know what data could be relevant for the given user group. We also chose to introduce some external datasets to provide context, such as Worldpop.
However, there is a trade-off between using external datasets and data available in OSM. Population data can be re-downloaded from Worldpop and updated in case there is newer data.
For economic analysis, we chose to use a composite of data available in OSM in order to avoid the need for updating. After some trial and error, we used the following keys to show economic analysis
Number of buildings
Distance from the main road
Number of people (This came from Worldpop)
This is what gave the best result in Uganda, however, note that it might not give the same result in other areas due to the different nature of data available there. Other than that, we encountered the following challenges with the data for financial service providers available in OSM:
Clear conventions were not followed during data collection: example:
The operator property on some of the banks was empty while in other cases, some bank names were saved in short form, i.e. DTB was used in many cases instead of Diamond Trust Bank and this means that not all Diamond Trust Bank branches are rendered when you select it since some are saved under DTB.
Mobile money agents are do not necessarily belong to an operator and the majority have services of up to 3 telecom companies. This made it quite hard to do comparison on mobile operators
Data for financial service providers in Uganda is predominantly available in Eastern Uganda and Kampala because this is where the data collection exercises have taken place. For now, this, therefore, gives a distorted picture of the reality on the ground. We are hoping that as the data collection efforts continue, this will improve.
While there are over 5000 Tags in OpenStreetMap, currently, there are only 3 tags configured for analysis in OSMA. There could be several reasons but the major cause is the fact that the process of adding a new Tag is quite complex and fragile. One would have to go through 5 step and edit 4 files just to have a new tag. An example is adding waterways here
Create a filter configuration for the Tag
Add Code to the run.sh script to enable crunching
Edit oqt-user-experience/map.js file to add static experience initial value
Edit oqt-user-experience/index.js file to add static experience initial value
Furthermore, in order for the cruncher to support thematic analysis, there is a need to make the code highly dynamic and configurable such that it allows for easier and more scalable addition of features to the backend, reducing the amount of code and configuration needed to change exposed features. these datasets should also be easy to integrate with an API.
The solution
This proposal is heavily supported by the front-end proposal and all the changes and context are highly dependant on the front end implementation.
OSM-Cruncher architecture
A proper understanding of the generic architectural flow of the OSM cruncher is vital in appreciating the refactoring required to implement thematic analysis within the existing system. This process can be broken down into three major steps.
Filtering phase
At this point the entire world dataset from OSM if filtered to obtain the required TAG. The output is a minimal .mbtiles file
Binning and aggregation
AT this stage, OSM features are grouped into geocells for analysis.
Downscaling
At this stage, the geocells are aggregated to generate tiles at a lower zoom level, for example, zoom level 12 mbtiles are aggregated to obtain zoom level 11 tiles
A more detailed cruncher architecture can be found here
As described above, this process is very cumbersome, however, with the new implementation; we made a couple of additions and refactorings to greatly improve this process. A user only needs to edit one place and the rest of the process is automated, i.e. Compared to the fomer 4 steps, current to add a new tag for example power;
Create the config file, and place it in fsp-config folder with the format below:
And that's all, during the crunching process, the cruncher automatically picks up this configuration file and creates a .mbtiles file in the results folder. This will also be automatically be picked up by the tile server and made available for the client to consume.
Thematic analysis
WIth the new dynamic configuration, we are able to create even more complex filters and consequently do very detailed analysis on OSM data. For example, we wanted to be able to find out the economic activity of given areas in Uganda. For this, we needed a couple of datasets.
Number of buildings
Number of roads
Number of mobile money agents
Population density
To this end, we created two filter files.
A json configuration that is passed to the filter phase (as described above) of the cruncher
During the crunching process, these file is picked dynamically and processed. the output tile data is then consumed by the client side OSMA to render this map. As a financial service provider, I can then decide to or not set up a mobile money agent within that radius. This decision will, of course, be based on how many people are found in that radius as well as the type of economic activity. This map visualizes how mobile money agents are distributed relative to population and economic activity. The map initially renders grid cells with people per mobile money agents.
Serving tile data to the front end
All the generated tiles and geojson files are stored in the results folder. We modified the server script to dynamically picked up any .mbtile file in this folder and make it available for the client as pbf with the URL format {{server}}/tag/{z}/{x}/{y}.pbf
The server also statically exposes JSON Files int the results/json directory which can be used for point analysis.
API
To demonstrate the ease with which this data can be used with an API, we developed a simple API, to compute the number of mobile money agent in a given distance from a bank or atm
{{server}}/distance/${from}/${to}
where from = Minimum distance, to= Maximum distance
This simple API is used by Map2 which visualises the overall distance from mobile money agents to a specific (operator) bank. Both the presence within [distance] is important, but also underserved mobile money agents; who are outside of [distance] of any/specific operator bank.
Updating data
All the data generated can be updated whenever HOT has mapped out more features to give a more current analysis to the user of this tool. See x-run.sh for an example invocation of the scripts above and integration with the example server.
Finacial service providers
The list below shows the list of OSM amenities that we chose for our analysis
Data for financial service providers in Uganda is predominantly available in Eastern Uganda and Kampala because this is where the data collection exercises have taken place. For now, this, therefore, gives a distorted picture of the reality on the ground. We are hoping that as the data collection efforts continue, this will improve.
Background
In an effort to improve access to financial services to the population in Uganda, HOT has been participating in a project attempting to map a different kind of financial service providers in Uganda. Financial service providers are entities that provide services like deposits, credit, transfers, withdrawals, foreign exchange etc, both in the formal and the informal sectors. In Uganda, examples are banks, ATMs, microfinance institutions, mobile money agents and Savings and credit co-operatives (SACCOs). Mapping them and viewing them through different thematic maps will allow different stakeholders such as the financial service providers themselves, the general public, regulators, donors and others to identify what areas are underserved or where there is potential for rolling out new coverage. The thematic analysis is built into the OSMA framework, and as such also provides an example of how thematic analysis can be added to OSMA.
Overview
HOT chose four thematic views which each answer a specific question that financial service providers might ask. The main goal was to create an analysis tool that can provide a basis for decision-making, mainly for financial service providers to see where there could be a potential market (population density and economic activity), where their competitors are active or where there are areas that do not have existing FSPs. The same approach can be chosen for other thematic analysis; however, it is crucial that the information showed on the maps is already providing a certain degree of analysis when it loads. The analysis can then be refined or adjusted using the controls and parameters available. It was somehow challenging to find a good balance between providing sufficient analysis for the maps to be good decision-making tools, but at the same time not be too complex for the users to understand.
Thematic analysis and context data
There is great potential for thematic analysis using the data from OSM. Our technical proposal outlines how we made improvements in OSMA to allow for easier thematic analysis. In addition, good knowledge of the OSM keys is necessary in order to know what data could be relevant for the given user group. We also chose to introduce some external datasets to provide context, such as Worldpop.
However, there is a trade-off between using external datasets and data available in OSM. Population data can be re-downloaded from Worldpop and updated in case there is newer data.
For economic analysis, we chose to use a composite of data available in OSM in order to avoid the need for updating. After some trial and error, we used the following keys to show economic analysis
This is what gave the best result in Uganda, however, note that it might not give the same result in other areas due to the different nature of data available there. Other than that, we encountered the following challenges with the data for financial service providers available in OSM:
The operator property on some of the banks was empty while in other cases, some bank names were saved in short form, i.e. DTB was used in many cases instead of Diamond Trust Bank and this means that not all Diamond Trust Bank branches are rendered when you select it since some are saved under DTB.
Proposed by
Timothy Kasasa (Laboremus), Benjamin Lutaaya (Laboremus)
The Problem
While there are over 5000 Tags in OpenStreetMap, currently, there are only 3 tags configured for analysis in OSMA. There could be several reasons but the major cause is the fact that the process of adding a new Tag is quite complex and fragile. One would have to go through 5 step and edit 4 files just to have a new tag. An example is adding waterways here
Furthermore, in order for the cruncher to support thematic analysis, there is a need to make the code highly dynamic and configurable such that it allows for easier and more scalable addition of features to the backend, reducing the amount of code and configuration needed to change exposed features. these datasets should also be easy to integrate with an API.
The solution
This proposal is heavily supported by the front-end proposal and all the changes and context are highly dependant on the front end implementation.
OSM-Cruncher architecture
A proper understanding of the generic architectural flow of the OSM cruncher is vital in appreciating the refactoring required to implement thematic analysis within the existing system. This process can be broken down into three major steps.
Filtering phase
At this point the entire world dataset from OSM if filtered to obtain the required TAG. The output is a minimal
.mbtiles
fileBinning and aggregation
AT this stage, OSM features are grouped into geocells for analysis.
Downscaling
At this stage, the geocells are aggregated to generate tiles at a lower zoom level, for example, zoom level 12 mbtiles are aggregated to obtain zoom level 11 tiles
A more detailed cruncher architecture can be found here
Code Base
Adding a new Tag
As described above, this process is very cumbersome, however, with the new implementation; we made a couple of additions and refactorings to greatly improve this process. A user only needs to edit one place and the rest of the process is automated, i.e. Compared to the fomer 4 steps, current to add a new tag for example power;
fsp-config
folder with the format below:And that's all, during the crunching process, the cruncher automatically picks up this configuration file and creates a
.mbtiles
file in the results folder. This will also be automatically be picked up by the tile server and made available for the client to consume.Thematic analysis
WIth the new dynamic configuration, we are able to create even more complex filters and consequently do very detailed analysis on OSM data. For example, we wanted to be able to find out the economic activity of given areas in Uganda. For this, we needed a couple of datasets.
To this end, we created two filter files.
A json configuration that is passed to the filter phase (as described above) of the cruncher
Filter script, which is passed to the aggregation and downscale phases
During the crunching process, these file is picked dynamically and processed. the output tile data is then consumed by the client side OSMA to render this map. As a financial service provider, I can then decide to or not set up a mobile money agent within that radius. This decision will, of course, be based on how many people are found in that radius as well as the type of economic activity. This map visualizes how mobile money agents are distributed relative to population and economic activity. The map initially renders grid cells with people per mobile money agents.
Serving tile data to the front end
All the generated tiles and geojson files are stored in the results folder. We modified the server script to dynamically picked up any
.mbtile
file in this folder and make it available for the client aspbf
with the URL format{{server}}/tag/{z}/{x}/{y}.pbf
The server also statically exposes JSON Files int the results/json directory which can be used for point analysis.
API
To demonstrate the ease with which this data can be used with an API, we developed a simple API, to compute the number of mobile money agent in a given distance from a bank or atm
This simple API is used by Map2 which visualises the overall distance from mobile money agents to a specific (operator) bank. Both the presence within [distance] is important, but also underserved mobile money agents; who are outside of [distance] of any/specific operator bank.
Updating data
All the data generated can be updated whenever HOT has mapped out more features to give a more current analysis to the user of this tool. See
x-run.sh
for an example invocation of the scripts above and integration with the example server.Finacial service providers
The list below shows the list of OSM amenities that we chose for our analysis
Challenges and open issues
References
The text was updated successfully, but these errors were encountered: