This python package gives some basic tools to easily collect information on open data soft portals (SaaS):
- Number of federated or non-federated data
- Themes
- Data producers
You'll need these librairies.
- Pandas:
pip install pandas
- Requests
pip install requests
- Seaborn & Matplotlib
pip install matplotlib
pip install seaborn
- tqdm (optional)
pip install tqdm
from ScrappingPackage.opendatasoftscrapper import Opendatasoftscrapper
url = "https://data.regionreunion.com"
ods_scrapper = Opendatasoftscrapper(lien = lien)
results = ods_scrapper.get_all_results()
The method get_all_results()
returns a dict that holds the extracted information.
You can use bar_plot_fed()
from Opendatasoftvisualisations
class to make plots out of these results and save theme.
from ScrappingPackage.opendatasoftvisualisations import Opendatasoftvisualisations
ods_vis = Opendatasoftvisualisations(results)
ods_vis.bar_plot_fed(save = True, title = "bar plot 1" )
ods_vis.bar_plot_themes(save = True,title = "bar plot 2")
If you want to test the program immediately, you can run :
python3 main.py