Python script for automated access to world-bank website and retreiving datasets with datapackages.
World bank is a global partnership that provides loans to countries for capital projects. It tracks global development indicators and allows free and open access to its data. There are indicators from a variety of fields such as agriculture, economy, climate change etc.
- Birth rate, crude (per 1,000 people)
- Charges for the use of intellectual property, payments (BoP, current US$)
- Children out of school, primary, female
- Risk of impoverishing expenditure for surgical care (% of people at risk)
- Income share held by second 20%
- Unemployment, male (% of male labor force) (modeled ILO estimate)
- Livestock production index (2004-2006 = 100)
- High-technology exports (% of manufactured exports)
- PM2.5 air pollution, mean annual exposure (micrograms per cubic meter)
- Inflation, consumer prices (annual %)
- Diabetes prevalence (% of population ages 20 to 79)
- Proportion of seats held by women in national parliaments (%)
- Plant species (higher), threatened
- Completeness of death registration with cause-of-death information (%)
- Labor force participation rate, male (% of male population ages 15+) (modeled ILO estimate)
- Fossil fuel energy consumption (% of total)
- Income share held by lowest 20%
- Vulnerable employment, female (% of female employment) (modeled ILO estimate)
- Unemployment, youth female (% of female labor force ages 15-24) (modeled ILO estimate)
- GDP per capita growth (annual %)
- Employment in agriculture, female (% of female employment) (modeled ILO estimate)
- Income share held by highest 20%
- CO2 emissions (metric tons per capita)
- Contraceptive prevalence, any methods (% of women ages 15-49)
- School enrollment, primary (% gross)
- Commercial bank branches (per 100,000 adults)
- Literacy rate, youth female (% of females ages 15-24)
- Prevalence of anemia among children (% of children under 5)
- GDP per person employed (constant 2011 PPP $)
- Annual freshwater withdrawals, total (billion cubic meters)
- Cereal yield (kg per hectare)
- Cause of death, by injury (% of total)
- Population ages 65 and above (% of total)
- School enrollment, primary and secondary (gross), gender parity index (GPI)
- Refugee population by country or territory of asylum
- Annualized average growth rate in per capita real survey mean consumption or income, bottom 40% of population (%)
- Crop production index (2004-2006 = 100)
- Net official development assistance received (current US$)
- Children in employment, total (% of children ages 7-14)
- Literacy rate, adult male (% of males ages 15 and above)
- Renewable energy consumption (% of total final energy consumption)
- Market capitalization of listed domestic companies (current US$)
- Inflation, GDP deflator (annual %)
- Survey mean consumption or income per capita, total population (2011 PPP $ per day)
- Electric power consumption (kWh per capita)
- Population in urban agglomerations of more than 1 million (% of total population)
- GDP per capita, PPP (current international $)
- Unemployment, total (% of total labor force) (modeled ILO estimate)
- Total tax rate (% of commercial profits)
- Cause of death, by communicable diseases and maternal, prenatal and nutrition conditions (% of total)
- Refugee population by country or territory of origin
- Income share held by fourth 20%
- Time required to start a business (days)
- GDP growth (annual %)
- External debt stocks, public and publicly guaranteed (PPG) (DOD, current US$)
- PM2.5 air pollution, population exposed to levels exceeding WHO guideline value (% of total)
- Gross intake ratio in first grade of primary education, female (% of relevant age group)
- Children in employment, female (% of female children ages 7-14)
- Firms with female top manager (% of firms)
- Maternal mortality ratio (modeled estimate, per 100,000 live births)
- Air transport, registered carrier departures worldwide
- Short-term debt (% of total reserves)
- Current account balance (BoP, current US$)
- GDP (current US$)
- Start-up procedures to register a business, male (number)
- Domestic credit provided by financial sector (% of GDP)
- Agricultural machinery, tractors per 100 sq. km of arable land
- Nondiscrimination clause mentions gender in the constitution (1=yes; 0=no)
- Bank capital to assets ratio (%)
- Terrestrial and marine protected areas (% of total territorial area)
- Fixed telephone subscriptions (per 100 people)
- Progression to secondary school, male (%)
- Technical cooperation grants (BoP, current US$)
- Labor force, female (% of total labor force)
- Foreign direct investment, net inflows (BoP, current US$)
- Pregnant women receiving prenatal care (%)
- Mortality rate, infant (per 1,000 live births)
- Total reserves (includes gold, current US$)
- CPIA policies for social inclusion/equity cluster average (1=low to 6=high)
- Mortality rate, neonatal (per 1,000 live births)
- Energy use (kg of oil equivalent per capita)
- Deposit interest rate (%)
- Trademark applications, direct nonresident
- Primary completion rate, total (% of relevant age group)
- New businesses registered (number)
- Gross savings (% of GDP)
- Persistence to last grade of primary, male (% of cohort)
- GDP per unit of energy use (constant 2011 PPP $ per kg of oil equivalent)
- Urban population
- Start-up procedures to register a business, female (number)
- Immunization, measles (% of children ages 12-23 months)
- Employment in services, male (% of male employment) (modeled ILO estimate)
- Completeness of birth registration (%)
- GNI per capita, PPP (current international $)
- CPIA public sector management and institutions cluster average (1=low to 6=high)
- Literacy rate, adult total (% of people ages 15 and above)
- Renewable electricity output (% of total electricity output)
- Death rate, crude (per 1,000 people)
- Contributing family workers, male (% of male employment) (modeled ILO estimate)
- Mortality rate, under-5 (per 1,000 live births)
- Renewable internal freshwater resources per capita (cubic meters)
- Population growth (annual %)
- Energy intensity level of primary energy (MJ/$2011 PPP GDP)
- Real interest rate (%)
- Rural poverty gap at national poverty lines (%)
- Wage and salaried workers, male (% of male employment) (modeled ILO estimate)
- Total debt service (% of exports of goods, services and primary income)
- Urban population (% of total)
- Forest area (sq. km)
- GINI index (World Bank estimate)
- Population, female (% of total)
- Intentional homicides (per 100,000 people)
- Industry, value added (% of GDP)
- Permanent cropland (% of land area)
- Agriculture, value added (% of GDP)
- Rural population
- Interest rate spread (lending rate minus deposit rate, %)
- Fertility rate, total (births per woman)
- Business extent of disclosure index (0=less disclosure to 10=more disclosure)
- Primary completion rate, female (% of relevant age group)
- Life expectancy at birth, male (years)
- Government expenditure per student, secondary (% of GDP per capita)
- Container port traffic (TEU: 20 foot equivalent units)
- Trained teachers in primary education (% of total teachers)
- Research and development expenditure (% of GDP)
- Gross capital formation (% of GDP)
- Charges for the use of intellectual property, receipts (BoP, current US$)
- External debt stocks, total (DOD, current US$)
- Land area (sq. km)
- Merchandise exports (current US$)
- Specialist surgical workforce (per 100,000 population)
- Merchandise trade (% of GDP)
- Net barter terms of trade index (2000 = 100)
- Net ODA received (% of GNI)
- Annualized average growth rate in per capita real survey mean consumption or income, total population (%)
- Labor force, total
- Poverty headcount ratio at national poverty lines (% of population)
- Prevalence of HIV, male (% ages 15-24)
- School enrollment, secondary (% net)
- Poverty gap at $1.90 a day (2011 PPP) (%)
- Prevalence of underweight, weight for age (% of children under 5)
- Urban poverty headcount ratio at national poverty lines (% of urban population)
- Trademark applications, direct resident
- Risk premium on lending (lending rate minus treasury bill rate, %)
- Fixed broadband subscriptions (per 100 people)
- Expense (% of GDP)
- Teenage mothers (% of women ages 15-19 who have had children or are currently pregnant)
- Fish species, threatened
- Children out of school, primary, male
- GNI, PPP (current international $)
- Repeaters, primary, male (% of male enrollment)
- Depth of credit information index (0=low to 8=high)
- Ease of doing business index (1=most business-friendly regulations)
- Food production index (2004-2006 = 100)
- International migrant stock, total
- Firms using banks to finance working capital (% of firms)
- Employment in industry, male (% of male employment) (modeled ILO estimate)
- Renewable internal freshwater resources, total (billion cubic meters)
- Stocks traded, turnover ratio of domestic shares (%)
- Listed domestic companies, total
- Methane emissions (kt of CO2 equivalent)
- Bird species, threatened
- Arable land (hectares per person)
- Population living in areas where elevation is below 5 meters (% of total population)
- CPIA economic management cluster average (1=low to 6=high)
- GDP per capita (current US$)
- Lending interest rate (%)
- Unmet need for contraception (% of married women ages 15-49)
- S&P Global Equity Indices (annual % change)
- Rural population (% of total population)
- Employment in services, female (% of female employment) (modeled ILO estimate)
- CPIA structural policies cluster average (1=low to 6=high)
- Cause of death, by non-communicable diseases (% of total)
- Broad money (% of GDP)
- Persistence to last grade of primary, female (% of cohort)
- GNI per capita, Atlas method (current US$)
- Prevalence of overweight, weight for height (% of children under 5)
- Access to electricity (% of population)
- Hospital beds (per 1,000 people)
- GNI, Atlas method (current US$)
- Wage and salaried workers, female (% of female employment) (modeled ILO estimate)
- Adolescent fertility rate (births per 1,000 women ages 15-19)
- Automated teller machines (ATMs) (per 100,000 adults)
- Researchers in R&D (per million people)
- Net flows on external debt, total (NFL, current US$)
- Net ODA received (% of gross capital formation)
- Mobile cellular subscriptions (per 100 people)
- School enrollment, preprimary (% gross)
- Investment in telecoms with private participation (current US$)
- Scientific and technical journal articles
- School enrollment, secondary (% gross)
- Prevalence of undernourishment (% of population)
- Contributing family workers, female (% of female employment) (modeled ILO estimate)
- Lead time to export, median case (days)
- Population ages 15-64 (% of total)
- Energy imports, net (% of energy use)
- Military expenditure (% of GDP)
- Government expenditure on education, total (% of government expenditure)
- Time required to start a business, male (days)
- Agricultural land (% of land area)
- Arable land (% of land area)
- Merchandise imports (current US$)
- Total natural resources rents (% of GDP)
- Women making their own informed decisions regarding sexual relations, contraceptive use and reproductive health care (% of women age 15-49)
- CO2 emissions (kt)
- Prevalence of HIV, female (% ages 15-24)
- Population ages 0-14 (% of total)
- Land under cereal production (hectares)
- Net acquisition of financial assets (% of GDP)
- PPP conversion factor, GDP (LCU per international $)
- Employment to population ratio, 15+, total (%) (modeled ILO estimate)
- Annual freshwater withdrawals, total (% of internal resources)
- Primary completion rate, male (% of relevant age group)
- Patent applications, residents
- Technicians in R&D (per million people)
- Proportion of time spent on unpaid domestic and care work, female (% of 24 hour day)
- Life expectancy at birth, female (years)
- Prevalence of HIV, total (% of population ages 15-49)
- Logistics performance index: Overall (1=low to 5=high)
- Employment in industry, female (% of female employment) (modeled ILO estimate)
- Population density (people per sq. km of land area)
- Net migration
- Exports of goods and services (% of GDP)
- Life expectancy at birth, total (years)
- Immunization, DPT (% of children ages 12-23 months)
- Repeaters, primary, female (% of female enrollment)
- International tourism, expenditures (% of total imports)
- Statistical Capacity score (Overall average)
- Labor force participation rate, female (% of female population ages 15+) (modeled ILO estimate)
- Stocks traded, total value (% of GDP)
- International tourism, receipts (% of total exports)
- Time spent dealing with the requirements of government regulations (% of senior management time)
- Imports of goods and services (% of GDP)
- Mortality caused by road traffic injury (per 100,000 people)
- Investment in water and sanitation with private participation (current US$)
- School enrollment, primary (gross), gender parity index (GPI)
- School enrollment, tertiary (% gross)
- Rural poverty headcount ratio at national poverty lines (% of rural population)
- Broad money growth (annual %)
- External debt stocks (% of GNI)
- Ores and metals exports (% of merchandise exports)
- Central government debt, total (% of GDP)
- Unemployment, female (% of female labor force) (modeled ILO estimate)
- Net investment in nonfinancial assets (% of GDP)
- Prevalence of stunting, height for age (% of children under 5)
- Grants, excluding technical cooperation (BoP, current US$)
- Use of IMF credit (DOD, current US$)
- Literacy rate, youth total (% of people ages 15-24)
- Births attended by skilled health staff (% of total)
- Investment in transport with private participation (current US$)
- Start-up procedures to register a business (number)
- Literacy rate, youth male (% of males ages 15-24)
- Age dependency ratio (% of working-age population)
- Number of surgical procedures (per 100,000 population)
- Time required to start a business, female (days)
- Land area where elevation is below 5 meters (% of total land area)
- Risk of catastrophic expenditure for surgical care (% of people at risk)
- Proportion of time spent on unpaid domestic and care work, male (% of 24 hour day)
- Incidence of tuberculosis (per 100,000 people)
- Alternative and nuclear energy (% of total energy use)
- High-technology exports (current US$)
- Population living in slums (% of urban population)
- Unemployment, youth male (% of male labor force ages 15-24) (modeled ILO estimate)
- Net ODA received per capita (current US$)
- Vulnerable employment, male (% of male employment) (modeled ILO estimate)
- Employment in agriculture, male (% of male employment) (modeled ILO estimate)
- Nitrous oxide emissions (thousand metric tons of CO2 equivalent)
- Market capitalization of listed domestic companies (% of GDP)
- Firms with female participation in ownership (% of firms)
- School enrollment, primary (% net)
- Children in employment, male (% of male children ages 7-14)
- Income share held by third 20%
- Gross intake ratio in first grade of primary education, male (% of relevant age group)
- IBRD loans and IDA credits (DOD, current US$)
- Tax revenue (% of GDP)
- Population in the largest city (% of urban population)
- Lead time to import, median case (days)
- Survey mean consumption or income per capita, bottom 40% of population (2011 PPP $ per day)
- Strength of legal rights index (0=weak to 12=strong)
- Adjusted net savings, including particulate emission damage (% of GNI)
- Rail lines (total route-km)
- Fertilizer consumption (kilograms per hectare of arable land)
- Price level ratio of PPP conversion factor (GDP) to market exchange rate
- Secure Internet servers (per 1 million people)
- Literacy rate, adult female (% of females ages 15 and above)
- External debt stocks, private nonguaranteed (PNG) (DOD, current US$)
- Net incurrence of liabilities, total (% of GDP)
- Domestic credit to private sector (% of GDP)
- Personal remittances, received (current US$)
- External debt stocks, short-term (DOD, current US$)
- Total greenhouse gas emissions (kt of CO2 equivalent)
- Income share held by highest 10%
- Surface area (sq. km)
- Poverty headcount ratio at $1.90 a day (2011 PPP) (% of population)
- Government expenditure per student, primary (% of GDP per capita)
- Agricultural irrigated land (% of total agricultural land)
- Population, total
- IDA resource allocation index (1=low to 6=high)
- Progression to secondary school, female (%)
- Official exchange rate (LCU per US$, period average)
- Mammal species, threatened
- Fuel exports (% of merchandise exports)
- Time required to get electricity (days)
- Poverty gap at national poverty lines (%)
- Investment in energy with private participation (current US$)
- Revenue, excluding grants (% of GDP)
- Forest area (% of land area)
- Urban poverty gap at national poverty lines (%)
- Government expenditure per student, tertiary (% of GDP per capita)
The scripts are python based and use python 3.5+. The script get.py has no dependencies outside of the standard library. It is used to download and extract a specific indicator. Credit goes to rufuspolock. In order to obtain dataset with a specific indicator url run:
python scripts/get.py indicator_url
Example:
python scripts/get.py https://data.worldbank.org/indicator/GC.DOD.TOTL.GD.ZS
# This will download Central government total debt as percentage of GDP dataset
The script extractFeaturedWorldBankDatasets.py obtains datasets from the world-bank featured page https://data.worldbank.org/indicator?tab=featured. It uses library BeautifulSoup in order to scrape indicator urls. To install BeautifulSoup run:
pip install bs4
To obtain all featured indicators run:
python scripts/extractFeaturedWorldBankDatasets.py