Built upon a Python package (export_ease) that calls JSON RESTful API to gather macroeconomic data, this program offers a streamlined interface through which you can request and analyze vast amounts of macroeconomic data from sources including UN Comtrade, International Monetary Fund (IMF), and World Bank. Below is a step-by-step guide detailing effective use of this program.
- Configure your RStudio and GitHub accounts to use SSH—if you haven't added your SSH key, or if you don't have an SSH key, follow this setup guide: SSH-key setup.
- Clone the repository by clicking on
Code
and copying the SSH link. - Create a new project within RStudio by clicking on the 3D box in the upper-left corner. Click
Version Control
and thenGit
, and then paste the link that you copied. - Navigate to the
src
folder and openSanctions.qmd
; this is the only file that you'll need to have open in order to run the program (though you may also want to view a csv/xlsx file in thedata
folder--you can do this by clicking on said file).- Side note: If you want to view an Excel file, it's easiest to download it from this repository directly.
- Run the first code chunk to install all the necessary packages. You can comment out this chunk after installation. After saving your changes locally, you can commit and push your changes to GitHub by running the following commands in the terminal:
git add .
git commit -m "your message here"
git push
- Load the packages you just installed by running the second code chunk.
- Now, run the next code chunk to install/import the Python packages that the program needs in order to work properly.
- Every time you open RStudio, be sure to run
git pull
in the terminal--this will update your local clone of the project with the most up-to-date changes.
- Simply specify the year for which you'd like to gather data by editing the year variable in the first code chunk of the
Viewing + Getting Data
section and then run all code chunks. The functions provided will query, clean, merge, convert frequency, and calculate beta for all available countries. - Make sure you are in the
src
folder before running the program; otherwise, files won't be able to be written to the data folder, and the function will throw an error.
This part of the program utilizes the comtradeapicall Python package to implement additional functionality. To query export and import data from Comtrade, you can use the following methods:
get_all_exports
: writes csv file containing export data for all available country pairs (i.e., exports from each reportner to all its partners)get_total_exports
: writes csv file containing total-export (exports to world) data for all available reportersget_all_imports
: writes csv file containing import data for all available country pairs (i.e., imports to each reporter from all its partners)get_total_imports
: writes csv file containing total-import (imports from world) data for all available reportersget_cmd_exports
: writes csv file containing import data for a commodity for a specific reporter-partner pair. If you want the total amount of a commodity imported by a reporter, make the partner0
(the world). If you want the amount of a commodity imported by a reporter from each of its individual partners, make the partner argumentNULL
(all possible partners).get_cmd_imports
: does same thing asget_cmd_exports
but for imports
When you run these functions, you'll need to enter the criteria for your query. All of these functions require at least two arguments, the first being the frequency ("A" for annual, "M" for monthly, or "B" for both) and the second being the year for which to gather data (e.g., 2021). Once the program's finished running, it'll output the names of the files that were just created. get_cmd_exports
and get_cmd_imports
also require reporter, partner, and cmd_code arguments. You can find Comtrade's country codes in data
under the file name "comtrade_countries.pdf" and HS-classified commodity codes here.
Functions with which to query data:
get_reporter_exports
: writes csv file containing value exports from reporter provided in console input to all its partnersget_total_exports
: same as Comtrade'sget_total_exports
method
The code to get IMF data is almost identical in structure to Comtrade's. The only difference is the arguments for get_reporter_exports
(get_total_exports
is the same for both Comtrade and IMF): the first argument is the name of the reporting country for which you'd like to gather (e.g., "France"), followed by the frequency and year (in the same format as Comtrade's).
If you input an incorrect country name for get_reporter_exports
, you can check country_codes.csv to see valid country names (this file is written by the program when you run get_reporter_exports
).
- Unlike the previous two sources, the code for getting World Bank data does not make any API calls (at least not directly)—all functionality is provided by the wbstats R package.
get_wb
: writes two csv and Excel files, one containing GDP and the other total-export data (both include every possible reporter) and outputs the names of the files written (in the console)
- Note: No matter which data frame you return, both will be written to csv and Excel files. If you'd like to view both data frames, either run
get_wb
twice, changing the data frame-to-return parameter each time, or runget_wb
once and then load the data frame that wasn't returned withdf <- read_csv(file.path(dirname(getwd()), "data", file_name.csv))
.
- You can use
clean_comtrade
andclean_imf
to clean the queried data returned by the Comtrade and IMF functions listed above. (World Bank data is automatically cleaned whenget_wb
is called.) - The other functions should not be used directly, as they are called by the
everything
function when merging, converting frequency, and calculating beta to create the final combined data set. (Even these functions don't need to be used directly, as they are also called automatically wheneverything
runs.)
- Reporter-to-all-Partners Export Sources: Comtrade (all country pairs) and IMF (one country pair @ a time)
- Total-Export Sources: Comtrade (all reporters), IMF (all reporters), and World Bank (all reporters)
- GDP Source: World Bank (all reporters)
- Import Source: Comtrade
- All data is expressed in USD, and most data is available in both monthly and annual quantities, with World Bank being the only source that supports annual data only. None of the data is seasonally adjusted, nor is it inflation adjusted.
- Comtrade functions will retrieve data for the given year only, while IMF functions will retrieve data starting at the given year and ending with the most recently published data.
- The
get_reporter_exports
function for IMF has a quirk that occurs when the user requests a query for annual data starting at a year within 3 years of the current year. In this case, the function will "override" the user's indicated year and instead make the starting year 3 years less than the current year--I had to add this padding in order to work around the varying structures of the JSON file returned by the IMF API. This has no serious implications, as it still gets all the data you requested (and then some). Ctrl-shift-c
/cmd-shift-c
uncomments/comments out a block/line of code; commented code will not run. (I often use this with the viewing function to only view the data frames when needed.)
- New User Guide
- Python Package Documentation
- API Interface
- Requires an account and a subscription key
- Allows you to determine what call you need to make given specific parameters and see if a call if valid or not, but doesn't allow you to download data directly from this page
- API
- Bulk Query