Federal Treasury API
‹ € | | €€€€≤±‹€€€€≤±‹ € .| -- | -+ € ' | | | €€€€≤±±≤€€€€€€€€€€€€≤±±≤€€€€€≤± € `| | € |`. | €€€€≤±±≤€€€€€€€€€€€€≤±±≤€€€€€≤± € | `.| € | |`. €€€€≤±±≤€€€€€€€€€€€€≤±±≤€€€€€≤± € + | | ' € | _ | __ | _' €€€≤±±≤€€€€€€€€€€€€≤±±≤€€€€€≤± € | | €€€€€€€€€€€€€€€€€€€€€€≤±±≤€€€€€€€€€€€€≤±±≤€€€€€≤± € €€€€€€€€€€€€€€€€€€€€€€≤±±≤€€€€€€€€€€€€≤±±≤€€€€€≤± € € ﬂ±≤€€€€€€€€€€€€≤±±≤€€€€€≤± € ﬂ±≤€€€€€≤± € €
About this branch
master branch of
federal-treasury-api contains code specific to running our project on ScraperWiki. the
just-the-api branch contains only the code needed to download the data locally and launch a queryable api. In other words, if you're just looking to get and host the database, go here.
About the API
federal-treasury-api is the first-ever electronically-searchable database of the Federal government's daily cash spending and borrowing. It updates daily and the data can be exported in various formats and loaded into various systems.
About the data
There are eight tables.
- I. Operating Cash Balance (
- II. Deposits and Withdrawals (
- IIIa. Public Debt Transactions (
- IIIb. Adjustment of Public Dept Transactions to Cash Basis (
- IIIc. Debt Subject to Limit (
- IV. Federal Tax Deposits (
- V. Short-term Cash Investments (
- VI. Incom Tax Refunds Issued (
Obtaining the data
Optionally set up a virtualenv. (You need this on ScraperWiki.) Run this from the root of the current repository.
pip install -r requirements.pip
Enable the git post-merge hook.
cd .git/hooks ln -s ../../utils/post-merge .
This one command downloads the (new) fixies and converts them to an SQLite3 database.
cd parser python download_and_parse_fms_fixies.py
Testing the data
Various tests are contained in
Tests are run everyday with
./run.sh and the results are emailed to
Run everything each day around 4:30 - right after the data has been released.
30 16 * * * cd path/to/federal-treasury-api && ./run.sh
Optional: set up logging
30 16 * * * cd path/to/federal-treasury-api && ./run.sh >> run.log 2>> err.log
Deploying to ScraperWiki
You can run this on any number of servers, but we happen to be using ScraperWiki. You can check out their documentation here
To use ScraperWiki, log in here, make a project, click the "SSH in" link, add your SSH key and SSH in. Then you can SSH to the box like so.
Or add this to your
Host fms HostName premium.scraperwiki.com User cc7znvq
and just run
What this ScraperWiki account is
Some notes about how ScraperWiki works:
- We have a user account in a chroot jail.
- We don't have root, so we install Python packages in a virtualenv.
- Files in
/home/httpget served on the web.
- The database
/home/scraperwiki.sqlitegets served from the SQLite web API.
- NOTE: the
home/scraperwiki.sqliteis simply a symbolic link to
data/treasury_data.dbgenerated by this command:
ln -s data/treasury_data.db scraperwiki.sqlite
- NOTE: the
The directions below still apply for any other service, of course.