The code that runs the DigitalGov Search Non-.gov URLs API—a list of government URLs that don't end in .gov or .mil.—is here on Github. If you're a Ruby developer, keep reading. Fork this repo to add features (such as additional datasets) or fix bugs.
The documentation on request parameters and response format is on the API developer page. This README just covers software development of the API service itself.
We maintain this list at http://govt-urls.usa.gov/tematres/vocab/index.php and make periodic updates as we come across changes.
Each quarter, we also post the URLs here in Github in two text files.
- Alphabetic—an A-Z list of URLs with accompanying notes and relationships collected over time.
- Hierarchical—a flat list of URLs segmented by category (see BT description in the following section).
Cross reference symbols that you'll find in the alphabetic list include:
- BT (broader term)—indicates the category for each URL, such as federal (usagovFED) or state, commonwealth, or territory (usagov__, 2-letter postal abbreviation).
- NT (narrower term)—the opposite of BT.
- UF (used for)—indicates the nonpreferred URL. Nonpreferred URLs no longer resolve or redirect to another URL. UF is the reciprocal of USE and means "don't use" the term following it.
- USE—indicates the preferred, resolving URL. "Use" the term following it.
- RTET (related equivalent term)—indciates two URLs that both resolve to the same website. Neither is preferred.
- RT (related term)—indicates an association between two related terms when it seems helpful.
Running the API Locally
The project requires Ruby 2.2.2.
We use bundler to manage gems. You can install bundler and other required gems like this:
gem install bundler bundle install
brew install elasticsearch
Otherwise, follow the instructions to download and run it. Elasticsearch must be running locally.
You'll need to have redis installed on your machine.
brew install redis,
apt-get install redis-server, etc. Redis must also be running locally and can be started with the
Starting the Server
bundle exec rails s
Sidekiq must be running to import the data into Elasticsearch.
bundle exec sidekiq
Importing the Data
bundle exec rake tematres:import
Viewing the Results
Navigate to http://localhost:3000/government_urls/search?.