A Identity Management System powering login.gov.
Setting up and running the app
- Make sure you have a working development environment with all the dependencies installed. On OS X, the easiest way to set up a development environment is by running our Laptop script. The script will install all of this project's dependencies.
If using rbenv, you may need to alias your specific installed ruby version to the more generic version found in the
.ruby-version file. To do this, use
git clone git://github.com/tpope/rbenv-aliases.git "$(rbenv root)/plugins/rbenv-aliases" # install rbenv-aliases per its documentation rbenv alias 2.3 2.3.5 # create the version alias
- Make sure Postgres and Redis are running.
For example, if you've installed the laptop script on OS X, you can start the services like this:
$ brew services start redis $ brew services start postgresql
- Create the development and test databases:
$ psql -c "CREATE DATABASE upaya_development;" $ psql -c "CREATE DATABASE upaya_test;"
- Run the following command to set up the environment:
$ make setup
This command copies sample configuration files, installs required gems and sets up the database.
- Run the app server with:
$ make run
Before making any commits, you'll also need to run
This verifies that the commit hooks defined in our
.overcommit.yml file are
the ones we expect. Each change to the
.overcommit.yml file, including the initial install
performed in the setup script, will necessitate a new signature.
For more information, see overcommit
If you want to develop without an internet connection, you can set
RAILS_OFFLINE=1 in your environment. This disables the
check on email addresses.
If you want to measure the app's performance in development, set the
rack_mini_profiler option to
restart the server. See the rack_mini_profiler gem for more details.
If you want to visualize and query the event and log data, you can install the latest versions of Elasticsearch, Logstash, and Kibana. On OS X, the easiest way is with Homebrew:
brew tap homebrew/services brew install elasticsearch logstash kibana brew services start elasticsearch brew services start kibana
Start logstash by running this command from this repo's root directory:
logstash -f logstash.conf
When you trigger an event in the app (such as signing in), you should see some output in the logstash window.
To explore the data with Kibana, visit http://localhost:5601
Troubleshooting Kibana errors
Below are some common errors:
On the Kibana website: "Your Kibana index is out of date, reset it or use the X-Pack upgrade assistant."
In the logstash output:
Failed to parse mapping [_default_]: [include_in_all] is not allowed for indices created on or after version 6.0.0 as [_all] is deprecated. As a replacement, you can use an [copy_to] on mapping fields to create your own catch all field.
Solution, assuming you don't use these services for other apps and are OK with deleting existing data:
- Stop all services:
ctrl-cto stop logstash if it's running
brew services stop elasticsearch brew services stop kibana
- Uninstall everything:
brew uninstall --force elasticsearch brew uninstall --force logstash brew uninstall --force kibana
- Reinstall everything:
brew install elasticsearch logstash kibana
- Start the services:
brew services start elasticsearch brew services start kibana
- Delete the old Kibana index:
curl -XDELETE http://localhost:9200/.kibana
- Delete the old logstash template:
- Visit http://localhost:5601/app/kibana#/dev_tools/console?_g=()
DELETE /_template/logstashin the box on the left and click the green "play" button to run the command
- Start logstash in a new Terminal tab:
logstash -f logstash.conf
Launch the IdP app and sign in to generate some events. You should see output in the logstash tab without any errors.
Visit http://localhost:5601/ and click "Discover" on the left sidebar. If you get a warning that no default index pattern exists, copy the last pattern that appears in the list, which will have the format
logstash-year.month.day. Paste it into the "Index pattern" field, then click the "Next step" button.
Step 2 of 2: Configure settings, select
Time Filter field namedropdown, then click "Create index pattern".
Create some more events on the IdP app.
Refresh the Kibana website. You should now see new events show up in the Discover section.
Download, install, and launch Docker
Set up the Docker image
$ bin/setup --docker
More useful Docker commands:
- Start the container:
- Stop this running container:
- Stop and delete the containers:
- Open a shell in the web container:
docker-compose run --rm web bash
See the Docker Compose docs for more information.
Viewing the app locally
Once it is up and running, the app will be accessible at
http://localhost:3000/ by default.
To view email messages, Mailcatcher must be running. You can check if it's running by visiting http://localhost:1080/. To run Mailcatcher:
If you would like to run the application on a different port:
- Change the port number for
- Run the app on your desired port like
make run PORT=1234
If you would like to see the Spanish translations on a particular page, add
?locale=es to the end of the URL, such as
Currently, you'll need to add
?locale=es to each URL manually. We are working
on a more robust and user-friendly way to switch between locales.
To run all the tests:
$ make test
To run a subset of tests excluding slow tests (such as accessibility specs):
$ make fast_test
Speeding up local development and testing
To automatically run the test that corresponds to the file you are editing,
bundle exec guard with the env var
GUARD_RSPEC_CMD set to your preferred
command for running
rspec. For example, if you use Zeus,
you would set the env var to
GUARD_RSPEC_CMD="zeus rspec" bundle exec guard
If you don't specify the
GUARD_RSPEC_CMD env var, it will default to
bundle exec rspec.
We recommend setting up a shell alias for running this command, such as:
alias idpguard='GUARD_RSPEC_CMD="zeus rspec" bundle exec guard'
If you are on a mac, if you receive the following prompt the first time you run the test suite, enter
sekret as the passphrase:
Documentation for the testing tools we use
We have an automated tool for generating user flows using real views generated from the application. These specs are excluded from our typical spec run because of the overhead of generating screenshots for each view.
The local instance of the application must be running in order to serve up the assets (eg.
make run). Then, you can specify where the assets are hosted from and generate the views with:
$ RAILS_ASSET_HOST=localhost:3000 rake spec:user_flows
Then, visit http://localhost:3000/user_flows in your browser!
The user flows tool also has an export feature which allows you to export everything for the web. You may host these assets with someting like
simplehttpserver or publish to Federalist. To publish user flows for Federalist, first make sure the application is running locally (eg. localhost:3000) and run:
$ RAILS_ASSET_HOST=localhost:3000 FEDERALIST_PATH=/site/user/repository rake spec:user_flows:web
This will output your site to
public/site/user/repository for quick publishing to Federalist. To test compatibility, run
simplehttpserver from the app's
public folder and visit
http://localhost:8000/<FEDERALIST PATH>/user_flows in your browser.
We provide some Locust.io Python scripts you can run to test how the
app responds to load. You'll need to have Python and
installed on your machine. If you're on a Mac, the easiest way to set up Python
pyenv-virtualenvwrapper is to run the laptop script.
Next, you'll need to set the following values in your local
disable_email_sending: 'true' enable_load_testing_mode: 'true' telephony_disabled: 'true'
Then, run the app with
make run, and in a new Terminal tab or window, run:
make load_test type=create_account
This will simulate 3 concurrent users going through the entire account creation
flow and then signing out. To change the number of concurrent users, number of
requests, and the rate at which users are created, modify the
-r Locust parameters in
locust --help for
By default, the test will target the host running at
To change the target host, set the
TARGET_HOST environment variable.
TARGET_HOST=https://awesome.loadtesting.com make load_test type=create_account
Some proofing vendor code is located in private Github repositories because of NDAs. You can still use it in your local development environment if you have access to the private repository.
Check out the private repository for
$ cd vendor $ git clone email@example.com:18F/identity-somevendorname-api-client-gem.git somevendorname
Add the vendor configuration
Add appropriate vendor environment variables to
config/application.yml -- see a member of the
login.gov team for credentials and other values.
"skill in means" https://en.wikipedia.org/wiki/Upaya
Managing translation files
To help us handle extra newlines and make sure we wrap lines consistently, we have a script called
./scripts/normalize-yaml that helps format YAML consistently. After importing translations (or making changes to the *.yml files with strings, run this for the IDP app:
$ make normalize_yaml