diff --git a/Makefile b/Makefile index 85720d50..6d149ac7 100644 --- a/Makefile +++ b/Makefile @@ -72,32 +72,4 @@ load_test_prime_workflow: clean ## Run load testing on the Prime API open http://localhost:8089 locust -f locustfiles/prime_workflow.py --host local -.PHONY: local_docker_build -local_docker_build: clean ## Build a Docker container to run load testing locally - docker-compose -f docker-compose.local.yaml build locust - -.PHONY: local_docker_up -local_docker_up: ## Run load testing on the Prime API in local using a Docker container - open http://localhost:8089 - LOCUSTFILES=$(PRIME_LOCUSTFILES) docker-compose -f docker-compose.local.yaml up locust - -.PHONY: local_docker_down -local_docker_down: ## Shutdown any active local docker containers with docker-compose - docker-compose -f docker-compose.local.yaml down locust - -.PHONY: local_docker_office_up -local_docker_office_up: ## Run load testing on the GHC API in local using a Docker container - open http://localhost:8089 - LOCUSTFILES=$(OFFICE_LOCUSTFILES) docker-compose -f docker-compose.local.yaml up locust - -.PHONY: local_docker_report -local_docker_report: ## Run load testing automatically against a local server and generate reports - export DOCKER_CSV_PREFIX="${DOCKER_CSV_DIR}/$(shell date +'%Y-%m-%d-%H%M%S')"; LOCUSTFILES=$(PRIME_LOCUSTFILES) docker-compose -f docker-compose.local.yaml up --build prime-reporting - docker cp mmlt_prime_reporting:/app/static/reports static/local/ - -.PHONY: exp_load_test -exp_load_test: ## Run load testing against the MilMove Experimental Deployment - export DOCKER_CSV_PREFIX="${DOCKER_CSV_DIR}/$(shell date +'%Y-%m-%d-%H%M%S')"; docker-compose up --build prime-exp-reporting - docker cp mmlt_prime_exp_reporting:/app/static/reports static/ - default: help diff --git a/README.md b/README.md index 0b1a9dda..15aac055 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,6 @@ the [LICENSE.txt](./LICENSE.txt) file in this repository. * [Setup: Nix](#setup-nix) * [Installing Python Dependencies and Pre-commit Hooks](#installing-python-dependencies-and-pre-commit-hooks) * [Unsupported Setup](#unsupported-setup) - * [Alternative Setup: Docker](#alternative-setup-docker) * [Troubleshooting](#troubleshooting) * [Testing](#testing) * [Running Load Tests](#running-load-tests) @@ -40,7 +39,6 @@ the [LICENSE.txt](./LICENSE.txt) file in this repository. * [Setting up Tests in AWS](#setting-up-tests-in-aws) * [Running preset tests](#running-preset-tests) * [Running custom tests](#running-custom-tests) - * [Running Tests for Reporting](#running-tests-for-reporting) * [Adding Load Tests](#adding-load-tests) * [Starting from scratch](#starting-from-scratch) * [Creating TaskSets](#creating-tasksets) @@ -58,7 +56,7 @@ the [LICENSE.txt](./LICENSE.txt) file in this repository. * [Metrics](#metrics) * [References](#references) - + @@ -177,33 +175,25 @@ level of this repo and reload your shell. ### Unsupported Setup -This is a setup that existed previously but will no longer be kept up to date or supported. It will possibly be removed -entirely at a future date. +Maintaining many ways to set up locally can be time-consuming, which is why we removed the `asdf` and docker setups. -#### Alternative Setup: Docker +The `asdf` end set-up was similar to the `pyenv` setup, but required many commands to have tweaks to work the same as +the `pyenv` commands which made them harder to maintain. -It is also possible to run load tests from within a Docker container, eliminating the need to set up a valid python -environment. This requires Docker and `docker-compose` to be installed on your machine. Get them here: +As for docker, we had a few reasons for dropping support: -* [Get Docker](https://docs.docker.com/get-docker/) -* [Install Docker Compose](https://docs.docker.com/compose/install/) - version `1.27` or later +* Locust is a tool that needs to reach the target host and running it from inside docker makes it harder to reach a +server that is managed outside of docker. +* Docker adds yet another layer for possible issues. We've experienced some problems in the past with docker network +problems that were masked as locust errors. Errors like this are a pain to debug. +* Our current setup using `direnv` and `pipenv` is fairly quick to set up using either `nix` or the `make install_tools` +command, decreasing the "quick setup" case for using docker. -To get your load testing Docker container up and running for the local environment, use the following commands: +Setups were removed in the following PRs: -* `docker-compose -f docker-compose.local.yaml build` -* `docker-compose -f docker-compose.local.yaml up` - -And when you are done with testing: - -* `docker-compose -f docker-compose.local.yaml build` - -You can also use the Makefile equivalents of these commands: - -* `make local_docker_build` -* `make local_docker_up` -* `make local_docker_down` - -As long as your local MilMove server is up and running, you are ready to run your tests! +* [PR #74](https://github.com/transcom/milmove_load_testing/pull/74) +* [PR #78](https://github.com/transcom/milmove_load_testing/pull/78) +* [PR #89](https://github.com/transcom/milmove_load_testing/pull/89) ### Troubleshooting @@ -324,26 +314,6 @@ locust -f locustfiles/prime.py --host local For more information on running custom tests, refer to the [wiki](https://github.com/transcom/milmove_load_testing/wiki/Running-Load-Tests-Against-MyMove) -### Running Tests for Reporting - -*NOTE*: THESE COMMAND ARE DEPRECATED AND NOT WORKING AS OF 2021-11-17 - -There are a couple of preset tests that have been set up to generate reports for later analysis. These commands are: - -```shell -make local_docker_report -``` - -and: - -```shell -make exp_load_test -``` - -**`make exp_load_test` should only be used on a scheduled basis with InfraSec's supervision.** - -Both of these commands require a `docker-compose` version of `1.27` or greater to work. - ## Adding Load Tests ### Starting from scratch diff --git a/docker-compose.local.yaml b/docker-compose.local.yaml deleted file mode 100644 index bb2687c4..00000000 --- a/docker-compose.local.yaml +++ /dev/null @@ -1,29 +0,0 @@ -version: '3.8' - -services: - locust: - build: - context: . - network: host - environment: - - "LOCUSTFILES=${LOCUSTFILES}" - ports: - - "8089:8089" - - "9443:9443" - entrypoint: ["python", "local_locust.py"] - # To use local_locust.py, we have to pass the locust flags in as a string that can be split to call the locust - # command as normal: - command: "'-f ${LOCUSTFILES} --host local'" - prime-reporting: - container_name: mmlt_prime_reporting - extends: - service: locust - environment: - - LOCUST_LOGLEVEL=warning - - LOCUST_LOGFILE=${DOCKER_CSV_PREFIX}_log.txt - - LOCUST_CSV=${DOCKER_CSV_PREFIX} - - LOCUST_CSV_FULL_HISTORY=1 - - LOCUST_HEADLESS=1 - - LOCUST_RUN_TIME=5m - - LOCUST_USERS=10 - - LOCUST_SPAWN_RATE=2 diff --git a/docker-compose.yaml b/docker-compose.yaml deleted file mode 100644 index f88b41dd..00000000 --- a/docker-compose.yaml +++ /dev/null @@ -1,25 +0,0 @@ -version: '3.8' - -services: - prime-exp: - build: . - ports: - - "8089:8089" - environment: - - DEVLOCAL_AUTH=true - - MOVE_MIL_EXP_TLS_CERT - - MOVE_MIL_EXP_TLS_KEY - command: -f /app/locustfiles/prime.py --host exp - prime-exp-reporting: - container_name: mmlt_prime_exp_reporting - extends: - service: prime-exp - environment: - - LOCUST_LOGLEVEL=warning - - LOCUST_LOGFILE=${DOCKER_CSV_PREFIX}_log.txt - - LOCUST_CSV=${DOCKER_CSV_PREFIX} - - LOCUST_CSV_FULL_HISTORY=1 - - LOCUST_HEADLESS=1 - - LOCUST_RUN_TIME=2h - - LOCUST_USERS=1000 - - LOCUST_SPAWN_RATE=10 diff --git a/local_locust.py b/local_locust.py deleted file mode 100644 index cee342f8..00000000 --- a/local_locust.py +++ /dev/null @@ -1,60 +0,0 @@ -# -*- coding: utf-8 -*- -""" -This script updates the /etc/hosts file from within a Docker container to contain the hostnames used in the MilMove -project. ** It should only be run from within a container. ** - -This script also executes a locust command immediately after updating the hosts file. All locust arguments and flags can -be passed in as a string, but otherwise written with the usual syntax. Ex: - -$ python local_locust.py "-f /app/locustfiles/prime.py --host local" -""" -import os -import socket -import argparse - -# NOTE: we cannot use the constants file in this project for these values because this script must be able to run -# independently. -MILMOVE_HOSTNAMES = [ - "milmovelocal", - "officelocal", - "primelocal", -] - - -def update_docker_hosts(): - """ - Grabs the Docker internal IP address using host.docker.internal, then adds lines to the /etc/hosts file with each of - the MilMove hostnames pointing towards this IP address. - :return: None - """ - docker_internal_ip = socket.gethostbyname("host.docker.internal") - - with open("/etc/hosts", "a") as docker_hosts: - docker_hosts.write("\n# MilMove Hosts added by local_locust.py\n") - - for hostname in MILMOVE_HOSTNAMES: - docker_hosts.write(f"{docker_internal_ip} {hostname}\n") - - -if __name__ == "__main__": - command = argparse.ArgumentParser( - description="When executed, this command will update the /etc/hosts file of a running docker container with " - "the hostnames used in the MilMove, then it will execute the command. To run this script, " - "please put the command as the CMD or ENTRYPOINT for a docker container. " - "Put the locust arguments you wish to include in one '' argument for the command." - ) - command.add_argument( - "locust_flags", - type=str, - help="All the locust arguments, as they would be written for locust, should be written in this argument as one " - "string.\n Ex: '-f locustfiles/prime.py --host local'", - ) - - args = command.parse_args() - locust_flags = args.locust_flags.split() # We will need the locust arguments as a list so we can invoke it later - locust_flags.insert(0, "locust") - - update_docker_hosts() - - # Calls locust with the split arguments and stops execution of this (local_locust.py) command: - os.execvp("locust", locust_flags)