5. Testing and Debugging in Docker with Visual Studio Code

The REopt API has a built-in testing framework, where each Django app has a test directory that is run automatically via the Django manage.py function.

Note that the test framework does not rely on the Celery nor Redis servers. Tasks are run in "eager" mode when testing and are not managed. Also, tests do not affect your "reopt" Postgres database - the test framework creates a test database for the tests and then deletes the test database after the tests complete.

Some tests intentionally raise exceptions in order to test the API's ability to gracefully handle problems. In these cases you will see an error message in the terminal - but you should also see an OK or a dot after the test when it passes, depending on your verbosity settings.

Running tests inside of the "django" container for debugging in VSCode

Visual Studio Code (VSCode) has some really nice integration features for Docker. Install the "Remote – Containers" extension for VSCode. See detailed documentation of Remote - Containers for setup instructions. Spin up Docker containers as described in Setup for Local API Development. VSCode also has a nice Docker and Git extension to interface with both of those tools, so it can really be a complete IDE for API development.

We are going to use VSCode’s “Remote – Containers” extension to open up VSCode from “within the container” and then run a test (python manage.py test reo.tests.test_xyx) using the Python extension to allow for nice debugging features like setting breakpoints dynamically during the test runtime, interfacing with local variables, and stepping through the code.

After spinning up Docker containers, navigate to the “Remote – Containers” extension, and you should see a list of running containers (see image below, but yours may not have a "Dev Containers" folder yet). Click on the "Attach to Container" icon to the right of the django container (reopt_lite_api_django_1) which opens up a VSCode window in “attach mode” (attaches to the container).

The picture below shows what the VSCode window looks like when it opens within the container after you do the initial setup described next. The first time you open up the container, you’ll have to open up the folder within the linux-based container which contains the repository code (the whole repo was copied into the container file system); this is in the /opt/reopt directory.

After you've opened the folder the first time, VSCode will store the location of this folder and it will install the same extensions as you do the first time so that it is "ready to go" for subsequent times you attach to the container.

Note, the repo code is “mounted” from your local machine to this container’s folder, and any changes made here will also be made on your local machine.

Install the Python extension in your container. You can find this by searching for "Python" in the Extensions Marketplace.

Important: Add the container's local Python environment as your Python interpreter (from /usr/local/bin/python). If Python is not showing up as a debug option, move on to the next step “Set up launch.json file”) and then come back to this and hopefully Python should be an option.

Go to the Python extension in your VSCode. The first time you do this it will ask you to make a launch.json file which you should do. You can also create a launch.json file by clicking the "Run and Debug" icon, then clicking the gear (should say "Open "launch.json"" when you hover.)

Make your launch.json file look like this, but replace test_your_file with whatever test you want to run (also make the name whatever you want).

For REopt API v1-v2 (which is within the "reo" app):

Replace "test_your_file" with any test folder under reo/tests (e.g. test_third_party) or create your own test file and point the args to that file. You can even specify down to specific tests, e.g., "reo.tests.test_curtailment.WindTests.test_curtailment"

{
    "version": "0.2.0",
    "configurations": [
       { 
            "name": "Python: manage.py test", 
            "type": "python", 
            "request": "launch", 
            "program": "${workspaceFolder}/manage.py", 
            "console": "integratedTerminal", 
            "args": [ 
                "test", 
                "reo.tests.test_your_file", 
            ] 
       }
    ]
}

For REopt API v3+ (which is within the "reoptjl" app):

Follow the instructions above but change the "args" to the following, where "test_your_file" is an existing test file (e.g., "test_job_endpoint" or "test_job_endpoint.TestJobEndpoint.test_pv_battery_and_emissions_defaults_from_julia") or your own file:

"args": [
                "test",
                "reoptjl.test.test_your_file", 
            ]

Now you can set breakpoints wherever in the code, and use the Python extension to “run and debug” (which runs in debug mode and will stop at your breakpoints). You will get a nice “locals” and “globals” window to see the status of all the objects at the state of the breakpoint. You can drop more breakpoints in as needed, and step through the code using the Python extension tool bar.

You can check the console output of the tests in the "REopt_API" VSCode window (not the container).

Running tests without debugging

You can also run these tests without using the launch.json (if you do not wish to use the "run and debug" functionality) by typing in a terminal (at the top level of the "REopt_API" directory): To run the full test suite:

python manage.py test

To see more output from the test suite (eg. which test is currently running:

python manage.py test -v 2

A good option to know is --failfast, as in:

python manage.py test --failfast

which will stop the test suite as soon as one test fails or errors.

You can also test individual apps:

python manage.py test reo.tests

Or an individual test function:

python manage.py test reo.tests.test_wind

All they way down to a single test within an app's test suite:

python manage.py test reo.tests.test_reopt_url.EntryResourceTest.test_complex_incentive