Skip to content

v0.3.0 Creating Labs for JUnit Tests

AnkshitJain edited this page Jan 16, 2018 · 1 revision


  1. Please have your solution ready before you begin the solution setup on AutolabJS.
  2. For the sake of clarity, we will assume that you are planning to evaluate for lab1 in java language.

Setup Solution

Setup of a matching gitlab repository is mandatory before a lab evaluation is setup on AutolabJS. Please follow the steps given below to create and setup a lab solution on gitlab.

  1. Gitlab Login
    Login to gitlab as lab_author. In case you have not created an account with username of lab_author before, please do so before going to the next step.

  2. Create repository
    Create a gitlab repository for the lab. You can create a new gitlab repository by clicking on New Project button. After successful creation of a project repository, Gitlab creates a URL for the project.
    For example, lab1 of lab_author hosted at shall have a URL of

  3. Clone repository
    Clone the gitlab repository onto your local machine.

    $ export GIT_SSL_NO_VERIFY=1
    $ git clone

    This step clones the gitlab repository into lab1 directory of the present working directory.

  4. Create skeleton structure
    Create the following directory structure in the local copy of git repository.

     author_solution             ---> contains the complete solution to the lab
         /java                   ---> contains Java-language solutions
              /lib               ---> contains any third-party Java libraries you would like to use for evaluation
     test_cases/                 ---> contains testcases to be run against submitted code
         /checks                 ---> any sample input / output text files to be used for evaluation
         /java/                  ---> test cases for Java language
             /setup              ---> compile, execute and test-setup bash shell scripts
             ---> script to compile the code for each test (DON'T CHANGE THIS)
         ---> script to execute the code for each test (DON'T CHANGE THIS)
               ---> script to complete first test (calls,
                 (one script for each test case)
             /tests              ---> contains the java-language test files for each test
             ---> test file that runs a test. This file is indirectly called by
     (one language-specific sub-directory exists under test_cases)        ---> script to copy required files for evaluation (DON'T CHANGE THIS)                 ---> Language driver for Java (DON'T CHANGE THIS)
     (one driver exists for each language)
                  ---> orchestrating script for one submission (DON'T CHANGE THIS)
     test_info.txt               ---> text file containing an enumeration of all tests to be run

    You can see the sample directory structure in unit_tests. All the files that have DON'T CHANGE tag in the above listing have to be copied from unit_tests. The other files can be copied and modified according to your needs.

    Inside author_solution and test_cases directories, there is one language-specific sub-directory. One language-specific driver file exists at the top-level. Please copy the needed driver file(s) from unit_tests. You can evaluate C, C++ 2011, C++ 2014, Python2, and Python3 languages.

  5. Evaluation Setup
    Apart from the above directory structure, when the evaluation happens three more directories are created by AutolabJS. They are:

    student_solution        ---> the submitted code gets copied here
    working_dir             ---> a temporary directory used during evaluation
    results                 ---> results of evaluation are stored in this directory
        /log.txt            ---> evaluation log is in this file
        /scores.txt         ---> scores received for all the test cases
        /comments.txt       ---> comments received for all the test cases
  6. Copy the solution
    Copy the lab solution into author_solution/java. Any required third-party libraries need to be copied to author_solution/java/lib.

  7. Create List of Tests
    Copy test_info.txt from unit_tests and modify to suit your needs. Each line of test_info.txt has two fields separated by a tab space. A sample test_info.txt with two tests is shown below.

    Test1	1
    Test2	1

    The first field corresponds to name of the test and the second field corresponds to time limit to run the test. The last line ends without any new line character.

  8. Write tests
    A test should have four matching parts.

    1. Required solution and library files in author_solution/java/
    2. A test setup shell script (ex: test_cases/java/setup/
      Each test setup script has a copy section which helps copy the necessary files from author_solution and student_solution into working_dir.
    3. A Java test class (ex: test_cases/java/tests/
    4. An entry in test_info.txt file (ex: Test1 1).

    Please refer to unit_tests for sample setup. Repeat the above mentioned four steps for all the tests

  9. save and export the solutions to gitlab
    Make a commit in the local repository and push the changes to the gitlab.

    $ git add .
    $ git commit -m 'instructor's solutions'
    $ git push origin master
  10. Follow the details provided on Manage Course page to add the lab to website.

Clone this wiki locally
You can’t perform that action at this time.