hulqBASH

hulqBASH: Hosted Universal Language Qualifier in BASH

Description

A source code evaluation REST service that downloads two files from the S3 bucket used for AES, parses out the command line argument sets from the solution file (in the format specified below), then executes them against eachother through bash scripts.

Evaluation

We are hosting an independent service (on a separate EC2 instance) to compile, execute, and evaluate source code.

Languages Currently Supported: Java(8)[.java], C[.c], C++[.cpp], C#[.cs]

Uploads

An S3 bucket in Amazon uses a key instead of a filename to reference a specific file.

Code snippet template should be named with the key = template_templateId in the database.

• This file is an incomplete source file (to be provided to the individual taking the assessment).

Code snippet solution should be named with the key = solution_templateId in the database.

• This file is a complete source file (to be used for evaluation, containing command line argument sets).

Config

Use the following schema to define configuration for evaluation

  @Config
  mathmode:false
  trimwhitespace:false
  extractwhitespace:false
  caseignore:false
  grosserrormargin:0.0
  caseerrormargin:0.0
  keyfilename:key.cpp
  testfilename:test

Solutions

Use the following schema to define command line arguments (note: individual command line arguments cannot contain white space, single quotes, double quotes, or back-ticks). The comment block should be in the solution file.

Also, for the purposes of grading, only the OUTPUT of a file will be compared to the output of the solution for grading. Thus, the solution file MUST include a system.out.println(), << cout, printf(), etc. And the prompt should indicate that the submitted file must include the same for grading purposes.

/*
  @ArgSet(1)
  argument1 % Comment about argument1
  argument2 % Comment about argument2
  argument3 % Comment about argument3
  argument4 % Comment about argument4

  @ArgSet(2)
  argument1 % Comment about argument1
  argument2 % Comment about argument2

  @ArgSet(3)
  argument1 % Comment about argument1
  argument2 % Comment about argument2
  argument3 % Comment about argument3
*/

Each @ArgSet(X) defines a test case. For example, if we want to test a Palindrome class, we would use

java Palindrome argument1 argument2 argument3 argument4

for ArgSet(1),

java Palindrome argument1 argument2

for ArgSet(2), and

java Palindrome argument1 argument2 argument3

for ArgSet(3).

---

Full Example

public class Palindrome
{
    public static boolean isPalindrome(String s)
    {
        if(s.length()==0)
            return false;
        else if(s.length()==1)
            return true;
        else if(s.length()==2)
        {
            if(s.charAt(0)==s.charAt(1))
                return true;
            else
                return false;
        }
        else if(s.charAt(0)==s.charAt(s.length()-1))
            return palindrome(s.substring(1,s.length()-1));
        else
            return false;
    }

    public static void main(String[] args)
    {
        System.out.println(isPalindrome(args[0]));
    }
}

/*
  @Config
  mathmode:false
  trimwhitespace:false
  extractwhitespace:false
  caseignore:false
  grosserrormargin:0.0
  caseerrormargin:0.0
  keyfilename:key.cpp
  testfilename:test

  @ArgSet(1)
  racecar % true

  @ArgSet(2)
  thisisnotapalindrome % false
  bob % this second argument is ignored

  @ArgSet(3)
  anna % true

  @ArgSet(4)
  raceCar % false (capitalization)
*/

Execution:

> java Palindrome racecar
true
> java Palindrome thisisnotapalindrome bob
false
> java Palindrome anna
true
> java Palindrome raceCar
false

Implementing Additional Languages:

In general, enabling additional languages in the service should be as simple as adding an else if case to the bash script that specifies the compilation command, output filename (if relevant), and execution command.