howto_autoresults.txt

=========================
HowTo: Autoresult Strings
=========================
Autoresult strings are a short easily parseable output of an autograder when
run against a student's submission.  Autoresult strings (AR) can be delivered
to Autolab in two ways:

1) Remotely: via an HTTP POST request to unofficial.fish.ics.cs.cmu.edu
2) Locally: via a result from an official autograder. 

AR strings represent all the possible information that an autograder can
return about a student's submission, including performance and correctness
metrics.  AR strings can be parsed into scores for the submission as well as
parsed into data for a scoreboard ranking for that student.  Assessments using
AR strings should define the following function

# parseAutoresult(autoresultString,isOfficial)
# @param autoresultString The autoresult string to parse
# @param isOfficial true if the autoresult string is to be considered official
#
# @return 
def parseAutoresult(autoresultString,isOfficial)


Add autoresult as a field of submissions. 

createScoreboardEntry(scores,autoresult)
returns array of column=>value
called per student
scoreboardHeader()

parseAutoresult(autoresultString,isOfficial)
returns map of problemName=>score
submissions.sub_score(scores)
returns a single value representing the sum of the scores

submissions.total_score 
sub_score + tweak + late_penalty

driver submits autoresult string via
1) unofficialSubmit.rb on port 80
->parseAutoresult(AR,false)
->store as submission version 0
->Append to log.txt for assessment
2)autogradeDone.rb
->AR = the last line of the results
->parseAutoresult(AR,true)
->store as submission version n+1
->Append to log.txt for assessment