Track how many characters were present in code part / complete response

[bauersimon]

Tasks

• Introduce 2 assessment keys:
  • AssessmentKeyResponseCharacterCount
  • AssessmentKeyGenerateTestsForFileCharacterCount
• LLM model
  • File: model/llm/llm.go
  • Function: GenerateTestsForFile
  • When parsing the model response, count the number of characters of the response and the test content
  • Add it to the assessments
• Symflower
  • File: model/symflower/symflower.go
  • Function: GenerateTestsForFile
  • Get the output of the symflower unit-tests command
  • Parse the output and return the list of test files generated
  • For each generated file, read its content and count the characters
    • Note: AssessmentKeyResponseCharacterCount == AssessmentKeyGenerateTestsForFileCharacterCount

[zimmski]

AssessmentKeyCharacterCountResponse
AssessmentKeyCharacterCountGeneratedTests

Just looked into what we have right now. Let's do the following to better group assessments:

AssessmentKeyResponseCharacterCount
AssessmentKeyGenerateTestsForFileCharacterCount

[zimmski]

Test it

I think you can always drop that task. Should go without saying. Every task you do should be tested.