Code Evaluation Script

[christoph-blessing]

Example evaluation script: eval.zip

[mohammadbashiri]

@KonstantinWilleke are you working on this? Or shall I get things rolling here?

[KonstantinWilleke]

@mohammadbashiri Sorry, I haven't worked on this at all yet. Would be great if you roll ahead. What do you think: In terms of evaluation, it would probably make use of the challenge repo to get the measures, right? And when we make this repo and the challenge repo public - the full piprline can be inspected by the participants.

[mohammadbashiri]

Yes, I agree.

[mohammadbashiri]

@cblessing24 is the evaluation script required to be a CLI? In other words, from the example file, which aspects do I need to care about?

[christoph-blessing]

@cblessing24 is the evaluation script required to be a CLI? In other words, from the example file, which aspects do I need to care about?

I think you just need a package/module called eval that contains a function called evaluate. That function should have the same signature as the one in eval/main.py in the example zip file.