Final Project Evaluation

Final Project Evaluation

	Reject	Conditional Accept (Recognition)	Weak Accept (Reasoning)	Strong Accept (Radiant)
Software Ecosystem	- No discernible software ecosystem	- Project is in Haskell using stack - Or other languages/tools if needed/justified - Project is version controlled	- Uses relevant external libraries from Hackage (beyond Template Haskell) when appropriate - Code is sufficiently modular	- Project is easily cloned, compiled/run, and understood - Project supports integration with related tools where desired/appropriate - Software ecosystem is appropriate for the end-user's needs
Software Engineering (Functional Programming)	- Lack of code artifacts or explanation of how the language is going to work in reference to the implementation - Code given refers to engineering tasks that can't be done within the language	- There is some code provided, but it's not idiomatic - The engineering aspect is explained, but it's vague and imprecise - The code has trivial refactors where code could have been changed around, but it hasn't been done	- The code is clean, idiomatic, and with good documentation - The documentation is a little weak in points, but relatively clear	- Project takes advantage of relevant host language features - Code explanation is precise - A user can interact with the language with minimal difficulty
Abstract Syntax and Internal Representations	- No abstract syntax - The internal representations are too complex, and don't maintain the invariants they claim	- Contains abstract syntax, but it missing some important things	- The foundation is good, but there are a few unnecessarily complex features	- The representation and syntax is appropriately tailored to the domain - There are no unnecessary, extra things, but it's not lacking anything important - Ideally, the representation enables domain-specific optimizations/representations
Related Work and Background (Technical Reading)	- No mention of related work - No mention of domain background - No mention of libraries used	- Mention but no comparison to related work	- Mentions other work, with maybe either a strong summary or a strong comparison to current project, but not both - Example: Slideshow	- Discusses in sufficient detail the differences between current project's implementation and the related work's implementation - Explains what the related work did, and compares and contrasts - Example: Frenetic
Clarity (Technical Writing and Rhetoric)	- No clear flow of ideas - Heavy usage of undefined terms - No stated claims or goals for the paper - Consistently bad spelling and grammar	- The claims are hard to find - The flow is confusing, but exists - Some terms defined, but not all - Poor spelling and grammar	- The claims are weak, or there are only a few - The flow could be improved a bit - Most terms are defined - Only a few spelling and grammar mistakes - Example: Slideshow	- The claims and flow are clear - All terms are well-defined for the target audience - Two or fewer spelling/grammar mistakes - Example: Frenetic
Documentation and Usability	- No documentation or usable examples	- The documentation and usability are present, but don't clearly tie into writing a program in the language	- Either strong documentation or good examples	- Contains both strong documentation and good examples
Code Generation, Compilation, and Metaprogramming	- No codegen or metaprogramming	- Codegen is not necessary or is not adapted from user input - Usage of metaprogramming seems cursory or unmotivated	- Metaprogamming does simplify user experience in some way, but implementer has not explicitly explained why it is useful - Example: Slideshow	- Usage of metaprogramming heavily simplifies artifact generation - Clear understanding of how code would like without metaprogramming and why metaprogramming simplifies user experience - Is clear about what the audience benefit should be if we show "under the hood" - Example: OptiML, Frenetic - If implementation has no codegen or metaprogramming, its exclusion is justified
Concrete Syntax and User Interface (Language FrontEnd Design)	- No explanation of concrete syntax present	- An expert in the domain would find it difficult to read the paper and relate the work back to the domain	- The concrete syntax and UI are well-defined, but the project doesn't take into account biases of the user base and their experience	- The reasoning behind all design decisions are apparent to the audience - An intended user could easily write a program in the language after reading the paper or watching the video - The level of detail for the video/paper is appropriate for the intended audience - The paper and video are both clear on who the intended audience (they could be different)
Evaluation (Composing Feedback and Evaluation)	- There is no evaluation at all	- There is evaluation, but in a non-useful way - The paper and/or video doesn't compare to the current standard or contains only minimal/irrelevant evaluation	- Within the bounds of the paper and the video, there is an evaluation of most of the relevant criteria	- Within the bounds of the paper and the video, there is evaluation of all relevant criteria
Discussion and Future Work (Metacognition)	- Does not discuss how the claims or goals were achieved - No discernible discussion whatsoever	- The discussion of the goals is disjointed from the original claims/goals stated - Discussion is generally missing substantial parts to it	- The paper does not talk about the questions raised by the work There is no discussion of the applicability of the result - The discussion is lacking slightly in some way	- The paper discussed the most important goals they tried to achieve and why they did or did not achieve them - The paper discussed concrete ways that the work could be extended or improved upon, delineating between 'future work' and 'non-features' of the DSL - The paper identified interesting or relevant questions raised by the work - The paper discussed the applicability of the results