Skip to content

Latest commit

 

History

History
128 lines (90 loc) · 5.02 KB

assign.md

File metadata and controls

128 lines (90 loc) · 5.02 KB
Raw

Subject

You may either enter a shared project (on question answering) or choose a project of your own.

If you choose to work on a project of your own choosing, it must:

  • Have readily available data
  • Be of general interest (there should be published guidelines for what constitutes "standard" performance) and should be a problem worth tackling
  • Be of specific interest to machine learning: there should be clear and obvious application of the techniques we have used in the course
  • Have a baseline that you can implement (or run yourself) easily within a week

If you choose to work on a project of your own choosing, you will be held to a higher standard. You must clearly document baselines and show improvement over those baselines. You'll also need to convince us to make sure you clearly convey why it's an interesting problem.

Groups

You must form a group of 4-5 students enrolled in this course. If you have a project idea and you're not able to convince two other people to work on it, it's probably not that interesting. You should instead join another group.

Meeting with Professor

Sometime during the week of 11. October 2017, you'll meet with the professor to discuss your project and to get feedback on your initial ideas.

Proposal

The project proposal is due 20. October. This one page PDF document should describe:

  • What project you're working on. If you're not doing the shared competition, you must describe the task, data, and baselines clearly.

  • Who is on your team

  • What techniques you will explore

  • Your timeline for completing the project (be realistic; you should have your first results by 10. November 2017)

Have the person in your group whose name is alphabetically first submit your assignment on Piazza. Late days cannot be used on this assignment.

First Deliverable

Your first deliverable is due 10. Nov 2017. This is a one page writeup detailing what you've done thus far. It should prove that your project idea is sound and that you've made progress. Good indications of this are:

  • You have your data
  • You've implemented baseline techniques that work
  • You've made some progress on the overall goal of your project

Post your first deliverable report publicly on Piazza.

Final Presentation

The final presentation will be in class on 15. December (at 13:30, not the usual class time). In the final presentation you will:

  • Explain what you did

  • Who did what

  • What challenges you had

  • Review how well you did

  • Provide an error analysis. An error analysis must contain examples from the development set that you get wrong. You should show those sentences and explain why (in terms of features or the model) they have the wrong answer. You should have been doing this all along as your derive new features (e.g., 2b), but this is your final inspection of your errors. The feature or model problems you discover should not be trivial features you could add easily. Instead, these should be features or models that are difficult to correct. An error analysis is not the same thing as simply presenting the error matrix, as it does not inspect any individual examples.

  • The motivation for your features. Features shouldn't come out of nowhere; you should have a clear domain-specific story for how you derived the features or an error analysis motivations for the features you create.

  • Presumably you did many different things; how did they each individually contribute to your final result?

Each group has 8 minutes to deliver their presentation, plus a few minutes for questions and swapping. It is okay to go under time, but do not go over time. It will negatively impact your grade; I will cut you off if necessary. (Sorry to be a jerk about this, but we have enough groups that we cannot have much give on time.)

Project Writeup

By 23:55 15. Dec, have the person in your group whose last name is alphabetically last submit their project writeup explaining what you did and what results you achieved on Moodle. This document should make it clear:

  • Why this is a good idea
  • What you did
  • Who did what
  • Whether your technique worked or not

Please do not go over 2500 words unless you have a really good reason. Images are a much better use of space than words, usually (there's no limit on including images, but use judgement and be selective).

Grade

The grade will be out of 25 points, broken into five areas:

  • Presentation: For your oral presentation, do you highlight what you did and make people care? Did you use time well during the presentation?

  • Writeup: Does the writeup explain what you did in a way that is clear and effective?

  • Technical Soundness: Did you use the right tools for the job, and did you use them correctly? Were the relevant to this class?

  • Effort: Did you do what you say you would, and was it the right ammount of effort.

  • Performance: How did your techniques perform?