-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Course projects
During the Practical RL course you can pick a course project. By doing so you will get a deeper understanding of one chosen area of expertise and get away with submitting less homework assignments.
Some possible projects are:
- Applying RL to something weird you have in mind
- Reproducing some nice & shiny article
- Benchmarking known ideas on new tasks
- Doing normal research (new algorithms & stuff)
In general, you are recommended to pick a project if you feel particularly fascinated by some project idea or if you are really bored with normal assignments.
In other words, course project is a great option if you're genuinely interested in some topic, not just going the way of least resistance.
The best possible scenario is when you already have an idea in mind, awaiting the opportunity. Alternatively, we have a list of project examples you can choose from.
When you chose or converged on the project idea, please tell us about that ASAP. Working on your projects in secret and suddenly approaching us with them a few days before course end is a bad practice.
The work on the project is split into milestones(stages). You are awarded points by the end of every milestone.
The first milestone is about reproducing some baseline or creating a minimal working prototype of your idea, depending on project complexity and contents.
Upon reaching the milestone, if you still have time and enthusiasm, you begin a new iteration of approximately the same size. The new iteration may be "trying out this new idea" or "Applying the same idea to several experiments to find out if it reproduces well".
For some projects, one milestone is sufficient. For example, if you'd have settled out to reproduce this guy, it can be done in literally 2 lines of code and 2-3 evenings of runtime.
Finally, project is to be concluded with some kind public activity. At minimum, this includes reporting the results to other fellow students. However, if the results are appropriate, you are encouraged to instead pick something of greater impact (for bonus points ofc):
- Writing a blog post and making sure people read it
- Pull-requesting the modification (if you contribute to opensource libraries)
- Creating a demo[e.g. http://likemo.net/] or a repository[e.g. https://github.com/kuz/DeepMind-Atari-Deep-Q-Learner] and making sure people see it
- Publishing article [arxiv+] and making sure people read it
If your project is six feet under NDA, you can of course choose not to uncover the details.
Points are awarded for each milestone of your project. For example, if you you can get +10 for initial baseline, +20 for some large contribution and +10 for publishing this in a repo. Or +20 for creating an mvp of something new and perspecive, +30 for initial experiments, +20 for more experiments and +30 for finalizing this at nips'17. Except you won't need those points because you're too cool for them.
Each milestone is calibrated out of the intention to make 10-point milestone worth as much effort as 10-point homework assignment.
If you are not sure about any grading tweak, feel free to ask
So in the worst case(idea proven wrong after a few iterations), you can expect your project to yield a total of 40 points or more.
Technically there are no restrictions on the framework you use for your project, you may even write it from scratch. However, there is no guarantee we will be able to aid you with issues outside numpy/theano/lasagne stack.
As a general rule, make sure that you are either a confident user of a framework you chose or have a regular access to someone who is.
Also, please make sure that your framework is compatible with all the project steps you planned and they take reasonable amount of time/effort. Again, if you doubt, ask us.
Teams are allowed but only as far as all teammates have a pre-determined role in the project. E.g.
- your team is benchmarking new publications on VizDoom, each person is responsible for some of the benchmarks
- your team is building a neural translation model; one person optimizes the NN architecture, the other tries to reduce gradient variance
If you feel a bit uneasy on the idea of taking on a course project, please don't be - feel free to ask us any questions you came up with.
It is perfectly legal to submit the same work as both your RL project and some other assignment, including
- some stage of bachelor/master/PHD thesis
- a project from a different course, as long as the other course allows that
- anything, really. We're MIT-licensed after all.