Adding a due date for assignments

[iamgoodbytes]

I'd like to use classroom to let students send in assignments and tests, but one requirement is that there is a specific due date and time. After that, students should not be able to send in any more modifications.

[tarebyte]

@iamgoodbytes we don't really have submissions on Classroom for GitHub per se, but one thought is that at a certain time the permissions on the repo could be read only so that the students couldn't push anymore.

[iamgoodbytes]

[... unrelated chat] I'll manually mark them as read-only, it's a bit of a drag with 40 students that have time until midnight for example, because I'll need to stay up and do that part manually :)

[johndbritton]

but one requirement is that there is a specific due date and time. After that, students should not be able to send in any more modifications.

I think this is a common use case that we should support. I've got a few ideas for how we could do this:

• Remove student access to the repository at the deadline.
• Store the latest commit sha on every push that happens before the deadline. Students can still push after the deadline, but the submission will not be used for grading.
• Add a required build status to all repositories created by classroom and don't all merging to master after the deadline. Additional branches can be pushed but not merged to master.

There are probably some other ways that we could implement this, would love to hear more ideas.

[iamgoodbytes]

I think those are some wonderful ideas. All three solutions would work for me. I'll test this first time by giving an assignment and manually changing repo access to read-only. Thanks for following up!

[ppannuto]

I'd like to put a vote in for using the best submission based on timestamp. I question the pedagogical value of stopping students from being able to improve something they've written just because a deadline has passed. Yes, it may not improve their grade, but that shouldn't stop them from continuing to explore / improve the project.

Actually, I would imagine an interesting project policy could have a deadline for a project and a secondary deadline of the end of the semester, which students could earn back up to XX% of their missed points by fixing issues in their graded submission.

Thinking out loud a bit, but hoping to encourage flexible policies here.

[johndbritton]

@ppannuto Thanks for the feedback. I also like the timestamp idea. One thought I had was to give students the ability to select their "submission" up until the due date but lock that down after the due date.

[ppannuto]

I've done that before in two different ways for projects: the first time
was an out-of-band google doc that students copied a commit hash into
(obviously could be done with a web interface). A refinement was to allow
students to add a tag labeled "submission" and use that (though the grader
also had to check the timestamp of the commit it pointed to). Students
liked the tag mechanism because it was clear and easy to reference their
submission and tracked in the repository. May be an approach worth
considering as well.

On Thu, Dec 10, 2015 at 1:56 PM John Britton notifications@github.com
wrote:

@ppannuto https://github.com/ppannuto Thanks for the feedback. I also
like the timestamp idea. One thought I had was to give students the ability
to select their "submission" up until the due date but lock that down after
the due date.

—
Reply to this email directly or view it on GitHub
#379 (comment)
.

[samwhitlock]

@johndbritton I like the "submission" selection. One useful thing in this model would be for the "submission" to include all remote branches. For the classes I've taught, we often want students to push remotes to github to see that they are indeed doing proper software development (e.g. feature branches).

As a "failsafe", the state of all repositories at the submission deadline should also be recorded. I foresee a bunch of students complaining about barely missing the web-based "record your submission" thing, and in some cases I'd like to have the ability to just take their most recent, pre-deadline repo state and use that to grade.

[johndbritton]

though the grader also had to check the timestamp of the commit it pointed to

It's possible to manipulate the commit timestamp. That's why I aim to have a hash recorded at the time the assignment is due.

As a "failsafe", the state of all repositories at the submission deadline should also be recorded.

This might not be entirely feasible, due to the numbers of assignments and storage requirements, and overhead of making sure all the copies work every time.

@tarebyte recently suggested using locked branches to stop students from submitting after the due date.

[samwhitlock]

Actually, the locked branches feature seems like the way to go. I wasn't aware that github had them now 😄

[amitojsingh]

Hello,

but one requirement is that there is a specific due date and time. After that, students should not be able to send in any more modifications.

As we are talking about assignments and test so due date or deadline is must.
I like your idea on how to implement it @johndbritton

Remove student access to the repository at the deadline.
Store the latest commit sha on every push that happens before the deadline. Students can still push after the deadline, but the submission will not be used for grading.
Add a required build status to all repositories created by classroom and don't all merging to master after the deadline. Additional branches can be pushed but not merged to master.

👍
I want to add few points in it.

• First we will have to add a calendar with time while creating an assignment for deadline.
• After that as you are saying To " Remove student access to the repository at the deadline". moreover students get informed through mails while deadline gets closer.
• Work is submitted by student to the Mentor and Mentor evaluate the work (we can add here some gratitude-based immaterial rewards so that students get motivated.)
• After the evaluation, student again get the write access to the repository so that they can push their improvement or modification in work.
• All the status of repository is recorded up to the deadline as you said .

This will reduce the overhead for students as well as for mentors.

I would like to work on this project and want to know if anyone want to add more points in it?
@ALL Give their valuable reviews. :)

cheers!

[tarebyte]

@amitojsingh are you a University student / interested in participating in Google Summer of Code?

[amitojsingh]

@amitojsingh are you a University student / interested in participating in Google Summer of Code?

Yes @tarebyte

[tarebyte]

@amitojsingh then I'd ❤️ to 👉 you to github/mentorships#91.

[tarebyte]

@amitojsingh you've already commented on it :derp:

Ignore my comment

[ppannuto]

After the evaluation, student again get the write access to the repository so that they can push their improvement or modification in work.

I'm not a big fan of the arbitrary read-only window. I don't think that encourages good action for students. I don't really want anything that would discourage people from working on things. It feels like we should be able to develop a mechanism that grabs a snapshot at the deadline but doesn't impede student's ability to keep working if they so desire.

[tarebyte]

@ppannuto we could grab a snapshot of the sha from when the assignment is due and then when present the link to the professor it uses that sha

[johndbritton]

It feels like we should be able to develop a mechanism that grabs a snapshot at the deadline but doesn't impede student's ability to keep working if they so desire.

We're not planning to snapshot any code from GitHub, but we can easily grab the commit SHA as @tarebyte suggests. The most comprehensive way to do this would be to grab the SHA of the latest commit to each branch that was pushed to GitHub before the deadline. With this info, we can easily determine if any commit was submitted before or after the assignment deadline.

I'm not a big fan of the arbitrary read-only window.

There are a lot of competing desires for features around how due dates are implemented. Our goal is to implement something that will work for the largest group of users. We're considering using protected branches to make it impossible to merge to master after the due date but still allow students to commit to other branches.

[amitojsingh]

@johndbritton What if student want to submit their assignment before the deadline?

We're considering using protected branches to make it impossible to merge to master after the due date but still allow students to commit to other branches.

If we can do this it would be great.

[tarebyte]

@johndbritton What if student want to submit their assignment before the deadline?

@amitojsingh they could just stop to working on it if they were done. ¯\_(ツ)_/¯

Is there a use case for being able to submit early?

[amitojsingh]

Is there a use case for being able to submit early.

Suppose their is a assignment with deadline of 1 week and a student completed in a day or two. Now he has to wait for the deadline so that their assignment is to be submitted!

[tarebyte]

Suppose their is a assignment with deadline of 1 week and a student completed in a day or two. Now he has to wait for the deadline so that their assignment is to be submitted!

With the way the discussion is going it sounds like the student doesn't have to submit the assignment to begin with. We just take a snapshot (in one way or another) so that the teacher knows what to look at.

Is there benefit to alerting a teacher that an assignment is done early? As someone who used to grade projects even if someone gave it to me early I would grade them all at the same time. It's just easier, and it helps me get an idea of how the class did as a whole.

I think it's just more work for ourselves to add an option to submit if we can just automate it. But that's just my personal opinion.

[samwhitlock]

It's not about when the instructor does the grading; it's about the students' ability to signal which version should be used for grading.

Consider students writing a Rails app for a class. Let's assume that they are new to git and/or don't use best/practices. This often means that master may be broken, or not passing tests.

If the approach for a deadline is for Classroom to just get all the commit IDs on the master branches of the repositories, students who have already finished the stuff for assignment N and are working on N+1 (which many classes release before N is due) may have broken their master branch with the material they're working on for assignment N+1, even though the material for N works.

As a TA who had to sort out the above mess a few times, I know it's pain in the ass.

My suggestion for Classroom for assignments would be the following:

• Allow instructors to specify an assignment tag, e.g. weekN.
• When an assignment is due, Classroom collects the commit ID of the commit specified by the assignment tag as well as the commit ID of master. Collecting both would be necessary for most institutions, or at least my own (I've had loooong discussions with a few admins/bureaucrats at my school about these sort of MOOC-ish details).

For grading, check out the assignment tag's commit and use that if it is available, else default to the master tag.

Thoughts?

[tarebyte]

@samwhitlock @amitojsingh

If that's the case what do we think about making an assignment page per user/group? It would be permission locked obviously and then all the student has to do is go to the page and click submit.

[johndbritton]

What if student want to submit their assignment before the deadline?

The student could stop working and all the commits in the repository before the deadline would be considered "on time."

it's about the students' ability to signal which version should be used for grading.

I think this is a feature we should have. Students should have the ability to select which commit they intend to use as their submission.

If the approach for a deadline is for Classroom to just get all the commit IDs on the master branches of the repositories

The idea I put forward before was to to get the latest commit SHAs from all branches at the due date. This means that any commit at, or before, the recorded SHAs was available on time.

students who have already finished the stuff for assignment N and are working on N+1 (which many classes release before N is due) may have broken their master branch with the material they're working on for assignment N+1, even though the material for N works.

In this case the student could select any commit that came before the commits recorded at the due date. I quite like your idea of using a tag for that but that also means that student will have to learn to use tags.

I like the idea of making the more complex features optional. If a teacher wants to use tags they can, but it shouldn't be a requirement to use Classroom.

[samwhitlock]

In this case the student could select any commit that came before the commits recorded at the due date. I quite like your idea of using a tag for that but that also means that student will have to learn to use tags.

I'm teaching a beginner programming course now (this one, only in French), and we "solve" this issue by just using a done directory -> students commit their work in done/weekN. Reminds me of more primitive version control systems 😛

Regardless of how it ends up being implemented, not having a way for students to mark a commit for the "final" version of an assignment (i.e. the "just stop working for a few days" option) would likely be a deal-breaker for adoption by some instructors because it could feasibly hold back eager/motivated students.

Like @johndbritton said, there's no reason to require this, but most profs I've worked with like to release each week's assignment roughly 1.5-2 weeks before it's due, leading to some overlap. There should be some mechanism to deal with this.

I think tagging is reasonably easy to teach to even novice students. They already have to deal with more complex issues almost immediately, even if they're just working by themselves on master (e.g. how to "build" a commit with git add; I've had to walk through this concept with many beginner students this semester).

[arjenpdevries]

I like the proposed method:

We're considering using protected branches to make it impossible to merge to master
after the due date but still allow students to commit to other branches.

Can we already do it this way?
(What would be the CLI commands to protect a branch?)

[johndbritton]

What would be the CLI commands to protect a branch?

@arjenpdevries I think you're suggesting that you could lock all the master branches manually to enforce a due date right now. That's definitely possible, but it'd take you writing a simple script to hit the API and lock the branch. Here's the relevant API method. Please note that this is a preview API and as such requires that you send a custom media-type in your accept header.

[dhowe]

Any progress on this? Or is there some other submission (mark as final/lock branch) feature being implemented somewhere?

[cyhsutw]

Hi @dhowe 👋, thanks for commenting.

While the due date feature is not yet implemented, we have a possible way to let students specify the version of their works to be graded.

In #720, students need to create releases in their repos and we'll using the publishing timestamps as their submission dates.

edit: fixing the link to GitHub Help.

[jayvdb]

Editing the tag for a release is able to be detected?
https://help.github.com/articles/editing-and-deleting-releases/

[maurice-schleussinger]

As a (temporary) workaround I wrote a script which pulls all repositories from a given list and applies a checkout for the given deadline. (assuming the students repos all have their latest work on the master branch).

This was an easy way for us to ensure we only grade the students work up until a deadline without having them fumble around with tags, branches and such (many students were not familiar with these).

Script: https://github.com/I4-Projektseminar-HHU-2016/info/blob/master/pull_on_deadline.py

In order to run the script you need git installed and also have to install GitPython:
pip install GitPython==2.0.8

The script is ATM only tested with OSX and python 3.5.2 but should also work on other platforms

If people are interested I could move it to a own repository to track issues, feature requests etc.. 🙂

[sxlijin]

So I'm reminded of XKCD, a la

but we ended up developing our own workflow as well - none of us had realized that GH Classroom was open-source - which grabs the tree corresponding to master, checks out a feedback branch, resets HEAD to point at the tree corresponding to the code that they were provided (we keep this in a skeleton repository, skeleton, with a skeleton-ugrad and skeleton-grad branch for undergrads and grads taking the class, since they get different but similar assignments), and then commits the entire working tree.

The effect is that all of their code on the most recent commit on master before the deadline is squashed against some skeleton code, which also enables us to use GH commenting to provide feedback. If there's interest, I'm happy to release it (it's a Bash script with a Python script interpreting command-line flags).

I'll take a crack at PR'ing something that will automatically lock branches.

[sxlijin]

Cobbled together a Python script that doubles as a CLI tool for managing branch protection. Warning: the protected branches API is still under preview and is not stable, so there's no guarantee that this will work at any given point in the future.

[nwoodthorpe]

This is now a feature in production!

[arjenpdevries]

Thank you!