Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Assignment view with approval features. #1261

Merged
merged 20 commits into from
Nov 1, 2018
Merged

Assignment view with approval features. #1261

merged 20 commits into from
Nov 1, 2018

Conversation

JackUrb
Copy link
Contributor

@JackUrb JackUrb commented Oct 31, 2018

Introduces a new view to the frontend that allows viewing assignments that have been worked on and are in any completed state. This works by leveraging the directory structure established in #1253 and the database from earlier.

This was the table of info from before:
screen shot 2018-10-31 at 5 25 05 pm

This is the new assignment view you can get to by clicking through an assignment id:

screen shot 2018-10-31 at 5 25 47 pm

It scrapes task description instructions from the task_config.py file in the run's task folder (checking both parlai and parlai_internal folders).

screen shot 2018-10-31 at 5 26 03 pm

It can find onboarding chats if they were saved for an assignment and knows to avoid them otherwise.

Exist:
screen shot 2018-10-31 at 5 53 10 pm
Not exist:
screen shot 2018-10-31 at 5 26 16 pm

It has an approval flow at the bottom that allows approving and rejecting (with reason):

screen shot 2018-10-31 at 5 26 26 pm
screen shot 2018-10-31 at 5 26 46 pm

The approval flow knows the current assignment state, (approved, rejected) and can override rejected HITs to acceptances.

screen shot 2018-10-31 at 5 27 24 pm
screen shot 2018-10-31 at 5 26 56 pm
screen shot 2018-10-31 at 5 27 14 pm

It also has a bonus modal:

screen shot 2018-10-31 at 5 28 04 pm

And a block modal:

screen shot 2018-10-31 at 5 29 06 pm

Unordered list of next steps

  • Functionality for converting approved assignments into a ParlAI-Formatted dataset
  • Better displays for the content in the tables
  • Implement run status tracking
  • Keep track of worker blocks to swap block to unblock for a worker
  • Context displays (requires moving frontend to React)
  • Provide the block option in the workers page for a worker
  • Create a view that allows stepping through
  • Add qualification tracking
  • Add a view for testing hits without needing to launch heroku

@alexholdenmiller
Copy link
Member

When you get the chance, please fix the unit test errors :)

@JackUrb
Copy link
Contributor Author

JackUrb commented Nov 1, 2018

@stephenroller is helping me out with that one on the travis side 👍

Copy link
Member

@alexholdenmiller alexholdenmiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow the screenshots look awesome! just need the unit test fixes then approved

parlai/mturk/core/mturk_data_handler.py Outdated Show resolved Hide resolved
parlai/mturk/webapp/server.py Outdated Show resolved Hide resolved
Copy link
Contributor

@klshuster klshuster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome, just some questions/comments

with self.table_access_condition:
conn = self._get_connection()
c = conn.cursor()
reason = "${} for {}\n".format(amount, reason)
# Bonus amount is stored in a cents int in the SQL table
cent_amount = int(amount * 100)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we enforce somewhere else that bonuses are set in dollar amounts?

edit: i notice that is the case for the webapp, curious if this is still enforced elsewhere (or if that even matters for this particular function)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't necessarily enforce it, though this call is only made from MTurkManager, and that API suggests the bonus amount is in dollars. The cent representation is only ever seen on the backend and it only exists for SQL storage reasons.

@@ -669,3 +727,39 @@ def get_all_task_pairings_for_worker(self, worker_id, task_group_id=None):
(worker_id, task_group_id))
results = c.fetchall()
return results

@staticmethod
def get_conversation_data(task_group_id, conv_id, worker_id, is_sandbox):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for some reason, this way of returning the data seems very satisfying to me

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the data is in an organized place, it's really easy to access it. The accessors along the way make sure that if someone manually alters the folder structure there is no catastrophic failure and I can give a good reason.

parlai/mturk/webapp/dev/app.jsx Outdated Show resolved Hide resolved
@@ -543,6 +638,679 @@ class RunPanel extends React.Component {
}
}

class AssignmentInstructions extends React.Component {
render() {
let instructions = this.props.data;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this similar to swift, where let gives you an immutable/constant ref?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let is limited in scope to the current block. I should be moving more of my vars to be lets, but I haven't taken the time to clean yet.

find_location = 'parlai.mturk.tasks.{}.task_config'.format(taskname)
find_location_internal = \
'parlai_internal.mturk.tasks.{}.task_config'.format(taskname)
try:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suppose you have two versions of a task, one internal and one external (e.g. you're extending an external task internally for a project), is there a way to specify overriding this order and using the internal task instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now I haven't added any flag to do such a thing. I'm unsure how frequently we'd run into a problem like this though, as usually things that end up public aren't collecting any more data, and if you're making an extension on a task it'll have an extended name (usually).

@JackUrb
Copy link
Contributor Author

JackUrb commented Nov 1, 2018

Merging with build failures because travis is having issues binding sockets today.

@JackUrb JackUrb merged commit 9d8a6ee into master Nov 1, 2018
@JackUrb JackUrb deleted the review-frontend branch November 1, 2018 20:56
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants