Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Toloka interface & info #5

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Toloka interface & info #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f7e55d1
toloka interface v0
maks5507 Nov 22, 2020
060aff8
readme
maks5507 Nov 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions toloka/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Yandex.Toloka assessment system for human-based evaluation

We use human-based evaluation for ELSA.

## Interface

<img src='http://maksimeremeev.com/files/elsa-toloka-interface-1.png'>

<img src='http://maksimeremeev.com/files/elsa-toloka-interface-2.png'>

## Settings

## Questions

## Instruction

## Data

## Results

85 changes: 85 additions & 0 deletions toloka/interface/interface.css
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
.fix_height {
max-height: calc(400px);
display: inline-block;
}

.fix_width {
max-width: calc(45%);
display: inline-block;
}

.left {
vertical-align: top;
}

.page {
overflow-y: scroll !important;
margin-left: 30px;
line-height: 25px;
}

.content {
display: inline-block;
text-align: justify;
margin-left: 10px;
margin-right: 10px;
margin-bottom: 20px;
}

.right {
margin-top: 25px;
margin-left: 30px;
line-height: 25px;
}

.clearfix {
overflow: hidden;
width: 100%;
}

.question {
font-weight: 600;
}

.question-grid {
display: inline;
margin-top: 50px;
}

.question-item {
max-width: 45%;
display: inline-block;
text-align: left;
margin-bottom: 20px;
margin-left: 0px;
}

.question-item-last {
text-align: left;
margin-bottom: 20px;
}

.question-item-penult {
max-width: 45%;
display: inline-block;
text-align: left;
margin-bottom: 20px;
float: right;
}

.source-text-title {
font-weight: 600;
margin-left: 20px;
}

.button {
margin: 10px;
max-width: 182px;
}

.button__label {
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
max-width: 150px;
}
97 changes: 97 additions & 0 deletions toloka/interface/interface.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
<div class="content clearfix">
<div class="left fix_width">
<p class='source-text-title'>Article</p>
<p class="fix_height page">{{source_text}}</p>
<p class='source-text-title' id='aaa'>Summary #1</p>
<p class='page'>{{human_based_summary}}</p>
<p class='source-text-title'>Summary #2</p>
<p class='fix_height page'>{{machine_based_summary}}</p><br>

{{button label="To questions!" href=url_right size="L"}}
</div>

<div class="right fix_width">
<div class='question-grid'>
<div class='question-item'>
<span class='question'>
Q1. Does the summary #1 retain the main idea of the source text?
</span><br>
{{field type="radio" name="s1-main-idea" label="Yes" value=true}}<br>
{{field type="radio" name="s1-main-idea" label="No" value=false}}<br>
</div>
<div class='question-item'>
<span class='question'>
Q2. Does the summary #2 retain the main idea of the source text?
</span><br>
{{field type="radio" name="s2-main-idea" label="Yes" value=true}}<br>
{{field type="radio" name="s2-main-idea" label="No" value=false}}<br>
</div>
<div class='question-item'>
<span class='question'>
Q3. Does the summary #1 syntatically and grammatically coherent?
</span><br>
{{field type="radio" name="s1-coherence" label="Yes" value=true}}<br>
{{field type="radio" name="s1-coherence" label="No" value=false}}<br>
</div>
<div class='question-item'>
<span class='question'>
Q4. Is the summary #2 syntatically and grammatically coherent?
</span><br>
{{field type="radio" name="s2-coherence" label="Yes" value=true}}<br>
{{field type="radio" name="s2-coherence" label="No" value=false}}<br>
</div>
<div class='question-item'>
<span class='question'>
Q5. Does the summary #1 distort any facts present in the source text?
</span><br>
{{field type="radio" name="s1-factual-misrepresentation" label="Yes" value=true}}<br>
{{field type="radio" name="s1-factual-misrepresentation" label="No" value=false}}<br>
</div>
<div class='question-item'>
<span class='question'>
Q6. Does the summary #2 distort any facts present in the source text?
</span><br>
{{field type="radio" name="s2-factual-misrepresentation" label="Yes" value=true}}<br>
{{field type="radio" name="s2-factual-misrepresentation" label="No" value=false}}<br>
</div>
<div class='question-item'>
<span class='question'>
Q7. Does summary #1 contain sentences identical to those found in the source text?
</span><br>
{{field type="radio" name="s1-identical-sentences" label="Yes" value=true}}<br>
{{field type="radio" name="s1-identical-sentences" label="No" value=false}}<br>
</div>
<div class='question-item'>
<span class='question'>
Q8. Does summary #2 contain sentences identical to those found in the source text?
</span><br>
{{field type="radio" name="s2-identical-sentences" label="Yes" value=true}}<br>
{{field type="radio" name="s2-identical-sentences" label="No" value=false}}<br>
</div>
<div class='question-item'>
<span class='question'>
Q9. Would you be able to deduce that a human did not write summary #1?
</span><br>
{{field type="radio" name="s1-machine" label="Yes" value=true}}<br>
{{field type="radio" name="s1-machine" label="No" value=false}}<br>
</div>
<div class='question-item-penult'>
<span class='question'>
Q10. Would you be able to deduce that a human did not write summary #2?
</span><br>
{{field type="radio" name="s2-machine" label="Yes" value=true}}<br>
{{field type="radio" name="s2-machine" label="No" value=false}}<br>
</div>
<div class='question-item-last'>
<span class='question'>
Q11. On a scale of 1 to 5 (5 being nearly identical), how closely does summary #2 resemble summary #1?
</span><br>
{{field type="radio" name="resemblance" label="1" value=1}}
{{field type="radio" name="resemblance" label="2" value=2}}
{{field type="radio" name="resemblance" label="3" value=3}}
{{field type="radio" name="resemblance" label="4" value=4}}
{{field type="radio" name="resemblance" label="5" value=5}}
</div>
</div>
</div>
</div>
28 changes: 28 additions & 0 deletions toloka/interface/interface.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
exports.Task = extend(TolokaHandlebarsTask, function (options) {
TolokaHandlebarsTask.call(this, options);
}, {
onRender: function() {
const task = this.getDOMElement();
task.querySelector('.right').style.display = 'none';

task.querySelector('button').onclick = function(){
task.querySelector('button').style.display = 'none';
task.querySelector('.right').style.display = 'inline-block';
};
},
onDestroy: function() {
// ������� ���������, ����� ����������� (���� ���� ������������) ���������� �������
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// ������� ���������, ����� ����������� (���� ���� ������������) ���������� �������
// The job is completed, you can free (if used) global resources

}
});

function extend(ParentClass, constructorFunction, prototypeHash) {
constructorFunction = constructorFunction || function () {};
prototypeHash = prototypeHash || {};
if (ParentClass) {
constructorFunction.prototype = Object.create(ParentClass.prototype);
}
for (var i in prototypeHash) {
constructorFunction.prototype[i] = prototypeHash[i];
}
return constructorFunction;
}