Skip to content

Commit

Permalink
docs: Rename Task inputs and outputs fields
Browse files Browse the repository at this point in the history
  • Loading branch information
luisaadanttas committed Jul 4, 2024
1 parent 5f84ec5 commit 4ef4739
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 10 deletions.
8 changes: 4 additions & 4 deletions docs/docs/adding_dataset.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ an Engish to French translation task or for a French to English translation task

The Task schema is a formal definition of the NLP task , including its inputs, outputs, and default evaluation metrics.

The `inputs` of the task are a set of fields that are used to format the textual input to the model.
The `output` of the task are a set of fields that are used to format the textual expected output from the model (gold references).
The `input_fields` of the task are a set of fields that are used to format the textual input to the model.
The `reference_fields` of the task are a set of fields that are used to format the textual expected output from the model (gold references).
The `metrics` of the task are a set of default metrics to be used to evaluate the outputs of the model.

While language models generate textual predictions, the metrics often evaluate on a different datatypes. For example,
Expand All @@ -46,8 +46,8 @@ We will use the `bleu` metric for a reference based evaluation.
.. code-block:: python
task=Task(
inputs= { "text" : "str", "source_language" : "str", "target_language" : "str"},
outputs= {"translation" : "str"},
input_fields= { "text" : "str", "source_language" : "str", "target_language" : "str"},
reference_fields= {"translation" : "str"},
prediction_type="str",
metrics=["metrics.bleu"],
),
Expand Down
8 changes: 4 additions & 4 deletions docs/docs/adding_task.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ Tasks are fundamental to Unitxt, acting as standardized interface for integratin

The Task schema is a formal definition of the NLP task, including its inputs, outputs, and default evaluation metrics.

The `inputs` of the task are a set of fields that are used to format the textual input to the model.
The `output` of the task are a set of fields that are used to format the expected textual output from the model (gold references).
The `input_fields` of the task are a set of fields that are used to format the textual input to the model.
The `reference_fields` of the task are a set of fields that are used to format the expected textual output from the model (gold references).
The `metrics` of the task are a set of default metrics to be used to evaluate the outputs of the model.

As an example, consider an evaluation task for LLMs to evaluate how well they are able to calculate the sum of two integer numbers.
Expand All @@ -25,8 +25,8 @@ The task is formally defined as:
from unitxt.blocks import Task
task = Task(
inputs={"num1" : "int", "num2" : "int"},
outputs={"sum" : "int"},
input_fields={"num1" : "int", "num2" : "int"},
reference_fields={"sum" : "int"},
prediction_type="int",
metrics=[
"metrics.sum_accuracy",
Expand Down
4 changes: 2 additions & 2 deletions prepare/tasks/language_identification.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@

add_to_catalog(
Task(
inputs={"text": "str"},
outputs={"label": "str"},
input_fields={"text": "str"},
reference_fields={"label": "str"},
prediction_type="str",
metrics=["metrics.accuracy"],
),
Expand Down

0 comments on commit 4ef4739

Please sign in to comment.