feat: create responses and suggestions with spans

[frascuchon]

Description

This PR defines a basic flow to support responses and suggestions with span values.

import spacy
from datasets import load_dataset

import argilla as rg
from argilla.client.feedback.schemas import SpanValueSchema
from argilla.client.feedback.schemas.responses import ResponseSchema, ResponseBuilder


api_url = ...
api_key = ...

rg.init(api_url, api_key)

nlp = spacy.load("en_core_web_sm")
labels = nlp.get_pipe("ner").labels

gutenberg_config = rg.FeedbackDataset(
    fields=[rg.TextField(name="text")],
    questions=[rg.SpanQuestion(name="spans", title="Name entities recognition", field="text", labels=labels)]
)

ds = gutenberg_config.push_to_argilla("gutenberg_spacy-ner", workspace="argilla")
hf_ds = load_dataset("argilla/gutenberg_spacy-ner", split="train")

user = rg.User.me()

def to_response_spans(doc) -> list[SpanValueSchema]:
    """Converts a spacy doc to a list of response spans."""
    return [
        SpanValueSchema(start=ent.start_char, end=ent.end_char,label=ent.label_)
        for ent in doc.ents
    ]

question = ds.question_by_name("spans")
records = [
    rg.FeedbackRecord(
        fields={"text": r["text"]},
        responses=[
            ResponseSchema(
                status="draft",
                user_id=user.id,
                values=[question.response(to_response_spans(nlp(r["text"])))]
            )
        ],
        suggestions=[
            question.suggestion(value=to_response_spans(nlp(r["text"])), agent="spacy", type="model")
        ]
    )
    for r in hf_ds
]
ds.add_records(records)

Closes #4618

Type of change

(Please delete options that are not relevant. Remember to title the PR according to the type of change)

• New feature (non-breaking change which adds functionality)
• Refactor (change restructuring the codebase without changing functionality)
• Improvement (change adding some improvement to an existing functionality)

How Has This Been Tested

(Please describe the tests that you ran to verify your changes. And ideally, reference tests)

• Test A
• Test B

Checklist

• I added relevant documentation
• I followed the style guidelines of this project
• I did a self-review of my code
• I made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I filled out the contributor form (see text above)
• I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

[davidberenstein1957]

perhaps we should add else statements here to be sure it raises errors/has behaviour when we add question types?

[frascuchon]

I'm using the current approach to support span values mappings. I wouldn't add cross-cutting solution in the current version of the SDK. This is something that we can add for new SDK implementation

[davidberenstein1957]

perhaps we should add else statements here to be sure it raises errors/has behaviour when we add question types?

[frascuchon]

I'm using the current approach to support span values mappings. I wouldn't add cross-cutting solution in the current version of the SDK. This is something that we can add for new SDK implementation

[davidberenstein1957]

in terms of human-readability, I also like to add the extracted text normally. Something likevalue["text"][value["start"]:value["end"]] but perhaps this is difficult top map back into the correct format when calling from_huggingface?

[frascuchon]

This change is not hard to add, if it makes sense.

[davidberenstein1957]

in terms of human-readability, I also like to add the extracted text normally. Something likevalue["text"][value["start"]:value["end"]] but perhaps this is difficult top map back into the correct format when calling from_huggingface?

[frascuchon]

This change is not hard to add, if it makes sense.

[davidberenstein1957]

wasn't there a max for the labels too which we defined on the server side? perhaps we can use it here as well?

[frascuchon]

Yes, this is not hard to implement.

[frascuchon]

Anyway, this is not the current behaviour that exists for single and multi-label settings. In both cases, there is a min validation but not a max one.

Also, if we plan to support a configurable value for this, adding a hard validation here may introduce workflow problems. So, I will let as is.

[davidberenstein1957]

I saw we already tested for larger than 0 but do we also validate if the span is within the overall text field, i.e., they are not larger than the used text?

[frascuchon]

We're covering basic data validation from SDK, adding more complex validations is not quite simple to implement for this SDK. Since there are these kinds of validations from the server, and the rest of the suggestions/responses didn't include validations, I would keep it simple for this version.

[davidberenstein1957]

perphaps we should return None explicitly here? not 1 and not True also pass the if-statement and perhaps we require an explicit None to set the user_id automatically?

[frascuchon]

This code was already here. I just moved it. I wouldn't change anything except the span feature. This can be tackled in the new SDK.

[davidberenstein1957]

to be clear, all of "type", "score", "value", "agent" are stored but not all of them are used, correct?

[frascuchon]

'score' is used to filter records at least from UI. The rest are not used yet, even if they're available for searches.