Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Span offsets not accounted for in results #46

Closed
evaline-ju opened this issue May 23, 2024 · 0 comments · Fixed by #47
Closed

Span offsets not accounted for in results #46

evaline-ju opened this issue May 23, 2024 · 0 comments · Fixed by #47
Assignees
Labels
bug Something isn't working

Comments

@evaline-ju
Copy link
Collaborator

Describe the bug

The span outputs of results with detection models are incorrect.

Sample Code

curl -v -H "Content-Type: application/json" --request POST \
--data '{"model_id": "model_name",
    "inputs": "There was once a really dumb cat. The chicken crossed the road. The social security number is 123-45-6789. His phone number is (408) 123-4567. ",
    "guardrail_config": {
        "input": {
            "models": {"detector_name": {"threshold": 0.5}
            }
        },
        "output": {"models":{}}
    },
    "text_gen_parameters": {
        "min_new_tokens": 40,
        "max_new_tokens": 200
    }
    }' \
    http://localhost:8081/api/v1/task/classification-with-text-generation

Expected behavior

Span starts and ends should correctly indicate text positions

Observed behavior

Spans all start with 0 in results

{"token_classification_results":{"input":[{"start":0,"end":42,"word":"The social security number is 123-45-6789.","entity":"has_HAP","entity_group":"hap","score":0.00021640431077685207},{"start":0,"end":33,"word":"There was once a really dumb cat.","entity":"has_HAP","entity_group":"thing","score":0.9265003204345704},{"start":0,"end":35,"word":"His phone number is (408) 123-4567.","entity":"has_thing","entity_group":"hap","score":0.0003851974615827203},{"start":0,"end":29,"word":"The chicken crossed the road.","entity":"has_thing","entity_group":"thing","score":0.0013679045950993896}]},"input_token_count":35,"warnings":[{"id":"UNSUITABLE_INPUT","message":"Unsuitable input detected. Please check the detected entities on your input and try again with the unsuitable input removed."}]}
@evaline-ju evaline-ju added the bug Something isn't working label May 23, 2024
@evaline-ju evaline-ju linked a pull request May 24, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants