Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

default blank row of launch_gradio_widget in web demo app may confuse users #327

Closed
Raibows opened this issue Oct 18, 2022 · 1 comment · Fixed by #335
Closed

default blank row of launch_gradio_widget in web demo app may confuse users #327

Raibows opened this issue Oct 18, 2022 · 1 comment · Fixed by #335

Comments

@Raibows
Copy link
Contributor

Raibows commented Oct 18, 2022

I notice that the default row of dataFrame of launch_gradio_widget is set to 2. Since the gradio does not provide delete_row option in the webui interface, the calculation of some web demos (e.g., exact_match, I don't test all) includes the blank row, which may confuse the user.
For example, if we as the users only hope to calculate the exact match score of one sample, thus we only fill the first row in the webui and left the second unremovable row blank, the returned calculation results is always wrong since it counts the second blank text row as a 1.0 score (exactly match! because they are both 0 length '' str).
The above example is shown in this figure.

1664165296420-633122d3f242a8532b7a928d

Actually, I have posted a discussion in huggingface web demo nearly one month ago and no one responsed.

Here are some potential useful suggestions to fix this issue:

  1. Considering the upstream package gradio is not tending to add delete a row option (related issue discussed in here), a quick fix is set the default row to 1 instead of 2 in evaluate/utils/gradio.py/launch_gradio_widget/LINE 115.

  2. Check the user's input data and discard these null inputs (blank text input), which could be done in evaluate/utils/gradio.py/parse_test_cases function.

And I would like to have a PR for fixing this if the the community welcome.😁

@lvwerra
Copy link
Member

lvwerra commented Nov 1, 2022

Hi @Raibows, thanks for pointing this out! Indeed that's an issue that would be great to fix. You suggestion of setting the default to 1 and in addition check for null inputs as a failsafe sounds good to me! Feel free to open a PR and let me know if you need any help! Sorry for not replying on the Hub issue, these sometimes slip :)

lvwerra pushed a commit that referenced this issue Nov 8, 2022
#335)

set default row of gradio webui to 1 and drop empty/blank row
NimaBoscarino pushed a commit to NimaBoscarino/evaluate that referenced this issue Nov 9, 2022
…ty/blank row (huggingface#335)

set default row of gradio webui to 1 and drop empty/blank row
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants