Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Staging) possible Bing annotation or template issue #418

Open
tonyespinoza1 opened this issue Aug 11, 2021 · 6 comments
Open

(Staging) possible Bing annotation or template issue #418

tonyespinoza1 opened this issue Aug 11, 2021 · 6 comments
Assignees
Labels
P1 We're working on it right now

Comments

@tonyespinoza1
Copy link

both of these cases should work.

image

====
# main/1f27104a-8183-4ae2-8c25-caa7cdee3666
#! timestamp: 2021-08-11T01:24:08.966Z
U: ask bing what is the capital of the united states
UT: $failed;
#! timestamp: 2021-08-11T01:24:08.978Z
A: Sorry, I did not understand that. Can you rephrase it?
====
# main/bbd0b847-55c8-4ee8-b33a-0a3cd7867fbb
#! timestamp: 2021-08-11T01:24:18.369Z
U: ask bing capital of united states
UT: $dialogue @org.thingpedia.dialogue.transaction.execute;
UT: @com.bing.web_search(query="capital of united states");
@gcampax gcampax added the P1 We're working on it right now label Aug 11, 2021
@jmhw0123
Copy link
Contributor

jmhw0123 commented Aug 21, 2021

It's neither an annotation nor a template issue.
This issue was because, on the first try, Bing returned nothing in our supported response type list.
Basically, no value was returned along with its answerType in the rankingResponse object.
In the end, the Bing skill had a clueless response and turned it into an out-of-domain.
On the second try, Bing actually returned something valid. So the answer popped out on Genie.
This issue can be easily reproduced with other API platforms as well.

We can possibly address this issue by doing a limited number of retries internally (so we have a higher chance of having an answer from Bing) or raising some kind of a Bing does not have an answer error in the system.

@jmhw0123 jmhw0123 added the ready-for-qa Ready for testing by original reporter label Sep 3, 2021
@tonyespinoza1
Copy link
Author

The spelling I used in my example was a common misspelling (Capital vs Capitol), which is why I too assumed it was a training issue.

Interestingly, I dropped "Ask Bing" and it did go to Bing correctly. So I will revise my request here to make this work with the correct spelling as it does with the common misspelling.

# main/0443689a-7d76-4b62-a944-27ab60e1264a
#! timestamp: 2021-09-09T01:30:17.462Z
U: what is the capital of the united states
UT: $dialogue @com.bing.webPages;
C: $dialogue @com.bing.webPages;
#! timestamp: 2021-09-09T01:30:17.478Z
A: Using Bing I found List of capitals in the United States - Wikipedia. Washington, D.C. has been the federal capital of the United States since 1800. Each U.S. state has its own capital city, as do many of its insular areas..
AT: @com.bing.reply;
#! timestamp: 2021-09-09T01:30:23.224Z
U: what is the capitol of the united states
UT: $dialogue @org.thingpedia.dialogue.transaction.execute;
UT: @org.thingpedia.builtin.thingengine.builtin.faq_reply(question=enum about_almond_location);

image

@tonyespinoza1 tonyespinoza1 removed the ready-for-qa Ready for testing by original reporter label Sep 9, 2021
@tonyespinoza1
Copy link
Author

@jmhw0123, i've finally got a testing environment up and running with the latest and will go through everything marked Ready for QA. see comments above on this one.

@jmhw0123
Copy link
Contributor

We should test this with "ask bing" in the query and see what happens.

@tonyespinoza1
Copy link
Author

sorry, i should have mentioned that i did try that! here's the proof:
image

====
# main/9f909cdd-4af0-4ce0-92df-e5a81e2683a7
#! timestamp: 2021-09-10T01:37:50.841Z
U: ask bing what is the capitol of the united states
UT: $dialogue @com.bing.webPages;
C: $dialogue @com.bing.webPages;
#! timestamp: 2021-09-10T01:37:50.856Z
A: Using Bing I found United States Capitol - Wikipedia. The United States Capitol, often called The Capitol or the Capitol Building, is the meeting place of the United States Congress and the seat of the legislative branch of the U.S. federal government. It is located on Capitol Hill at the eastern end of the National Mall in Washington, D.C..
AT: @com.bing.reply;
#! timestamp: 2021-09-10T01:38:04.096Z
U: ask bing what is the capital of the united states
UT: $dialogue @com.bing.webPages;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 We're working on it right now
Projects
None yet
Development

No branches or pull requests

3 participants