Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dialogue PL, part 1 #805

Draft
wants to merge 48 commits into
base: master
Choose a base branch
from
Draft

Dialogue PL, part 1 #805

wants to merge 48 commits into from

Conversation

gcampax
Copy link
Contributor

@gcampax gcampax commented Oct 5, 2021

All the way to running the agent and synthesizing dialogues, without regressions, and hooking up the existing transaction model, but while keeping most of the logic in the form of templates.

With most of the interesting logic stubbed out
Or whether it should throw an exception.

This includes all the logic to dispatch to multiple dialogues in
parallel.
Extract the dialogue generation code into a new module, and
wire up the interfaces used by DialogueInterface in a new
PartialDialogue class.

The IO of DialogueInterface is rewritten in terms of templates,
so we can integrate with the existing synthesis. These templates
are added dynamically to the user/agent generator, which are
then expanded normally.
Separate the "dialogue runtime" which tracks conversation histories/APIs/logs
and is independent of ThingTalk/dialogue PL, from the "thingtalk dialogues"
(dialogue PL)
Move the responsabilities of the old DialoguePolicy class to
InferenceSentenceGenerator, which initializes the SentenceGenerator
on-demand based on the skills currently in the context.

Introduce a InferenceTimeDialogue, which is a DialogueHandler
for ThingTalk dialogues but also bridges to DialogueInterface
(dialogue PL).

Refactor non-standard dialogue code (interactive-annotate, simulate-dialogs)
to also use InferenceTimeDialogue as a dialogue handler.

Notifications are still WIP.
Refactor the executor classes so they interact with the user
through the DialogueInterface instead of directly through
DialogueLoop. Special questions asked prior to execution are
now represented as regular states in the state machine,
with dedicated dialogue acts.

Notifications are still messy.
With this commit, we have "hello" and "cancel" commands working,
and the agent can start and stop the dialogue as needed.
Templates written in Genie language will remain as "building blocks",
but the dialogue logic will be written in imperative js code.

In preparation for that, separate lib/templates containing the
single-sentence templates, and lib/transaction-dialogues containing
the logic of the transaction policy.
The module has grown to contain all sorts of unrelated things. Split
into a module containing the ContextInfo class, a module containing
the code to create context phrases, and a module with actual state
manipulation. Later, the state manipulation will stop using the
context info and will move to utils/thingtalk
Except for a few helpers that I need to get rid of. As part of
this change, state manipulation code is now updated to return
target states instead of full states, because that makes more
sense.
And stop "tagging" the context to know what the agent should do

Also fix entity handling so replies have the correct content.
@gcampax gcampax added enhancement New feature or request templates Issues with templates and the state machine dialogue-agent Issues with the dialogue agent at runtime (not state machine related) training Issues with dataset generation, augmentation, training cleanup Clean ups, refactorings, and code improvements that don't fall under other categories labels Oct 5, 2021
State tracking in the dialogue interface can get messed up if
the calls are nested.
To do that, we need to track what the agent actually said in
the last turn, after all the templates are expanded and the
eithers are resolved.
This commit contains all the small fixes needed to get the agent
tests to pass, and pass without too many changes.
@sileix
Copy link
Member

sileix commented Oct 7, 2021

The PR is not finished because synthesis is not done, but I got to the point where the agent works at inference time, and the agent tests pass, so all functionality is there. I think it's a good milestone to start the review. @rayslxu do you want to take a look at this? Maybe we can schedule some time to go over it in person.

yeah, let's do this in person. maybe next week? still have a bit of coughing, probably a bad idea to meet this week

Map notifications to a command with a special dialogue act, and
then dispatch as usual.
It doesn't yet do anything useful, but at least it goes through
We need to include the last turn of the user when the agent terminates
the dialogue, otherwise we don't include any "cancel" turn (which
is all we generate at the moment)
So we can see that we generate some real dialogues
And in the process catch various places where we use context tags
that were removed.
Concatenate them together into a single dynamic template
@gcampax
Copy link
Contributor Author

gcampax commented Nov 24, 2021

Alright, synthesis is not done yet but at least tests pass now. @rayslxu how do you want to review this? Do we want to merge this first PR somewhere?

@sileix
Copy link
Member

sileix commented Nov 27, 2021

Alright, synthesis is not done yet but at least tests pass now. @rayslxu how do you want to review this? Do we want to merge this first PR somewhere?

Ugh... this is a lot to read. I guess I will just skim through it to get a sense of what's going on... (unless you will split the PR into smaller ones)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cleanup Clean ups, refactorings, and code improvements that don't fall under other categories dialogue-agent Issues with the dialogue agent at runtime (not state machine related) enhancement New feature or request templates Issues with templates and the state machine training Issues with dataset generation, augmentation, training
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants