Development of DF Designer

[oserikov]

difficulty: medium
duration: 350 hours
mentor: Maria, @mablin7 , @kudep
requirements:

1. Python
2. understanding of ASTs
3. experience in optimizing python code
4. understanding of graphs
5. TypeScript
6. React
7. CSS
8. bonus if familiar with VSCode extension API
   useful links:

tl;dr: DF Designer is a tool for developing scenario-driven skills for multi-skill AI assistants and simple bots in a visual environment.
DF Designer includes basic skill creation capabilities and advanced functionality such as recommender systems in scenario-driven skills, statistical information, etc.
The GSoC student will have to develop the functionality of the tool for working with the recommender system for bot’s responses and the functionality for working with statistics.

Idea Description:

Dialog Flow Framework

Dialog Flow Framework (DF Engine, DF Addons, etc.). DF Engine is a finite-state machine dialog engine that enables development of the scenario-driven skills in DeepPavlov Dream (and also in standalone fashion, more on that later).
Dialog Flow Framework provides you with a native Python-based DSL that you can use to define the logic of your skill:

You can learn more about Dialog Flow Framework and its accompanying tech in this video. Denis Kuznetsov, creator of Dialog Flow Framework , presented it at AI Journey 2021 in the end of the last year.

DF Designer

DF Designer is a tool for developing scenario-driven skills for multi-skill AI assistants and simple bots in a visual environment.
DF Designer includes basic skill creation capabilities and advanced functionality such as recommender systems that enables developers to predict next user steps in the current dialog state., statistical information, etc.

Current state

In addition to manual coding of DFF skills you can also use the alpha version of one of the Dev Tools that accompany DeepPavlov Dream, DD-IDDE aka DF Designer. DF Designer makes it much easier to visually construct your DFF skills from scratch within the comfort of the trusty VS Code.

DD-IDDE, the original name of the DF Designer, stands for Discourse-Driven Integrated Dialogue Development Environment. Initially developed as part of our mid-term applied research towards Strategic Dialog Planning, DD-IDDE was proudly presented at CODI 2021 workshop at the top AI conference EMNLP 2021 earlier this year.
One of the coolest experimental features of DD-IDDE (hence it was announced at EMNLP 2021 first) is it’s unique recommendation system that enables developers to predict next user steps in the current dialog state. You can learn more about this feature in the paper about it published in ACL Anthology here.

You can download it from the VS Code Marketplace here.
To use DD-IDDE aka DF Designer to build your own DFF Skills, use it to open scenario/main.py of your DFF Skill.

Here’s a short video introduction of DD-IDDE aka DF Designer made at AI Journey 2021.
Product state by the beginning of the internship
We are currently transitioning to a custom React-based flow editing library. We are also working on expanding the functionality of the product. By the beginning of the internship, the main functionality will be ready. The tool will already allow you to develop a scenario-driven skill.
Conditions for bot responses depending on the user's utterances

Bot responses functionality

Tasks for the Internship

During the internship, it is necessary to implement 2 blocks of tasks:

Modification of canvas in Stats Mode

One of the advanced features of the tool is the use of the collected statistical data to improve the DF skill. For example, based on the collected data, the system can determine that some part of the dialog flow is built inefficiently and suggest improvements. During the internship, you’ll be asked to add a new stats layer to the canvas. This layer would be used to provide recommendations of the aforementioned improvements. We have a preliminary representation of what it will look like. And by the beginning of the internship, it will be detailed.

Graph views depending on the layers (flow, specific nodes and recommendations)

Recommendation: The recommendation system is one of the benefits of developing a skill through DF Designer. This system helps to develop scenario-driven skills quickly and efficiently. We have developed a speech-functions-based recommender system and a dialog-act-based recommender system in the alpha version of the tool (DD-IDDE). During the internship, it is necessary to implement the view of the dialog graph in the recommendation mode. The layouts can be seen below.

global/local transitions: One of the characteristics of the skill is a type of transition (global and local). It is necessary to implement its view on the graph.

Statistics: Display of statistical data.

separating flows: One skill can include several dialogue flows. It is necessary to implement the view of the dialog graph for different flows.
custom features: tags - functions in node eg. This functionality will allow users to work with advanced features when developing skills.

Coding Challenge

Write a simple webapp with a python backend, which parses these .py files and renders the graphs inside them. (using ast, NOT REGEX, must draw graph with typescript)