ID5 Release

Final Release

Feature Summary:

• Ability to create multiple custom dialog tree to be executed upon scene play.
• Ability to attach unique animations to a custom dialog tree nodes to be run upon tree traversal.
• Ability to export/import custom dialog trees into and out of a standard JSON format (Animations are not preserved in export)
• Ability for system to auto-generate inputted responses within the dialog tree into audible responses, and parse verbal user input into dialog tree traversal steps (within a playing scene with a valid dialog tree attached).
• Ability to work on windows computers running Unity Unity 2018.4
• Ability to run within VR, specifically an Oculus Rift.

Known Bugs:

• In very noisy areas, the system can pick up on background noise, limiting its ability traverse dialog trees in a correct manor.
• In instances where the user has a thicker accent, the system can have difficulty parsing the users input. This can be somewhat mediated by allowing accent recognition within the users windows setting (as the speech-to-text portion of the project leverages windows speech-to-text libraries).
• Extremely large words may be difficult for the system to parse and recognize, causing incorrect traversal of a dialog tree.
• When attempting to match numbers within the debug tree, be aware that occasionally the system will parse the users input into an integer value well other times it will parse it into its English form (Example: One billion will be interpreted as "One billion" Where as Forty Two will be interpreted as "42")

Project Documentation

• Project Artifacts: https://drive.google.com/drive/folders/1wMpjKoeklQ2AEDc8_IOFsuS8ap3UHhSD
• Project Readme: https://github.com/UniversityOfSaskatchewanCMPT371/term-project-fall2019-team-1/blob/ID5/README.md

This project was created over the course of the fall semester at the University of Saskatchewan by the CMPT 371 class of 2019 (Team 1).

ID4 Release

Overview:

All artifacts related to ID3 can be found in the following folder: https://drive.google.com/drive/folders/12rPZbJhgbDgCRqo12AL7n_Jc8wtP4JYV?usp=sharing
For all group organization information, please see the Group Policy Document here:
https://drive.google.com/open?id=11sec8O1YRpXrZ_SmS7pG_yDxbLzbfYKHDlcEEOF8y9s
Our Jenkins server (used for Continuous integration) can be found at the following: http://cmpt371g1.usask.ca:8080/

Deliverable milestones:

• Implementation of Import/Export feature
  • Functionality added
  • Unity Test Runner tests implemented
  • Manual tests implemented
  • Bug party tested
• Implementation of more advanced NLP
  • Functionality added (Google or IronPython)
  • Unity Test Runner tests implemented
  • Manual tests implemented
  • Bug party tested
  • Debug menu details updated
• Implementation of animations on each node
  • Functionality added
  • Unity Test Runner tests implemented
  • Manual tests implemented
  • Bug party tested
• Host live demo for Mike of Luxsonic
• Update models in demo scene to be more representative of doctor's office

Project Navigation:

• Run Super State: As depicted in the run state diagram, this state runs when a dialog tree is run in a Unity scene (In game). All code associated to this super state can be found in Assets\PatientSystem\Scripts\.
• Tree Editor Super State: As depicted in the tree state diagram, this state runs when a user is creating and editing a scene (In editor). All code associated to this super state can be found in Assets\Editor\.

ID3

Incremental Deliverable 3

Overview:

All artifacts related to ID3 can be found in the following folder: https://drive.google.com/drive/folders/1Nc7zytRCG4MyBmbDWQX3XOz5LYbfVd95
For all group organization information, please see the Group Policy Document here:
https://drive.google.com/open?id=11sec8O1YRpXrZ_SmS7pG_yDxbLzbfYKHDlcEEOF8y9s
Our Jenkins server (used for Continuous integration) can be found at the following: http://cmpt371g1.usask.ca:8080/

Mini-Milestones:

For this deliverable, our milestones were as follows:

• Complete State Diagrams- Completed
• Refine tree GUI interface - Completed
• Document all testing within the test document - Completed
• Get Contentious Integration up and running and documented - Completed

For the next deliverable, our milestones are as follows:

• Implementation of Import/Export feature
  • Functionality added
  • Unity Test Runner tests implemented
  • Manual tests implemented
  • Bug party tested
• Implementation of more advanced NLP
  • Functionality added (Google or IronPython)
  • Unity Test Runner tests implemented
  • Manual tests implemented
  • Bug party tested
  • Debug menu details updated
• Implementation of animations on each node
  • Functionality added
  • Unity Test Runner tests implemented
  • Manual tests implemented
  • Bug party tested
• Host live demo for Mike of Luxsonic
• Update models in demo scene to be more representative of doctor's office

ID2

Incremental Deliverable 2

Overview:

All artifacts related to ID2 can be found in the following folder: https://drive.google.com/open?id=1R71pIVSP-kJQNpI3SlJSIXdcajwRuf9e
For all group organization information, please see the Group Policy Document here:
https://drive.google.com/open?id=11sec8O1YRpXrZ_SmS7pG_yDxbLzbfYKHDlcEEOF8y9s

Mini-Milestones:

For this deliverable, our milestones were as follows:

• Complete Design Document - Completed
• Implement Debugger UI - Completed
• Implement Interaction Logging System - Completed
• Integration of Speech-to-Text and Text-to-Speech with Dialogue Tree - Completed
• Demo for Shareholder - Incomplete

For the next deliverable, our milestones are as follows:

• Import/Export Feature Implementation
• First Version of Language Processing Engine Implemented
• Migration of Windows DLL Dependencies to Python Libraries
• Implement Slack CI Integration
• CI Implementation for All dev/master/release Commits

Running the Project:

Before attempting to run the project, ensure that if you are using Windows 10 that you go into Settings>Privacy>Speech and enable Online Speech Recognition. If this option is not enabled, the dialogue tree cannot be traversed and the demo will not work.
The project can be run by downloading the release, unzipping the "Test_Build_Unzip_Me.Zip" file, and pressing "VRPatientInteraction.exe"
There are 3 dialogue paths within the "VRPatientInteraction.exe" test scene. The expected responses from the user are given in quotes below, where the NPC prompts are in square brackets:

1. [Hello] > "Does your stomach hurt?" > [Yes it does]
2. [Hello] > "Hello" > [Goodbye] > "Goodbye" > [Thank you for grading the project]
3. [Hello] > "What is the average airspeed velocity of an unladen swallow?" > [African or European?]

Creating a Dialogue Tree:

In Resources/DialogueTree/Tree1, right-click and select Create>ScriptableObject>Dialogue to create a new Dialogue Tree Node

In the Inspector window:

The Prompt field is the text that you want the NPC to vocalize to the Scene User.
The Response field will allow you to specify how many branches come off of the current node, as well as what keywords trigger each of the branches.
The Next field will allow you to drag and drop new nodes into each of the corresponding branch locations.

Deliverable 1

All prototype branches are to be included in this release. This include the following branched:

• prototype
• prototype-GUI
• prototype-Speech
• prototype-vrtk

Each prototype branch has been merged to the major prototype branch, then to the release branch. To allow prototype functionality isolation, all previous branches have been contained into seperate folders. Each prototype folder contains a separate Unity Project (created with Unity Version 2019.2.8f1) demonstrating one of the key technologies to the project.

In addition a major portion of this release also includes the setup and configuration of Jenkins, which is documented in the wiki section of this git.

To run each prototype, install Unity Hub with Unity Version 2019.2.8f1 and do the following:

• prototype-GUI:
• prototype-Speech: Load project and press play. Once the scene is playing, the text to speech can be triggered by pressing space.
• prototype-vrtk:

All artifacts regarding ID1 can be found at Deliverable 1 Artifacts Link.