---

Interactive learning platform with dynamic visual analyzer and chatbot engagement.

---

Team Members | Background/Problem Statement and Publication | Our Solution | Demo | Technology Integrated Website | Technologies Used | Technical Description | System Architecture / Roadmap

---

Team Members

• "Kar Haw Chang" Watson man (Louisiana, USA)
• "Rohana Sosa" Holistic Web Leader (New York, USA)
• "Sanjay Kumar" Data Scientist Dude (Karachi, Pakistan)
• "Shikhar Kwatra" Data Scientist Lead (California, USA)
• "Yuki Ishibashi" Creativity Ambassador (Tokyo, Japan)

Background/Problem Statement and Publication

Technology is changing people. How people learn, how people perform their job, and even how people interact. Quite often, people tend to struggle with understanding what they are studying. One of the reasons for this struggle is that they have a hard time concentrating on our current progress of learning as a student/learner. Another reason is being us, human being, are interactive, and have a short-attention span. Hence, we will need some sort of other ways for us to repeat our process of learning.

We also submitted and received an IBM Publication found here: https://priorart.ip.com/IPCOM/000262638

IPCOM000262638D Dynamic Chatbot Engagement in Virtual Classrooms Publication Date 2020-Jun-17 Publication Venue IP.com PAD

Our Solution

We propose a method and system for dynamic chat-bot integration and initiation with plurality of students in a virtual setting based on configuration threshold inclusive of attention/level of engagement.

Training students and helping educators in a collaborative way with this Dynamic Watson Chatbot will lead to developing healthy habitual patterns in the mind that can continuously evolve in a positive growth correlation of gaining new skills and natural acquisition of knowledge. 

The value of this invention includes easily training students to observe their own thinking with the assistance of the chatbot and/or teacher. This is important to build the confidence of students learning new knowledge and it enables the assessment of learning retention for the ultimate objective of natural knowledge acquisition, meaning that the student’s new knowledge is now ‘second nature’ to them and they can independently utilize/remember knowledge. Based on Quantum Physics and Neuroscience, repeated thinking is a thought process that has tremendous impact because it is directly aligned with forming habits. If students utilize our Dynamic Watson Chatbot to divert that thought process in a positive growth direction, students will be able to achieve a more natural acquisition of valuable educational knowledge that is in direct correlation with how they uniquely learn as a person. In addition, students, people returning to the workforce, any learner, etc. can be trained in such a way that is also aligned in parallel with the instructor’s objectives in educating the person (e.g. potential future manager is training a new employee to meet a certain business objective). 

In Neural Plasticity, the neurons in the brain are activated and when those neurons are repeatedly activated in a particular pattern, a neural pathway develops leading to that thought being more easily retrieved again. Knowing this, our Dynamic Watson Chatbot can help students form educational habits based on their thinking pattens, which are mental material objects of data that the chatbot is collecting, annotating, and extracting to generate a progress report of the learner. For example, if the student asks the same question for several times, Dynamic Watson Chatbot learns that the student doesn't understand the term and it will give more detailed explanation. The data that the chatbot has obtained is a tree representation neural network just like thoughts are constructed as tree shaped structures in the human brain. As the chatbot determines the areas of improvement for the student, the neural network branches change shape by using an Intelligent Spaced Repetition Algorithm with GANs as an evidence based learning technique. The chatbot keeps track of the student’s recall of the skills being learned in spaced intervals of time so more information is encoded into long term memory.  

Demo

Watch our YouTube demo!! >>> https://www.youtube.com/watch?v=5_7lyhYxqb4&feature=youtu.be

Technology Integrated Website for Diverse Influencers Non-profit Org

Visit the website!! >>> https://diverseinfluencers.skillsnetwork.site/ The website is set to public so anyone can see it for now. If you'd like to sign in, please create an account.

A method and system for dynamic chat-bot and mood analysis are integrated.

Below is the the WebEx Video Recording of the meeting presentation with Diverse Influencers (DI) (whom we received their consent to record the WebEx) DI said they will continue to use the platform since they loved it very much.

Meeting with Diverse Influencers (DI) and the IBM CTO / Director of Developer Ecosystems | Strategic Partnerships and here is the recording : https://ibm.webex.com/webappng/sites/ibm/recording/play/62a3693a73f14fba9486df37ad8b1857

• WebEx Recording Password: HjjP3Ezr

Technologies Used

• IBM Cloud
• IBM Watson Assistant
• IBM Cloud Functions
• IBM Skillsnetwork Platform
• IBM Watson Natural Language Understanding (NLU)
• IBM Watston Knowledge Studio
• IBM Watson Discovery
• Mood Classifier: TensorFlow, OpenCV, Keras and Twilio
• GANs: TensorFlow, Keras, Numpy and Pandas

Technical Description

How to Incorporate Dynamic Watson Chatbot and Mood Analysis to your website

1. Setup your website (Skillsnetwork provided by IBM is recommended) IBM Developer Request Form

2. Add Dynamic Watson Chatbot

   1. Train your Watson Assistant.
      • Create your Watson Assistant service instance.
      • Create a dialog skill.
      • Add intents.
      • Build a dialog.
   2. Integrate your Waston Assistant
      • Copy and paste Watson Assistant's API and ID to your website's configuration.
      • Note: At the time of testing, we utilized the Watson Assistant Free Trial PLUS Plan Search Skill for 30 Days, which expires July 05th, 2020.

3. Add Mood Analysis

• Incorporate classifier algorithm to determine and analyze the reactions of students watching lectures to determine whether or not students are confused about a topic they need help with or to help determine students' comprehension. To view the actual demo on the IBM Skills Network, go to the Blog section of the website and select the link Mood Analysis Demo - Click Here

4. Add Watson NLU

• Annotate dialogs between students and chatbot so Watson NLU can interpret students' strengths and areas of improvement during learning. Please see readme in the Watson NLU folder to watch the GIF demos.

5. Add Generative Adversarial Networks - GANs

• Train a machine vision model to autonmously generate illustrations based on educational matertials and class teachings. With the combination of GANs and Watson NLU, the AI can learn about the student's personality, interests, and learning strengths/weaknesses so that it can create an evolving step-by-step learning plan in real-time as the student grows in abilities, knowledge and self-confidence. The GANs and Watson NLU would be represented by an animated AI personality to make learning creative, engaging and interactive.

System Architecture / Roadmap

Maturity of our solution today:

Figure 1. System Architecture. This figure illustrates how the system is integrate with each other services. Figure 2. System Architecture. This figure illustrates how the student's query is going through the system.

Link to Mood Classifier code on our team’s GitHub: https://github.ibm.com/skwatra/DyCE-Watson-Virtual-Classroom/blob/master/src/Mood_classifier/Facial_mood_classifier_Education.ipynb

Link to GANs code that draws images to model concept to illustrate knowledge being learned: https://github.ibm.com/skwatra/DyCE-Watson-Virtual-Classroom/tree/master/src/DyCE_GANs

Plan to improve our solution in the future:

1. An animated personality

To enhange the engagement of students by adding an element of entertainment, we are planning to animate our chatbot. We expect him to conduct virtual experiments to teach topics!!

2. Chatbot interaction with mood analysis

As students oftentimes have hard times studying and keep the motivation to sit at a desk, our chatbot integrated with mood analysis will detect student's emotion and based on it, this chatbot is changes his style to interact with students (we could say he has sympathy!). When a student is tired, he will propose him or her, for example, "let's take a break for 5 minutes and review what we have learned today together.". Or when a student is motivated, he will encourage him or her with gesture!

We plan on integrating the Mood Classifier with the Watson Assistant so that the Mood Classifier is triggered when opening the Watson Assistant with the user’s permission. With the combination of GANs and Watson NLU, the AI can learn about the student's personality, interests, and learning strengths/weaknesses so that it can create an evolving learning plan in real-time as the student grows in abilities, knowledge and self-confidence.

Figure 3. Intelligent Spaced Repetition. This figure illustrates the rehearsal process of learning new knowledge and skills over a 64-day time frame in order to strengthen long term memory.

Figure 4. System Architecture. This figure illustrates the process that start with School Organization or Teacher and which entities they can access.

Figure 5. System Architecture. This figure illustrates what is being store in IBM Cloud Object Storage.