JarvisAI

Last Updated: 17 Oct, 2021

1. What is Jarvis AI?
2. Prerequisite
3. Getting Started- How to use it?
4. How to contribute?
5. Future?

1. What is Jarvis AI-

Jarvis AI is a Python Module which is able to perform task like Chatbot, Assistant etc. It provides base functionality for any assistant application. This JarvisAI is built using Tensorflow, Pytorch, Transformers and other opensource libraries and frameworks. Well, you can contribute on this project to make it more powerful.

This project is crated only for those who is having interest in building Virtual Assistant. Generally it took lots of time to write code from scratch to build Virtual Assistant. So, I have build an Library called "JarvisAI", which gives you easy functionality to build your own Virtual Assistant.

Check more details here: https://github.com/Dipeshpal/Jarvis_AI

Check official website here: https://jarvis-ai-api.herokuapp.com/

API Documentations: https://jarvis-ai-api.herokuapp.com/api_docs/

2. Prerequisite-

• To use it only Python (> 3.6) is required.
• To contribute in project: Python is the only prerequisite for basic scripting, Machine Learning and Deep Learning knowledge will help this model to do task like AI-ML. Read How to contribute section of this page.

3. Getting Started (How to use it)-

Install the latest version-

pip install JarvisAI

It will install all the required package automatically.

If anything not install then you can install requirements manually.
pip install -r requirements.txt
The requirementx.txt can be found here.

https://pypi.org/project/JarvisAI/

Usage and Features-

After installing the library you can import the module-

Example-

1. Basic Usages: https://github.com/Dipeshpal/Jarvis-Assisant/blob/master/scripts/main.py

2. Advance Usages (Wake up using Hand Gesture): https://github.com/Dipeshpal/Jarvis-Assisant/blob/master/scripts/main_advance_usages.py

import JarvisAI 

obj = JarvisAI.JarvisAssistant(sync=True, token='5ec64be7ff718ac25917c198f3d7a4', disable_msg=False, load_chatbot_model=True, high_accuracy_chatbot_model=False,    
chatbot_large=False)  # or JarvisAI.JarvisAssistant(sync=False) 

response = obj.mic_input_ai() # or mic_input() can be also used print(response) ```    

Check this script for more examples- https://github.com/Dipeshpal/Jarvis-Assisant/blob/master/scripts/main.py

Available Methods-

The functionality is cleared by methods name. You can check the code for example. These are the names of available functions you can use after creating JarvisAI's object-

import JarvisAI 

obj = JarvisAI.JarvisAssistant(sync=True, token='5ec64be7ff718ac25917c198f3d7a4', disable_msg=False, load_chatbot_model=True, high_accuracy_chatbot_model=False,    
chatbot_large=False) # or JarvisAI.JarvisAssistant(sync=False) response = 

obj.mic_input_ai()  # mic_input() can be also used ```    

Available Parameters-

1. sync (bool): It is used to sync your JarvisAI setting with server. We don't use this information for any purpose, it's just for better user experience. If you enable this you need to add your token also. You can get your token from JarvisAI's official website.
2. Token (str): It is the token which you can obtain from the JarvisAI's official website. This features help to sync your setting each time run the assistant.
3. disable_msg (bool): It enables/disable the JarvisAI's initialization message.
4. load_chatbot_model (bool): If you want to use our AI based ChatBot model then you need to enable this. Without enabling this you can't use 'chatbot_base' or 'chatbot_large' functions. Disable this if you don't want to use JarvisAI's chatbot feature.
5. high_accuracy_chatbot_model (bool): All the AI's models will use some amount of bandwidth while downloading the models from Transformers Hub. Higher accuracy model will give you high accuracy, and size of these model is also high which required lot's or memory (RAM) while loading for the inference. If you have low memory system or less internet data then set this option to False. If it is false, it will load small model, which is around 1GB - 2GB and it has pretty much good accuracy.
6. chatbot_large (bool): If it is True it means, In case chatbot can't answer, or it recognizes the intent of your query is different from normal conversation then it will use Wikipedia/Internet to resolve your query, and it will analyze (summarize) extracted data from internet before response. You can use 'chatbot_large' with 'high_accuracy_chatbot_model=False' for better experience and lower RAM (internet data). Well, 'chatbot_large=False' only answer you queries based on it's AI model knowledge base, it doesn't use Wikipedia/Internet.

Note: First of all setup initial settings of the project by calling setup function.

res = obj.setup()

1. res = obj.mic_input(lang='en')

2. res = obj.mic_input_ai(record_seconds=5, debug=False)

3. res = obj.website_opener("facebook")

4. res = obj.send_mail(sender_email=None, sender_password=None, receiver_email=None, msg="Hello")

5. res = obj.launch_app("edge")

6. res = obj.weather(city='Mumbai')

7. res = obj.news()

8. res = obj.tell_me(topic='tell me about Taj Mahal')

9. res = obj.tell_me_time()

10. res = obj.tell_me_date()

11. res = obj.shutdown()

12. res = obj.text2speech(text='Hello, how are you?', lang='en')

13. res = obj.datasetcreate(dataset_path='datasets', class_name='Demo',
    haarcascade_path='haarcascade/haarcascade_frontalface_default.xml',
    eyecascade_path='haarcascade/haarcascade_eye.xml', eye_detect=False,
    save_face_only=True, no_of_samples=100,
    width=128, height=128, color_mode=False)

14. res = obj.face_recognition_train(data_dir='datasets', batch_size=32, img_height=128, img_width=128, epochs=10,
    model_path='model', pretrained=None, base_model_trainable=False)

15. res = obj.predict_faces(class_name=None, img_height=128, img_width=128,
    haarcascade_path='haarcascade/haarcascade_frontalface_default.xml',
    eyecascade_path='haarcascade/haarcascade_eye.xml', model_path='model',
    color_mode=False)

16. res = obj.setup()

17. res = obj.show_me_my_images()

18. res= obj.show_google_photos()

19. res = obj.tell_me_joke(language='en', category='neutral')

20. res = obj.hot_word_detect(lang='en')

21. status, response = obj.get_user_data(token="436c57eab581dbb2253cfa77c41574f6") # get your token from https://jarvis-ai-api.herokuapp.com/

22. obj.set_user_data()

23. obj.jarvisai_configure_hand_detector(camera=0, detectionCon=0.7, maxHands=2, cam_display=True, cam_height=480,
    cam_width=888)

24. obj.jarvisai_detect_hands(self, message="")

25. obj.chatbot_base(input_text='how are you') # You must set obj=JarvisAI.JarvisAssistant(load_chatbot_model=True)

26. obj.chatbot_large(input_text='how are you') # You must set obj=JarvisAI.JarvisAssistant(load_chatbot_model=True)

    ---

4. How to contribute?

1. Clone this reop
2. Create virtual environment in python.
3. Install requirements from requirements.txt.
   pip install requirements.txt 4. Now run, __ init__.py
   python __init__.py and understand the working.

Guidelines to add your own scripts / modules-
Lets understand the projects structure first-

• JarvisAI: Root folder containing all the files

• features: All the features supported by JarvisAI. This 'features' folder contains the different modules, you can create your own modules. Example of modules- "weather", "setup". These are the two folders inside 'features' directory.

• __ init__.py: You can code here and add call your functions from here. User will be able to directly access functions listed in this file.

4.2. You can create your own modules in this 'features' directory. Call you function in init file.

4.3. Let's create a module and you can learn by example-

• 4.3.1. We will create a module which will tell us a date and time.

• 4.3.2. Create a folder (module) name- 'date_time' in features directory.

• 4.3.3. Create a python script name- 'date_time.py' in 'date_time' folder.

• 4.3.4. Write this kind of script (you can modify according to your own script). Read comments in script below to understand format-

  'features/date_time/date_time.py' file- Make sure to add docs / comments. Also return value if necessary.

  import datetime      
      def date():      
          """      
          Just return date as string  :return: date if success, False if fail      
          """      
          try:      
              date = datetime.datetime.now().strftime("%b %d %Y")      
          except Exception as e:      
              print(e)      
              date = False      
         return date      
            
            
      def time():      
          """      
         Just return date as string  :return: time if success, False if fail      
         """    
  try:              time = datetime.datetime.now().strftime("%H:%M")      
          except Exception as e:      
              print(e)      
              time = False      
          return time    
  

  *** you can run and test your script by calling from main-***

    if __name__ == '__main__': 
   	 response = date() 
   	 print(response) 
   	 response = time() 
   	 print(response)   
  

• 4.3.4. Integrate your module to Jarvis AI-

  • Open JarvisAI\JarvisAI\__init__.py

  • Format of this py file-

     # import custom features 
     try: 
         import features.date_time.date_time
     except: 
         from JarvisAI.features.date_time import date_time  # integrate your features    
     
     class JarvisAssistant:
         def __init__(self):      
             pass   
              
         def tell_me_date(self):
             return date_time.date()      
                         
         def tell_me_time(self):
             return date_time.time()    
     
     # test your features from main    
     if __name__ == '__main__':         
         obj = JarvisAssistant()           
          res = obj.tell_me_time()      
          print(res)    
          res = obj.tell_me_date()  
          print(res)  
    

4.4. That's it, if you applied all the things as per as guidelines then now just run __ init__.py it should works fine.

4.5. Push the repo, we will test it. If found working and good then it will be added to next PyPi version.

Next time you can import your created function from JarvisAI
Example: import JarvisAI.tell_me_date

5. Future?

Lots of possibilities, GUI, Integrate with GPT-3, support for android, IOT, Home Automation, APIs, as pip package etc.

Todo list-

5.1. More API features

5.2. You tell me

FAQs for Contributors-

1. What I can install?
   Ans: You can install any library you want in your module, make sure it is opensource and compatible with win/linux/mac.

2. Code format?
   Ans: Read the example above. And make sure your code is compatible with win/linux/mac.

3. What should I not change?
   Ans: Existing code.

4. Credits-
   Ans: You will definitely get credit for your contribution.

5. Note-
   Ans: Once you created your module, test it with different environment (windows / linux). Make sure the quality of code because your features will get added to the JarvisAI and publish as PyPi project.

6. Help / Contact?
   Ans. Contact me on any of my social media or Email.

Let's make it big.

What's new?-

1. 17 Oct, 2021-
   - Bug Fixes - Docs Update

2. 19 Sep, 2021-

   Chatbot feature is added. Two methods are newly added (26, 27 check 'Usage and Features').
   It used Transformers based AI models to reponse users general queries.

   Below answers depends on the type of chatbot you have choosen and type of accuracy you have choosen.

• Example (chatbot_small) [Directly answered from chatbot model's knowledge base]-

   user >> How are you?  
   AI >> I am good, how are you?  
  

• Eaxmple (chatbot_large) [Fetched data from internet and answered it after analyzing the gathered data]-

    user >> Who is president of India?  
    AI >> Ram Nath Kovind  
  

• Eaxmple (chatbot_large) [Fetched some of the URL from Internet]-

    user >> who is the captain of team India? 
    AI >> URL1, URL2, URL3  
  

3. Before 19 Sep, 2021-

   Features 1-25 added. Check 'Usage and Features'

Feel free to use my code, don't forget to mention credit.
All the contributors will get credits in this repo.