Jarvis

Last Updated: May 15, 2022

NOTE

Please note that this is a continuation of the JarvisAI library by Dipeshpal

1. What is Jarvis AI?
2. Prerequisite
3. Getting Started- How to use it?
4. How to contribute?
5. Future?

1. What is Jarvis AI-

Jarvis AI is a Python Module which is able to perform task like Chatbot, Assistant etc. It provides a base functionality for any assistant application. This Jarvis is built using Tensorflow, Pytorch, Transformers and other open source libraries and frameworks. Feel free to contribute on this project to make it more powerful.

This project is crated only for those who is having interest in building a virtual assistant. Before, it took a lot of time to write code from scratch to build a virtual assistant with only basic features. So, I made a library called Jarvis, which gives even beginners the opportunity to build a virtual assistant.

Check more details here: https://github.com/Dipeshpal/Jarvis_AI

Check official website here: https://jarvis-ai-api.herokuapp.com/

API Documentations: https://jarvis-ai-api.herokuapp.com/api_docs/

2. Prerequisite-

• To use it only Python (> 3.6) is required.
• To contribute in project: Python is the only prerequisite for basic scripting, Machine Learning and Deep Learning knowledge will help this model to do task like AI-ML. Read How to contribute section of this page.

3. Getting Started (How to use it)-

Install the latest version-

pip install jarvis-akul2010

It will install the required packages automatically.

If nothing installs then you can install requirements manually.
pip install -r requirements.txt
The requirementx.txt can be found here.

https://pypi.org/project/jarvis-akul2010/1.0.0

Usage and Features-

After installing the library you can import the module-

Example-

1. Basic Usages: https://github.com/Dipeshpal/Jarvis-Assisant/blob/master/scripts/main.py

2. Advance Usages (Wake up using Hand Gesture): https://github.com/Dipeshpal/Jarvis-Assisant/blob/master/scripts/main_advance_usages.py

import Jarvis 

obj = Jarvis.JarvisAssistant(sync=True, token='5ec64be7ff718ac25917c198f3d7a4', disable_msg=False, load_chatbot_model=True, high_accuracy_chatbot_model=False,    
chatbot_large=False)  # or Jarvis.JarvisAssistant(sync=False) 

response = obj.mic_input_ai() # or mic_input() can be also used print(response) ```    

Check this script for more examples- https://github.com/Dipeshpal/Jarvis-Assisant/blob/master/scripts/main.py

Available Methods-

The functionality is cleared by methods name. You can check the code for example. These are the names of available functions you can use after creating Jarvis's object-

import Jarvis 

obj = Jarvis.JarvisAssistant(sync=True, token='5ec64be7ff718ac25917c198f3d7a4', disable_msg=False, load_chatbot_model=True, high_accuracy_chatbot_model=False,    
chatbot_large=False) # or Jarvis.JarvisAssistant(sync=False) response = 

obj.mic_input_ai()  # mic_input() can be also used ```    

Available Parameters-

1. sync (bool): It is used to sync your Jarvis setting with server. We don't use this information for any purpose, it's just for better user experience. If you enable this you need to add your token also. You can get your token from Jarvis's official website.

2. Token (str): It is the token which you can obtain from the Jarvis's official website. This features help to sync your setting each time run the assistant.

3. disable_msg (bool): It enables/disable the Jarvis's initialization message.

4. load_chatbot_model (bool): If you want to use our AI based ChatBot model then you need to enable this. Without enabling this you can't use 'chatbot_base' or 'chatbot_large' functions. Disable this if you don't want to use Jarvis's chatbot feature.

5. high_accuracy_chatbot_model (bool): All the AI's models will use some amount of bandwidth while downloading the models from Transformers Hub. Higher accuracy model will give you high accuracy, and size of these model is also high which required lot's or memory (RAM) while loading for the inference. If you have low memory system or less internet data then set this option to False. If it is false, it will load small model, which is around 1GB - 2GB and it has pretty much good accuracy.

6. chatbot_large (bool): If it is True it means, In case chatbot can't answer, or it recognizes the intent of your query is different from normal conversation then it will use Wikipedia/Internet to resolve your query, and it will analyze (summarize) extracted data from internet before response. You can use 'chatbot_large' with 'high_accuracy_chatbot_model=False' for better experience and lower RAM (internet data). Well, 'chatbot_large=False' only answer you queries based on it's AI model knowledge base, it doesn't use Wikipedia/Internet.

7. backend_tts_api (str): Male Voice Added (if your system support pyttsx3 module and your system have multiple voices inbuilt)

   You can try different voices. This is one time setup. You can reset your voice by deleting 'configs/Jarvis-Voice.txt' file in your working directory. This file will be created during first run of the program only if you are using pyttsx3.

   Usages-

   import Jarvis
   
   # backend_tts_api='pyttsx3' for different voices options
    # backend_tts_api='gtts' for female voice by google text to speech library
   obj = Jarvis.JarvisAssistant(sync=True, token='5ec64be7ff718ac25917c198f3d7a4', disable_msg=False,
                              load_chatbot_model=False, high_accuracy_chatbot_model=False,
                              chatbot_large=False, backend_tts_api='pyttsx3')
   

   What is 'configs/Jarvis-Voice.txt' file contains?

   • Voice information in plain text (do not modify this file manually)

Note: First of all setup initial settings of the project by calling setup function.

res = obj.setup()

1. res = obj.mic_input(lang='en')

2. res = obj.mic_input_ai(record_seconds=5, debug=False)

3. res = obj.website_opener("facebook")

4. res = obj.send_mail(sender_email=None, sender_password=None, receiver_email=None, msg="Hello")

5. res = obj.launch_app("edge")

6. res = obj.weather(city='Mumbai')

7. res = obj.news()

8. res = obj.tell_me(topic='tell me about Taj Mahal')

9. res = obj.tell_me_time()

10. res = obj.tell_me_date()

11. res = obj.shutdown()

12. res = obj.text2speech(text='Hello, how are you?', lang='en')

13. res = obj.datasetcreate(dataset_path='datasets', class_name='Demo',
    haarcascade_path='haarcascade/haarcascade_frontalface_default.xml',
    eyecascade_path='haarcascade/haarcascade_eye.xml', eye_detect=False,
    save_face_only=True, no_of_samples=100,
    width=128, height=128, color_mode=False)

14. res = obj.face_recognition_train(data_dir='datasets', batch_size=32, img_height=128, img_width=128, epochs=10,
    model_path='model', pretrained=None, base_model_trainable=False)

15. res = obj.predict_faces(class_name=None, img_height=128, img_width=128,
    haarcascade_path='haarcascade/haarcascade_frontalface_default.xml',
    eyecascade_path='haarcascade/haarcascade_eye.xml', model_path='model',
    color_mode=False)

16. res = obj.setup()

17. res = obj.show_me_my_images()

18. res= obj.show_google_photos()

19. res = obj.tell_me_joke(language='en', category='neutral')

20. res = obj.hot_word_detect(lang='en')

21. status, response = obj.get_user_data(token="436c57eab581dbb2253cfa77c41574f6") # get your token from https://jarvis-ai-api.herokuapp.com/

22. obj.set_user_data()

23. obj.Jarvis_configure_hand_detector(camera=0, detectionCon=0.7, maxHands=2, cam_display=True, cam_height=480,
    cam_width=888)

24. obj.Jarvis_detect_hands(self, message="")

25. obj.chatbot_base(input_text='how are you') # You must set obj=Jarvis.JarvisAssistant(load_chatbot_model=True)

26. obj.chatbot_large(input_text='how are you') # You must set obj=Jarvis.JarvisAssistant(load_chatbot_model=True)

27. obj.create_new_list('add milk in my shopping list')

28. obj.delete_particular_list('delete my shopping list')

29. obj.show_me_my_list()

30. obj.show_me_some_tech_news() # It will show tech news in your browser

31. obj.show_me_some_tech_videos() # It will show tech videos in your browser

32. obj = Jarvis.JarvisAssistant(sync=True, token='5ec64be7ff718ac25917c198f3d7a4', disable_msg=False, load_chatbot_model=False, high_accuracy_chatbot_model=False, chatbot_large=False, backend_tts_api='pyttsx3') # you must set backend_tts_api='pyttsx3' for different voices options (Read 'What's new' of '20 Dec, 2021' update section of this page for more details.)

    ---

4. How to contribute?

1. Clone this reop
2. Create virtual environment in python.
3. Install requirements from requirements.txt.
   pip install requirements.txt 4. Now run, __ init__.py
   python __init__.py and understand the working.

Guidelines to add your own scripts / modules-
Lets understand the projects structure first-

• Jarvis: Root folder containing all the files

• features: All the features supported by Jarvis. This 'features' folder contains the different modules, you can create your own modules. Example of modules- "weather", "setup". These are the two folders inside 'features' directory.

• __ init__.py: You can code here and add call your functions from here. User will be able to directly access functions listed in this file.

4.2. You can create your own modules in this 'features' directory. Call you function in init file.

4.3. Let's create a module and you can learn by example-

• 4.3.1. We will create a module which will tell us a date and time.

• 4.3.2. Create a folder (module) name- 'date_time' in features directory.

• 4.3.3. Create a python script name- 'date_time.py' in 'date_time' folder.

• 4.3.4. Write this kind of script (you can modify according to your own script). Read comments in script below to understand format-

  'features/date_time/date_time.py' file- Make sure to add docs / comments. Also return value if necessary.

  import datetime      
      def date():      
          """      
          Just return date as string  :return: date if success, False if fail      
          """      
          try:      
              date = datetime.datetime.now().strftime("%b %d %Y")      
          except Exception as e:      
              print(e)      
              date = False      
         return date      
            
            
      def time():      
          """      
         Just return date as string  :return: time if success, False if fail      
         """    
  try:              time = datetime.datetime.now().strftime("%H:%M")      
          except Exception as e:      
              print(e)      
              time = False      
          return time    
  

  *** you can run and test your script by calling from main-***

    if __name__ == '__main__': 
   	 response = date() 
   	 print(response) 
   	 response = time() 
   	 print(response)   
  

• 4.3.4. Integrate your module to Jarvis AI-

  • Open Jarvis\Jarvis\__init__.py

  • Format of this py file-

     # import custom features 
     try: 
         import features.date_time.date_time
     except: 
         from Jarvis.features.date_time import date_time  # integrate your features    
     
     class JarvisAssistant:
         def __init__(self):      
             pass   
              
         def tell_me_date(self):
             return date_time.date()      
                         
         def tell_me_time(self):
             return date_time.time()    
     
     # test your features from main    
     if __name__ == '__main__':         
         obj = JarvisAssistant()           
          res = obj.tell_me_time()      
          print(res)    
          res = obj.tell_me_date()  
          print(res)  
    

4.4. That's it, if you applied all the things as per as guidelines then now just run __ init__.py it should works fine.

4.5. Push the repo, we will test it. If found working and good then it will be added to next PyPi version.

Next time you can import your created function from Jarvis
Example: import Jarvis.tell_me_date

5. Future?

I will try to add anything I can, but feel free to give any suggestion, and if they are free, I will try.

Todo list-

5.1. Make package size smaller but with same efficiency. 5.2. Make separate website for tokens.

FAQs for Contributors-

1. What I can install?
   Ans: You can install any library you want in your module, make sure it is opensource and compatible with win/linux/mac.

2. Code format?
   Ans: Read the example above. And make sure your code is compatible with win/linux/mac.

3. What should I not change?
   Ans: Existing code. If you want to change it, make a pull request and I will look into it.

4. Credits-
   Ans: You will definitely get credit for your contribution.

5. Note-
   Ans: Once you created your module, test it with different environment (windows / linux). Make sure the quality of code is good because your features will get added to the Jarvis program and publish as PyPi project.

6. Help / Contact?
   Ans. Contact me on my email.

Let's make it big.

What's new?-

1. May 15, 2022- Initial release. Lots of features, uses Dipeshpal's jarvis-ai website for tokens.

Feel free to use my code, don't forget to mention credit.
All the contributors will get credits in this repo.