Home

Everything Jarvis

📓   Wiki updated on August 2, 2021

I created a virtual bot to act like Jarvis. With the current state of things, Jarvis can recognize conversations and respond accordingly. Jarvis uses specific keywords to trigger the respective function which performs a specific task. It is not 100% fool proof but most of the common errors are caught and responses are configured accordingly.

Features:

Jarvis works by automatically detecting the Operating System it is being run, so all you have to do is to run the right installs file following the instructions. Some of the key features require API keys but they can be generated for free. Currently Jarvis uses icloud API to track the current location but will soon be replaced with a more handy option.

• Jarvis is programmed to perform self-recoveries in case of most common exceptions. This includes

  • RecursionError - during repeated function calls.

    Note::By default the recursionlimit() is set to 10 times the default limit.

  • Fatal Python error::Cannot recover from stack overflow - during stuck thread. Jarvis will reboot himself after running 5000 times on sentry_mode() changing the PID which resets the thread count.

Jarvis can:

• Facial recognition and detection. Click here to read the setup instructions
• Wish you on events/festivals and birthday (requires an env var. example: birthday=01-January)
• Guard the surroundings using face detection and audio recognition. Sends an email and SMS alert in case of an intruder
• Increase or decrease master volume of your machine via voice commands
• Locate, ring and enable lost mode on any of the user's apple devices
• Control smart lights in the same IP range
• Read outlook calendar and inform about meetings in the next 12 hours
• Tell public and private IP address of the running device
• Swap voices on demand
• Adjust Screen Brightness
• Tell system vitals including fan speed, CPU and GPU temperature and boot time
• Restart the running device and suggest a restart in case of high boot time
• Mount and Unmount volumes connected
• Trigger personal_cloud with secure username and password (generated during runtime) sent as SMS
• Scan local ip range to get all the devices connected - acts as IP feeder for smart lights and TV
• Tell random facts
• Random heads or tails
• Control your TV
• Can scan and connect/disconnect to near by bluetooth devices
• Send a text message to most US based cell numbers
• Take notes and saves it to a notes.txt file
• Tell a joke
• Tell the number of repositories on your GitHub and clone a particular one
• Print the list of google home devices in your IP range
• Set up a reminder at a said time and sends a message to your phone at the said time
• Show the list of alarms on screen and deletes the one requested
• Set an alarm with the desired time, AM or PM is mandatory
• Get you the distance between two places or distance from your location to a particular place
• Create a database named tasks.db for TO-DO
• Add a new item and category as a task in your todo (Also tells if you it is a duplicate)
• Remove existing tasks from DB
• Get all the items in tasks table
• Delete the entire database
• Scan your Music folder for .mp3 and plays them
• Look for unread emails in your gmail account and speaks the sender and subject line alone
• Initiate chatter bot
• Locate your iPhone and also provides options to ring the phone and enable lost mode
• Get your current location
• Get the weather information for your location (tracks location using your IP)
• Get the news update from fox news (news source can be changed in the news() function)
• Get your investment details using Robinhood api (Refer notes for setup)
• Get your system hard drive
• Get facts from Wikipedia using wiki api
• Open up the webpage you want in your default browser
• Open any application installed in running machine
• Get the current date and time
• Speak the current date, time, weather and latest news
• Open a google search for any query
• Repeat what ever it hears
• Shutdown the running device
• Respond to most basic conversations
• Put himself on sentry mode and wait to hear back from the user to wake up

• If anything apart from the below is requested, Jarvis uses google's Places API to match the requested phrase and suggest options. If the requested phrase doesn't resolve on Places API, it then uses google search engine parser to get results for the phrase. If anything beyond that, Jarvis will launch a google search on your default browser.

Environment Variables:

Default args:

 Used to retrieve the current location.

• icloud_user - iCloud account username.
• icloud_pass - iCloud account password.

Additional args:

• git_user - GitHub Username
• git_pass - GitHub Token
• weather_api - API Key from openweathermap
• news_api - API Key from newsapi
• maps_api - API Key for maps from google
• gmail_user - Gmail account username to send and read emails.
• gmail_pass - Gmail account password to send and read emails.

To read emails, access to less secure apps or app specific access is required.

• robinhood_user - Robinhood account username.
• robinhood_pass - Robinhood account password.
• robinhood_qr - Robinhood login QR code
• birthday - Birth date in the format DD-MM - Example: 24-April
• icloud_recovery - Recovery phone number to activate lost mode on a target device - Example: +11234567890
• phone_number - To send SMS from Jarvis - Example: +11234567890
• think_id - API Key from wolfram alpha.
• root_password - System password for your mac to get the system vitals.

Args for PersonalCloud integration:

• personal_cloud_host - Directory path which has to shared through the internet.

Args to control TV - Applies only for LGWebOS

• tv_client_key - Client key to control the TV using pywebostv module.
• tv_mac - Mac address of the TV which will be retrieved by the arp command but just in case.

Args for ip_scanner - Applies only for Netgear routers:

• router_pass - Router's admin password to get the available devices using pynetgear module.

  Note that this may be done even without the module by simply scanning the whole network. Using the module makes it easier since the devices are already connected to the router.

Args for offline-communicator:

Built for a personalized usage.

• offline_receive_user - Alternate gmail account username to send response from Jarvis as SMS. (gmail_user can be re-used)
• offline_receive_pass - Alternate gmail account password to send response from Jarvis as SMS. (gmail_pass can be re-used)
• offline_phrase - Secure phrase to be used for authentication purpose.
• ngrok_auth_key - Auth token from ngrok - Only used for offline_communicator

Please note that these environment variables are required only if you need the mentioned features. If not Jarvis can work with less features.

Investment Details:

• Login to your Robinhood Web App
• Go to Account -> Settings or click me
• Turn on Two-Factor Authentication
  • Select “Authentication App”
  • Click “Can’t Scan It?”, and copy the 16-character QR code

Facial Recognition:

• For this feature to work, a little bit of setup is required
  • Create a parent directory named train and create more sub directories with the name of a people within the parent directory train
  • Now drop the images directly within the sub directories named after each person
  • Please note that even if the sub directories are not added, the face_recognition script is written in a way that it can learn from unrecognized/new faces by storing them with named sub directories within train
  • The camera id has been made dynamic so that it can choose the camera automatically
• Try to keep the images as light as possible (which means both the display and file size of each image)
  • Reduced image size will not affect the accuracy as the facial recognition script converts each image into a pixel matrix
  • Reduced image size and display area can actually help in faster scanning and recognition
• Make good of the tolerance level in the face recognition script
  • The learning_rate can be switched to match your needs
  • Learning rate generally depends on the clarity of the images stored and the lighting at which it was taken
  • Try to avoid images with very low resolution to maintain an adequate learning rate
  • Lower the learning rate, higher the tolerance level (which means the lower you go on learning rate, the stricter or perhaps more accurate matching it does)
  • Leeman's terms:
    • Increased learning_rate: Exact to No match
    • Decreased learning_rate: Incorrect to close match
    • 0.5 - 0.7 should suffice in most cases though

The API keys can be generated for free once you sign up except for Robinhood, which you know how it works. If not probably you shouldn't be using it.