Home

Everything Jarvis

📓   Wiki updated on March 21, 2022

Jarvis uses specific keywords to trigger the respective function which performs a specific task. It is not 100% fool proof but most of the common errors are caught and responses are configured accordingly. If there are any unexpected exceptions please raise an issue.

Restrictions:

Currently, runs only of macOS and Windows 10.

Features:

Jarvis works by automatically detecting the Operating System it is being run on, so all you have to do is to run the right installs script following the instructions. Some key features require API keys, but they can be generated for free.

Offline Communication with Jarvis:

• Using Telegram:

  1. A Telegram bot has to be created using BotFather
  2. Token has to be added to the env var BOT_TOKEN
  3. List of chat ids have to be added to the env var BOT_CHAT_IDS
  4. List of bot users have to be added to the env var BOT_USERS

• Using FastAPI

  1. Hosts the offline communicator on localhost.
  2. Requires a port number added to the env var OFFLINE_PORT. Defaults to 4483
  3. Requires a password for authentication added to the env var OFFLINE_PASS. Defaults to OfflineComm

• Lock, unlock, remote start/stop and set the AC temperature in any JLR (Jaguar Land Rover) vehicles.
• Control any LG Web OS Television.
• Guard the surroundings using face detection and audio recognition. Sends an email and SMS alert in case of an intruder.
• Send a text message to most US based mobile numbers.
• Facial recognition and detection. Click here to read the setup instructions.
• Set up a reminder at a said time and send a message to your phone at the given time.
• Set an alarm with the desired time.
• Wish you on events/festivals and birthday using an env var.
• Increase or decrease master volume of your machine via voice commands.
• Locate, ring and enable lost mode on any of the user's Apple devices.
• Control smart lights in the same IP range.
• Read Outlook/Calendar and inform about meetings in the next 12 hours. macOS only
• Tell public and private IP address of the running device.
• Swap voices on demand.
• Adjust Screen Brightness. macOS only
• Tell system vitals including boot time, fan speed*, CPU and GPU temperature*. macOS only
• Restart the running device and suggest a restart in case of high boot time.
• Scan local ip range to get smart devices connected which acts as IP feeder for smart lights and TV.
• Tell random facts.
• Heads or Tails.
• Can scan and connect/disconnect to nearby bluetooth devices.
• Take notes and saves it to a notes.txt file.
• Tell a joke.
• Tell the number of repositories on your GitHub and clone a particular one.
• Tell the list of google home devices in your IP range.
• Get you the distance between two places or distance from your location to a particular place.
• Create or remove tasks for todo lists.
• Scan your Music folder for .mp3 and plays them on any smart devices in IP range.
• Look for unread emails in your gmail account.
• Get the weather information at any location.
• Get the news update from fox-news.
• Get your investment details using Robinhood API.
• Get facts from Wikipedia using wiki api.
• Open up the webpage you want in your default browser.
• Open any application installed in running machine.
• Get the current date and time.
• Open a Google search for any query.
• Shutdown/restart the running device.
• Respond to most basic conversations.

• If anything apart from the below is requested, Jarvis uses google's Places API to match the requested phrase and suggest options. If the requested phrase doesn't resolve on Places API, it then uses google search engine parser to get results for the phrase. If anything beyond that, Jarvis will launch a google search on your default browser.

Environment Variables:

Default Args

• TITLE - Title which Jarvis should address the user by. Defaults to sir
• NAME - Name which Jarvis should address the user by. Defaults to Vignesh
• LEGACY_KEYWORDS - List of wake words for legacy macOS. Defaults to ['jarvis']

Offline communicator:**

• OFFLINE_PORT - Port number to initiate offline communicator. Defaults to 4483
• OFFLINE_PASS - Secure phrase to authenticate offline requests. Defaults to OfflineComm

Accurate Location: (Defaults to the location based on Public IP which is approximate)**

• ICLOUD_USER - iCloud account username.
• ICLOUD_PASS - iCloud account password.

Features

• GIT_USER - GitHub Username
• GIT_PASS - GitHub Token
• WEATHER_API - API Key from openweathermap
• NEWS_API - API Key from newsapi
• MAPS_API - API Key for maps from google
• GMAIL_USER - Gmail account username to send and read emails.
• GMAIL_PASS - Gmail account password to send and read emails.
• ALT_GMAIL_USER - Alternate gmail account username to send an SMS. (gmail_user can be re-used)
• ALT_GMAIL_PASS - Alternate gmail account password to send an SMS. (gmail_pass can be re-used)

To read emails, access to less secure apps or app specific access is required.

• ROBINHOOD_USER - Robinhood account username.
• ROBINHOOD_PASS - Robinhood account password.
• ROBINHOOD_QR - Robinhood login QR code
• BIRTHDAY - Birth date in the format DD-MM - Example: 24-April
• ICLOUD_RECOVERY - Recovery phone number to activate lost mode on a target device - Example: +11234567890
• PHONE_NUMBER - To send SMS from Jarvis - Example: +11234567890
• ROOT_PASSWORD - System password for your mac to get the system vitals.
• MEETING_APP - Application where the meetings are listed. Defaults to calendar but value can be calendar or outlook
• WOLFRAM_API_KEY - API Key from wolfram alpha.

Args to control TV - Applies only for LGWebOS

• TV_CLIENT_KEY - Client key to control the TV using pywebostv module.
• TV_MAC - Mac address of the TV which will be retrieved by the arp command but just in case.

IP Scanner** - Applies only for Netgear routers

• ROUTER_PASS - Router's admin password to get the available devices using pynetgear module.

  Note that this may be done even without the module by simply using the arp table. Using the module improves accuracy and support between different bandwidths since the devices are already connected to the router.

Car Controls** - Applies only for JLR vehicles using InControl API.

• CAR_EMAIL - Email address to log in to InControl API.
• CAR_PASS - Password to authenticate InControl API.
• CAR_PIN - InControl PIN.

Telegram Bot integration**

• BOT_TOKEN - Telegram BOT token.
• BOT_CHAT_IDS - UserID/ChatID for a particular user.
• BOT_USERS - Usernames that should have access to Jarvis.

Please note that these environment variables are required only if you need the mentioned features. If not Jarvis can work with less features.

Investment Details:

• Login to your Robinhood Web App
• Go to Account -> Settings or click me
• Turn on Two-Factor Authentication
  • Select “Authentication App”
  • Click “Can’t Scan It?”, and copy the 16-character QR code

Facial Recognition:

• For this feature to work, a little bit of setup is required
  • Create a parent directory named train and create more sub directories with the name of a people within the parent directory train
  • Now drop the images directly within the sub directories named after each person
  • Please note that even if the sub directories are not added, the face_recognition script is written in a way that it can learn from unrecognized/new faces by storing them with named sub directories within train
  • The camera id has been made dynamic so that it can choose the camera automatically
• Try to keep the images as light as possible (which means both the display and file size of each image)
  • Reduced image size will not affect the accuracy as the facial recognition script converts each image into a pixel matrix
  • Reduced image size and display area can actually help in faster scanning and recognition
• Make good of the tolerance level in the face recognition script
  • The learning_rate can be switched to match your needs
  • Learning rate generally depends on the clarity of the images stored and the lighting at which it was taken
  • Try to avoid images with very low resolution to maintain an adequate learning rate
  • Lower the learning rate, higher the tolerance level (which means the lower you go on learning rate, the stricter or perhaps more accurate matching it does)
  • Leeman's terms:
    • Increased learning_rate: Exact to No match
    • Decreased learning_rate: Incorrect to close match
    • 0.5 - 0.7 should suffice in most cases though

The API keys can be generated for free once you sign up except for Robinhood, which you know how it works. If not probably you shouldn't be using it.