Skip to content

Home

Jump to bottom
Vignesh Rao edited this page Apr 3, 2022 · 78 revisions

x# Everything Jarvis

📓   Wiki updated on April 3, 2022

Jarvis uses specific keywords to trigger the respective function which performs a specific task. It is not 100% fool proof but most of the common errors are caught and responses are configured accordingly. If there are any unexpected exceptions please raise an issue.

Restrictions:

Currently, Jarvis can run only of macOS and Windows.
Tested on MacOS High Sierra, Mojave, Catalina, Big Sur, and Monterey and Windows 10

Features:

Jarvis works by automatically detecting the Operating System it is being run on. To set up requirements, run the install.sh script following the instructions.
Some key features require API keys, but they can be generated for free. None of the features in Jarvis require subscriptions.

Offline Communication with Jarvis:

  • Using Telegram:

    1. A Telegram bot has to be created using BotFather
    2. Token has to be added to the env var BOT_TOKEN
    3. List of chat ids have to be added to the env var BOT_CHAT_IDS
    4. List of bot usernames have to be added to the env var BOT_USERS

  • Using FastAPI:

    1. Hosts the offline communicator on localhost.
    2. Requires a port number added to the env var OFFLINE_PORT. Defaults to 4483
    3. Requires a password for authentication added to the env var OFFLINE_PASS. Defaults to OfflineComm
  • Lock, unlock, remote start/stop and set the AC temperature in any JLR (Jaguar Land Rover) vehicles.
  • Control any LG Web OS Television.
  • Guard the surroundings using face detection and audio recognition. Sends an email and SMS alert in case of an intruder.
  • Send a text message to most US based mobile numbers.
  • Facial recognition and detection. Click here to read the setup instructions.
  • Set up a reminder at a said time and send a message to your phone at the given time.
  • Set an alarm with the desired time.
  • Wish you on events/festivals and birthday using an env var.
  • Increase or decrease master volume of your machine via voice commands.
  • Locate, ring and enable lost mode on any of the user's Apple devices.
  • Control smart lights in the same IP range.
  • Read Outlook/Calendar and inform about meetings in the next 12 hours. macOS only
  • Tell public and private IP address of the running device.
  • Swap voices on demand.
  • Adjust Screen Brightness. macOS only
  • Tell system vitals including boot time, fan speed*, CPU and GPU temperature*. macOS only
  • Restart the running device and suggest a restart in case of high boot time.
  • Scan local ip range to get smart devices connected which acts as IP feeder for smart lights and TV.
  • Tell random facts.
  • Heads or Tails.
  • Can scan and connect/disconnect to nearby bluetooth devices.
  • Take notes and saves it to a notes.txt file.
  • Tell a joke.
  • Tell the number of repositories on your GitHub and clone a particular one.
  • Tell the list of google home devices in your IP range.
  • Get you the distance between two places or distance from your location to a particular place.
  • Create or remove tasks for todo lists.
  • Scan your Music folder for .mp3 and plays them on any smart devices in IP range.
  • Look for unread emails in your gmail account.
  • Get the weather information at any location.
  • Get the news update from fox-news.
  • Get your investment details using Robinhood API.
  • Get facts from Wikipedia using wiki api.
  • Open up the webpage you want in your default browser.
  • Open any application installed in running machine.
  • Get the current date and time.
  • Open a Google search for any query.
  • Shutdown/restart the running device.
  • Respond to most basic conversations.
  • If anything apart from the below is requested, Jarvis uses google's Places API to match the requested phrase and suggest options. If the requested phrase doesn't resolve on Places API, it then uses google search engine parser to get results for the phrase. If anything beyond that, Jarvis will launch a google search on your default browser.

Environment Variables:

Default Args
  • TITLE - Title which Jarvis should address the user by. Defaults to sir
  • NAME - Name which Jarvis should address the user by. Defaults to the creator: Vignesh
  • LEGACY_KEYWORDS - List of wake words for macOS High Sierra or older. Defaults to ['jarvis']
Offline communicator:
  • OFFLINE_PORT - Port number to initiate offline communicator. Defaults to 4483
  • OFFLINE_PASS - Secure phrase to authenticate offline requests. Defaults to OfflineComm
Features
  • GIT_USER - GitHub Username
  • GIT_PASS - GitHub Token
  • WEATHER_API - API Key from openweathermap
  • NEWS_API - API Key from newsapi
  • MAPS_API - API Key for maps from google
  • GMAIL_USER - Gmail account username to send and read emails.
  • GMAIL_PASS - Gmail account password to send and read emails.
  • ALT_GMAIL_USER - Alternate gmail account username to send an SMS. (GMAIL_USER can be re-used)
  • ALT_GMAIL_PASS - Alternate gmail account password to send an SMS. (GMAIL_PASS can be re-used)
  • ROBINHOOD_USER - Robinhood account username.
  • ROBINHOOD_PASS - Robinhood account password.
  • ROBINHOOD_QR - Robinhood login QR code
  • BIRTHDAY - Birth date in the format DD-MM - Example: 24-April
  • ICLOUD_USER - iCloud account username.
  • ICLOUD_PASS - iCloud account password.
  • ICLOUD_RECOVERY - Recovery phone number to activate lost mode on a target device - Example: +11234567890
  • PHONE_NUMBER - To send SMS from Jarvis - Example: +11234567890
  • ROOT_PASSWORD - System password for your mac to get the system vitals.
  • WOLFRAM_API_KEY - API Key from wolfram alpha.
  • ICS_URL - Shared calendar URL to get meetings information from. Should end with .ics
  • EVENT_APP - To read events from outlook or calendar. Defaults to calendar
Background scans [Defaults to 1 hour]
  • SYNC_NETGEAR - Interval in seconds to scan for smart devices using Netgear module.
  • SYNC_MEETINGS - Interval in seconds to generate meetings information using ics URL.
  • SYNC_EVENTS - Interval in seconds to generate events information using calendar or outlook application.
VPNServer integration
  • VPN_USERNAME - Username to create vpn-server. Defaults to profile username or openvpn
  • VPN_PASSWORD - Password to authenticate vpn-server. Defaults to profile password or aws_vpn_2021
Args to control TV - Applies only for LGWebOS
  • TV_CLIENT_KEY - Client key to control the TV using pywebostv module.
IP Scanner - Applies only for Netgear routers
  • ROUTER_PASS - Router's admin password to get the available devices using pynetgear module.

    Note that this may be done even without the module by simply using the arp table. Using the module improves accuracy and support between different bandwidths since the devices are already connected to the router.

Car Controls - Applies only for JLR vehicles using InControl API.
  • CAR_EMAIL - Email address to log in to InControl API.
  • CAR_PASS - Password to authenticate InControl API.
  • CAR_PIN - InControl PIN.

Smart Devices

A source file smart_devices.yaml is used to store smart devices' IPs. Jarvis supports MagicHome lights and LGWebOS TVs.

Setup Instructions

  • For the Netgear module to scan for smart devices, a hostnames.yaml file is required to be placed in fileio directory.

  • The source file should be as following:

bedroom:
  - 'HOSTNAME1'
  - 'HOSTNAME2'
hallway:
  - 'HOSTNAME1'
  - 'HOSTNAME2'
  - 'HOSTNAME3'
kitchen:
  - 'HOSTNAME1'
  - 'HOSTNAME2'
  - 'HOSTNAME3'
tv: 'LGWEBOSTV'

Note: Jarvis currently supports only one hostname for TV but multiple for lights.

By using smart_devices.yaml, the Netgear module can be avoided at the cost of manually updating the source file in case of IP changes.

  • If the source file is hand-created, it should be as following:
bedroom_ip:
- 'LOCAL_IP_ADDRESS'
hallway_ip:
- 'LOCAL_IP_ADDRESS'
kitchen_ip:
- 'LOCAL_IP_ADDRESS'
tv_ip: 'LOCAL_IP_ADDRESS'
tv_mac: 'TV_MAC_ADDRESS'

Automation Setup [Optional]

Executes offline compatible tasks at pre-defined times without any user interaction. Uses an automation.yaml file as source which should be stored within the directory fileio

Setup Instructions

The YAML file should be a dictionary within a dictionary that looks like the below.

OPTIONAL: The key, day can be a list of days, or a str of a specific day or simply a str saying weekday or weekend when the particular automation should be executed.

Not having the key day will run the automation daily.

6:00 AM:
  day: weekday  # Runs only between Monday and Friday
  task: set my bedroom lights to 50%
6:30 AM:
  day:  # Runs only on Monday, Wednesday and Friday
  - Monday
  - wednesday
  - FRIDAY
  task: set my bedroom lights to 100%
8:00 AM:  # Runs only on Saturday and Sunday
  day: weekend
  task: set my bedroom lights to 100%
9:00 PM:  # Runs daily
  task: set my bedroom lights to 5%

Please note that these environment variables are required only if you need the mentioned features. If not Jarvis can work with less features.

Investment Details:

  • Login to your Robinhood Web App
  • Go to Account -> Settings or click me
  • Turn on Two-Factor Authentication
    • Select “Authentication App”
    • Click “Can’t Scan It?”, and copy the 16-character QR code

Facial Recognition:

  • For this feature to work, a little bit of setup is required
    • Create a parent directory named train and create more sub directories with the name of a people within the parent directory train
    • Now drop the images directly within the sub directories named after each person
    • Please note that even if the sub directories are not added, the face_recognition script is written in a way that it can learn from unrecognized/new faces by storing them with named sub directories within train
    • The camera id has been made dynamic so that it can choose the camera automatically
  • Try to keep the images as light as possible (which means both the display and file size of each image)
    • Reduced image size will not affect the accuracy as the facial recognition script converts each image into a pixel matrix
    • Reduced image size and display area can actually help in faster scanning and recognition
  • Make good of the tolerance level in the face recognition script
    • The learning_rate can be switched to match your needs
    • Learning rate generally depends on the clarity of the images stored and the lighting at which it was taken
    • Try to avoid images with very low resolution to maintain an adequate learning rate
    • Lower the learning rate, higher the tolerance level (which means the lower you go on learning rate, the stricter or perhaps more accurate matching it does)
    • Leeman's terms:
      • Increased learning_rate: Exact to No match
      • Decreased learning_rate: Incorrect to close match
      • 0.5 - 0.7 should suffice in most cases though

The API keys can be generated for free once you sign up except for Robinhood, which you know how it works. If not probably you shouldn't be using it.

Clone this wiki locally