Home

Everything Jarvis

📓   Wiki updated on November 19, 2022

Jarvis uses specific keywords to trigger the respective function which performs a specific task. It is not 100% fool proof but most of the common errors are caught and responses are configured accordingly. If there are any unexpected exceptions please raise an issue.

Restrictions:

Currently, Jarvis can run only of macOS and Windows.

Tested on MacOS High Sierra, Mojave, Catalina, Big Sur, Monterey, Ventura* and Windows 10

Known Issue with pyttsx3 module on macOS Ventura 13.0: This version of macOS does not hold the attribute VoiceAge. Workaround has been raised as a PR

Usage:

• Jarvis works by automatically detecting the Operating System it is being run on.
• To set up requirements, run the install.sh script following the instructions in README.md.
• Some key features require API keys, but they can be generated for free.

None of the features in Jarvis require subscriptions.

Stability

• There are broad exception clauses implemented to prevent Jarvis from breaking.
• To make sure Jarvis is always connected to the internet, it runs a connection checker in a background process.
• The connection checker uses OS agnostic modules to enable Wi-Fi and connect to a given SSID (stored as env vars).

Offline Communication with Jarvis:

• Using Telegram:

  1. A Telegram bot has to be created using BotFather
  2. Token has to be added to the env var BOT_TOKEN
  3. List of chat ids have to be added to the env var BOT_CHAT_IDS
  4. List of bot usernames have to be added to the env var BOT_USERS

• Using FastAPI:

  1. Hosts the offline communicator on localhost.
  2. Requires a port number added to the env var OFFLINE_PORT. Defaults to 4483
  3. Requires a password for authentication added to the env var OFFLINE_PASS. Defaults to OfflineComm

Features:

• Lock, unlock, honk & blink, remote start/stop and set the AC temperature in any JLR (Jaguar Land Rover) vehicles.
• Open and close garage door using MyQ garage controller.
• Control any LG Web OS Television.
• Guard the surroundings using face detection and audio recognition. Sends an email and SMS alert in case of an intruder.
• Send a text message to most US based mobile numbers.
• Facial recognition and detection. Click here to read the setup instructions.
• Automatically turn on Wi-Fi and connect to a set SSID. Also happens automatically anytime within 10 seconds of internet disconnect.
• Scale up/down a vpn server on demand using vpn-server module
• Monitor stocks using the stock monitor endpoint.
• Run cron scheduled jobs using regular cron expressions without the need of a crontab.
• Run certain tasks in the background every few seconds/minutes/hours/days.
• Set up a reminder at a said time and send a message to your phone at the given time.
• Set an alarm with the desired time.
• Wish you on events/festivals and birthday using an env var.
• Increase or decrease master volume of your machine via voice commands.
• Locate, ring and enable lost mode on any of the user's Apple devices.
• Control smart lights (that uses MagicHome application) in the same IP range.
• Read Outlook/Calendar and inform about meetings in the next 12 hours. macOS only
• Tell public and private IP address of the running device.
• Swap voices on demand.
• Adjust Screen Brightness. macOS only
• Tell system vitals including boot time, fan speed*, CPU and GPU temperature*. macOS only
• Restart the running device and suggest a restart in case of high boot time.
• Scan local ip range to get smart devices connected which acts as IP feeder for smart lights and TV.
• Tell random facts.
• Heads or Tails.
• Can scan and connect/disconnect to nearby bluetooth devices.
• Take notes and saves it to a notes.txt file.
• Tell a joke.
• Tell the number of repositories on your GitHub and clone a particular one.
• Tell the list of google home devices in your IP range.
• Get you the distance between two places or distance from your location to a particular place.
• Create or remove tasks for todo lists.
• Scan your Music folder for .mp3 and plays them on any smart devices in IP range.
• Look for unread emails in your gmail account.
• Get the weather information at any location.
• Get the news update from fox-news.
• Get your investment details using Robinhood API.
• Get facts from Wikipedia using wiki api.
• Open up the webpage you want in your default browser.
• Open any application installed in running machine.
• Get the current date and time.
• Open a Google search for any query.
• Shutdown/restart the running device.
• Respond to most basic conversations.

• If anything apart from the below is requested, Jarvis uses google's Places API to match the requested phrase and suggest options. If the requested phrase doesn't resolve on Places API, it then uses google search engine parser to get results for the phrase. If anything beyond that, Jarvis will launch a google search on your default browser.

Investment Details:

• Login to your Robinhood Web App
• Go to Account -> Settings or click me
• Turn on Two-Factor Authentication
  • Select “Authentication App”
  • Click “Can’t Scan It?”, and copy the 16-character QR code

Facial Recognition:

• For this feature to work, a little bit of setup is required
  • Create a parent directory named train and create more sub directories with the name of a people within the parent directory train
  • Now drop the images directly within the sub directories named after each person
  • Please note that even if the sub directories are not added, the face_recognition script is written in a way that it can learn from unrecognized/new faces by storing them with named sub directories within train
  • The camera id has been made dynamic so that it can choose the camera automatically
• Try to keep the images as light as possible (which means both the display and file size of each image)
  • Reduced image size will not affect the accuracy as the facial recognition script converts each image into a pixel matrix
  • Reduced image size and display area can actually help in faster scanning and recognition
• Make good of the tolerance level in the face recognition script
  • The learning_rate can be switched to match your needs
  • Learning rate generally depends on the clarity of the images stored and the lighting at which it was taken
  • Try to avoid images with very low resolution to maintain an adequate learning rate
  • Lower the learning rate, higher the tolerance level (which means the lower you go on learning rate, the stricter or perhaps more accurate matching it does)
  • Leeman's terms:
    • Increased learning_rate: Exact to No match
    • Decreased learning_rate: Incorrect to close match
    • 0.5 - 0.7 should suffice in most cases though

The API keys can be generated for free once you sign up except for Robinhood, which you know how it works. If not probably you shouldn't be using it.