Merge from main

.github/workflows/dockerize.yml

@@ -14,17 +14,6 @@ on:
       - prod.Dockerfile
       - docker-compose.yml
       - .github/workflows/dockerize.yml
-  pull_request:
-    branches:
-      - master
-    paths:
-      - src/khoj/**
-      - src/interface/web/**
-      - pyproject.toml
-      - Dockerfile
-      - prod.Dockerfile
-      - docker-compose.yml
-      - .github/workflows/dockerize.yml
   workflow_dispatch:
     inputs:
       tag:

documentation/docs/clients/obsidian.md

@@ -6,6 +6,8 @@ sidebar_position: 3
 
 > Query your Second Brain from Obsidian
 
+![demo](https://assets.khoj.dev/obsidian_khoj_side_panel_pak_telemedicine.gif)
+
 ## Features
 - **Chat**
   - **Faster answers**: Find answers quickly, from your private notes or the public internet
@@ -14,13 +16,8 @@ sidebar_position: 3
 - **Search**
   - **Natural**: Advanced natural language understanding using Transformer based ML Models
   - **Incremental**: Incremental search for a fast, search-as-you-type experience
-
-## Interface
-
-| Search | Chat |
-|:------:|:----:|
-| ![](/img/khoj_search_on_obsidian.png) | ![](/img/khoj_chat_on_obsidian.png) |
-
+- **Similar**
+  - **Discover**: Find similar notes to the current one
 
 ## Setup
 

documentation/docs/features/automations.md

@@ -0,0 +1,9 @@
+# Automations
+
+[Automations](https://app.khoj.dev/automations) are a powerful feature within Khoj to schedule repeated tasks for information retrieval directly from your account. You can run them at a specific time and interval. This is still an experimental feature, so please report any issues you encounter.
+
+Khoj will use your local time zone to determine the scheduling localization. You can go back and configure the prompt any time you want from the automations page. You can also delete the automation if you no longer need it.
+
+:::danger[Note]
+Automations will not deliver emails to self-hosted users out of the box. You'll have to have Resend and [Google Auth](/miscellaneous/google_auth) setup to send emails.
+:::

documentation/docs/features/chat.md

@@ -4,10 +4,12 @@ sidebar_position: 2
 
 # Chat
 
-You can configure Khoj to chat with you about anything. When relevant, it'll use any notes or documents you shared with it to respond.
+You can configure Khoj to chat with you about anything. When relevant, it'll use any notes or documents you shared with it to respond. It acts as an excellent research assistant, search engine, or personal tutor.
+
+<img src="/img/khoj_chat_on_web.png" alt="Chat on Web" style={{width: '400px'}}/>
 
 ### Overview
-- Creates a personal assistant for you to inquire and engage with your notes
+- Creates a personal assistant for you to inquire and engage with your notes or online information as needed
 - You can choose to use Online or Offline Chat depending on your requirements
 - Supports multi-turn conversations with the relevant notes for context
 - Shows reference notes used to generate a response
@@ -48,12 +50,16 @@ This will enable Khoj to send your chat queries and query relevant notes to Open
     - **On Emacs**: Run `M-x khoj <user-query>`
 2. Enter your queries to chat with Khoj. Use [slash commands](#commands) and [query filters](/miscellaneous/advanced#query-filters) to change what Khoj uses to respond
 
-![](/img/khoj_chat_on_web.png ':size=400px')
 
 #### Details
 1. Your query is used to retrieve the most relevant notes, if any, using Khoj search
 2. These notes, the last few messages and associated metadata is passed to the enabled chat model along with your query to generate a response
 
+#### Conversation File Filters
+You can use conversation file filters to limit the notes used in the chat response. To do so, use the left panel in the web UI. Alternatively, you can also use [query filters](/miscellaneous/advanced#query-filters) to limit the notes used in the chat response.
+
+<img src="/img/select_file_filter.png" alt="Conversation File Filter" style={{width: '400px'}}/>
+
 #### Commands
 Slash commands allows you to change what Khoj uses to respond to your query
 - **/notes**: Limit chat to only respond using your notes, not just Khoj's general world knowledge as reference
@@ -62,3 +68,4 @@ Slash commands allows you to change what Khoj uses to respond to your query
 - **/online**: Use online information and incorporate it in the prompt to the LLM to send you a response.
 - **/image**: Generate an image in response to your query.
 - **/help**: Use /help to get all available commands and general information about Khoj
+- **/summarize**: Can be used to summarize 1 selected file filter for that conversation. Refer to [File Summarization](summarization) for details.

documentation/docs/features/keyboard_shortcuts.md

@@ -0,0 +1,18 @@
+# Keyboard Shortcuts
+
+Oftentimes, having to leave your keyboard to move the mouse can break your flow. We want to make it as easy as possible for AI to flow with you as you are, so we've added some keyboard shortcuts to facilitate that.
+
+## Web App
+
+### Up/Down Arrow Keys (Chat Input)
+
+- **Up Arrow Key**: Move up in the list of most recent messages in your chat window.
+- **Down Arrow Key**: Move down in the list of most recent messages in your chat window.
+
+You can watch the demo to see how it works on [this sample conversation](http://app.khoj.dev/share/chat/in-particular-assess-the-prospect-for-brazil-/).
+
+<img src="https://assets.khoj.dev/up_down_shortcuts.gif" height="300" alt="Up/Down Arrow Keys"></img>
+
+### Enter (Chat Input)
+
+Press 'Enter' to send the message you've typed in the chat window.

documentation/docs/features/summarization.md

@@ -0,0 +1,26 @@
+---
+sidebar_position: 5
+---
+
+# File Summarization
+
+You can use the `/summarize` command to get Khoj to generate context driven summaries of your documents.
+Simply select a single file filter on the left panel menu and then use `/summarize [any context]` and Khoj
+will produce a tailored summary of the text.
+
+You can also try a natural language query which include the intent for summary without explicitly using the `/summarize` command.
+
+## Design Diagram
+
+<img src="/img/summarize.jpg" alt="Chat on Web" style={{width: '800px'}}/>
+
+## Example Usage
+
+* `/summarize in a way that can be used as practice questions for a test`
+* `/summarize in a way a toddler can understand`
+* `/summarize in one paragraph`
+
+Without using the `/summarize` command:
+* `create a summary of the document in a way that can be used as practice questions for a test`
+* `summarize the document in a way a toddler can understand`
+* `generate a one paragraph summary of the document`

documentation/docs/miscellaneous/authentication.md

@@ -0,0 +1,30 @@
+# Authentication when Self-Hosting
+
+By default, most of the instructions for self-hosting Khoj assume a single user, and so the default configuration is to run in anonymous mode. However, if you want to enable authentication, you can do so either with [Google Auth](/miscellaneous/google_auth) or with magic links, as shown below. This can be helpful if you want to make sure your Khoj instance is only accessible to you and your team.
+
+:::tip[Note]
+Remove the `--anonymous-mode` flag in your start up command to enable authentication.
+:::
+
+The most secure way to do this is to integrate with [Resend](https://resend.com) by setting up an account and adding an environment variable for `RESEND_API_KEY`. You can get your API key [here](https://resend.com/api-keys). This will allow you to automatically send sign-in links to users who want to log in.
+
+It's still possible to use the magic links feature without Resend, but you'll need to manually send the magic links to users who want to log in.
+
+## Manually sending magic links
+
+1. The user will have to enter their email address in the login form.
+They'll click `Send Magic Link`. Without the Resend API key, this will just create an unverified account for them in the backend
+<img src="/img/magic_link.png" alt="Magic link login form" width="400"/>
+
+2. You can get their magic link using the admin panel
+Go to the [admin panel](http://localhost:42110/server/admin/database/khojuser/). You'll see a list of users. Search for the user you want to send a magic link to. Tick the checkbox next to their row, and use the action drop down at the top to 'Get email login URL'. This will generate a magic link that you can send to the user, which will appear at the top of the admin interface.
+
+| Get email login URL | Retrieved login URL |
+|---------------------|---------------------|
+| <img src="/img/admin_get_emali_login.png" alt="Get user magic sign in link" width="400" />| <img src="/img/admin_successful_login_url.png" alt="Successfully retrieved a login URL" width="400" />|
+
+3. Send the magic link to the user. They can click on it to log in.
+
+Once they click on the link, they'll automatically be logged in. They'll have to repeat this process for every new device they want to log in from, but they shouldn't have to repeat it on the same device.
+
+A given magic link can only be used once. If the user tries to use it again, they'll be redirected to the login page to get a new magic link.