feat(frontend): integrate external apps with nesis

docs/README.md

@@ -7,7 +7,7 @@ We use mkdocs to create Nesis documentation.
 
 1. Install dependencies with
     ```commandline
-    pip install -r requirements-docs.txt
+    pip install -r requirements.txt
     ```
 2. Serve the documentation on your local with
     ```commandline

docs/mkdocs.yml

@@ -44,10 +44,15 @@ nav:
       - 'installing/helm.md'
       - 'installing/ametnes.md'
   - 'Access Control': 'rbac.md'
+  - 'App Integration': 'apps/example.ipynb'
   - 'Development Guide':
       - 'Local Development': 'dev-guide/local.md'
       - 'Architecture': 'dev-guide/architecture.md'
       - 'Contributing': 'dev-guide/contributing.md'
+
+plugins:
+  - mkdocs-jupyter
+
 extra_css:
     - css/extra.css
 extra_javascript:

docs/requirements.txt

@@ -3,3 +3,4 @@ jinja2==3.1.3
 mkdocs-material==9.0.5
 mkdocs-bootstrap==1.1
 mkdocs-bootswatch==1.1
+mkdocs-jupyter==0.24.7

docs/src/apps/example.ipynb

@@ -0,0 +1,357 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Developing Apps\n",
+    "\n",
+    "The Nesis' API gives you the ability to develop apps (new apps or extend existing apps) that interact with your private document and data repositories. Using Nesis' security model, you are able to apply role based access control to your data such that users are only able to seamlessly access data tied to their security profile.\n",
+    "\n",
+    "Some use cases for the Nesis API.\n",
+    "1. You already have a mobile banking app and want to give it generative AI capabilities to help you customers to interract with their bank statements from within your mobile banking app.\n",
+    "2. You have an existing internal legacy customer service application that you'd like to extend and give AI capabilities.\n",
+    "3. You want to add generative AI powered conversation interraction to your website so that your users have tailored responses based on their personalized data.\n",
+    "\n",
+    "\n",
+    "## Create a Nesis App\n",
+    "In the Nesis frontend,\n",
+    "\n",
+    "Navigate to **Settings** -> **Roles** and create a role with these properties\n",
+    "1. Name: `nesis-admin-app`\n",
+    "2. Policy:\n",
+    "\n",
+    "    ```\n",
+    "    {\n",
+    "        \"items\": [\n",
+    "            {\n",
+    "                \"action\": \"create\",\n",
+    "                \"resource\": \"users/*\"\n",
+    "            },\n",
+    "            {\n",
+    "                \"action\": \"update\",\n",
+    "                \"resource\": \"users/*\"\n",
+    "            },\n",
+    "            {\n",
+    "                \"action\": \"update\",\n",
+    "                \"resource\": \"roles/*\"\n",
+    "            },\n",
+    "            {\n",
+    "                \"action\": \"create\",\n",
+    "                \"resource\": \"roles/*\"\n",
+    "            },\n",
+    "            {\n",
+    "                \"action\": \"update\",\n",
+    "                \"resource\": \"datasources/*\"\n",
+    "            },\n",
+    "            {\n",
+    "                \"action\": \"create\",\n",
+    "                \"resource\": \"datasources/*\"\n",
+    "            }\n",
+    "        ]\n",
+    "    }\n",
+    "    ```\n",
+    "\n",
+    "Navigate to **Settings** -> **Apps** and then create an app with the following details\n",
+    "1. Name: `nesis-admin-app`\n",
+    "2. Description: `Nesis admin application`\n",
+    "3. Enabled: `Checked`\n",
+    "4. Role: `nesis-admin-app`\n",
+    "\n",
+    "Note down the API key generated"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "\n",
+    "Navigate to **Settings** -> **Apps** and then create another app with the following details\n",
+    "1. Name: `nesis-prediction-app`\n",
+    "2. Description: `Nesis prediction application`\n",
+    "3. Enabled: `Checked`\n",
+    "4. Role: Do not select a role\n",
+    "\n",
+    "Note down the API key generated\n",
+    "\n",
+    "> This is all you need to do in the UI. The rest is all API driven using the API keys."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Create a datasource"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "import requests\n",
+    "\n",
+    "nesis_api_endpoint = \"http://nesis-endpoint.com/v1\"\n",
+    "admin_api_key = \"the-admin-api-key\"\n",
+    "predictions_api_key = \"the-prediction-app-api-key\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "admin_api_headers = {'Content-Type': 'application/json', 'Authorization': f'Bearer {admin_api_key}'}\n",
+    "prediction_api_header = {'Content-Type': 'application/json', 'Authorization': f'Bearer {predictions_api_key}'}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "datasource = {\"type\": \"minio\", \"name\": \"documents\", \"connection\": {\"user\": \"your_username\",\"password\":\"your_password\",\"endpoint\": \"http://minio:port\",\"dataobjects\": \"bucket-name\"}}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "datasource_response = requests.post(f'{nesis_api_endpoint}/datasources', headers=admin_api_headers, json=datasource)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "datasource = json.loads(datasource_response.text)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Create a role to be used to query the datasource"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "role = {\n",
+    "    \"name\": \"data-reader\",\n",
+    "    \"policy\": {\n",
+    "      \"items\": [\n",
+    "        {\n",
+    "          \"action\": \"create\",\n",
+    "          \"resource\": \"predictions/*\"\n",
+    "        },\n",
+    "        {\n",
+    "          \"action\": \"read\",\n",
+    "          \"resource\": \"datasources/*\"\n",
+    "        }\n",
+    "      ]\n",
+    "    }\n",
+    "}\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "role_response = requests.post(f'{nesis_api_endpoint}/roles', headers=admin_api_headers, json=role)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 26,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{\"create_date\":\"2024-05-24 18:45:28\",\"id\":\"f0e63ee3-d1af-4d56-9b99-e28bf308db45\",\"name\":\"data-reader\"}\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(role_response.text)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "role = json.loads(role_response.text)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Create a user and assign the above role"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "user_response = requests.post(f'{nesis_api_endpoint}/users', headers=admin_api_headers, json={'email': 'test.email@domain.com', 'name': 'Da Zone', 'password': 'P@ssword', 'roles': [role['id']]})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{\"attributes\":null,\"create_date\":\"2024-05-24 18:47:01\",\"email\":\"some.email@domain.com\",\"enabled\":true,\"id\":\"b919c24d-7574-43ec-9889-200584e44d6b\",\"name\":\"Da Zone\",\"root\":false}\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(user_response.text)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "user = json.loads(user_response.text)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Query using the prediction API key. \n",
+    "\n",
+    "> This fails because no role was attached to the prediction app."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prediction_response = requests.post(f'{nesis_api_endpoint}/modules/qanda/predictions', headers=prediction_api_header, json={'query': 'summarize the only office agreement'})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "<Response [403]>\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(prediction_response)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Query using the prediction API key and user header\n",
+    "> This succeeds because the prediction app inherits permissions from the user (who has the right permission) using the `X-Nesis-Request-UserKey` header."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prediction_response = requests.post(f'{nesis_api_endpoint}/modules/qanda/predictions', headers={**prediction_api_header, 'X-Nesis-Request-UserKey': user['id']}, json={'query': 'summarize the only office agreement'})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 41,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{\"create_date\":\"2024-05-24 18:59:17\",\"data\":{\"choices\":[{\"delta\":null,\"finish_reason\":\"stop\",\"index\":0,\"message\":{\"content\":\"The Only Office Agreement is a legal document that outlines the terms and conditions of leasing office space. It typically includes details such as the duration of the lease, rental payments, maintenance responsibilities, and any other specific provisions agreed upon by the landlord and tenant. This agreement serves to protect the rights and obligations of both parties involved in the leasing of office space.\",\"role\":\"assistant\"},\"sources\":[]}],\"created\":1716577157,\"id\":\"a0bc9a93-a3e5-45e8-9d4f-090f652e8dc2\",\"model\":\"rag\",\"object\":\"completion\"},\"id\":null,\"input\":\"summarize the only office agreement\",\"module\":\"qanda\"}\n",
+      "\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(prediction_response.text)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Conclusion\n",
+    "This notebook shows you a simplistic workflow of how to programatically interract with Nesis. We see how to create an app and assign it **CREATE** permissions on **USERS**, **DATASOURCES** and **ROLES**. With this app, we are able to programmatically create multiple users, datasources and roles.\n",
+    "\n",
+    "By doing this you are able to integrate Nesis programically into your existing worklows, automating the creation of Nesis users and roles inline with your existing enterprise user management system and do this at scale.\n",
+    "\n",
+    "Lastly, we see how Nesis can be easily integrated into your existing application such as a website and easily tailored responses to your logged in website users, thus giving your website generative AI capabilities.\n",
+    "\n",
+    "Next steps;\n",
+    "\n",
+    "1. View the Nesis repository https://github.com/ametnes/nesis/\n",
+    "2. View the documentation https://ametnes.github.io/nesis/.\n",
+    "3. Join our [Slack community](https://join.slack.com/t/nesiscommunity/shared_invite/zt-2gpp38ts2-tAfea6R_q9RHudhPEcBJSA) and let us know how you get on or ask any questions."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}