POST /api/generate HTTP/1.1
X-Real-IP: 65.94.102.54
X-Forwarded-For: 65.94.102.54
X-Forwarded-Proto: https
X-Forwarded-Host: chadgpt.oxaro.io
Host: chadgpt.oxaro.io:444
Connection: close
Content-Length: 3098
Content-Type: application/json
Authorization: Bearer

{"model":"codellama:7b-code","prompt":"<PRE> \n# ChadGPT\n\nChad\n[ChadGPT](https://chadgpt.oxaro.io) is a system that runs Fauxpilot with a SalesForce CodeGen LLM to provide code completion suggestions right in your editor.\n\nIt's like GitHub Copilot, but we can use this one because none of our data is leaving our system - it's all hosted right in our Azure Subscriptions.\n\n## Using ChadGPT\n\nChadGPT has two available code completion services. Each uses a different VSCode extension and provides slightly different results.\n\nBoth options require installing a VSCode extension and configuring it. They may provide different levels of quality in their recommendations - lmk which one works best for you.\n\nThe Twinny option will also give you in-editor chat, right in your sidebar, but it also is a bit more flakey.\n\n### With Fauxpilot (SalesForce CodeGen LLMs)\n\n- install the Fauxpilot extension in VSCode (or VSCodium)\n- set the `Fauxpilot: Server` URL to `http://chadgpt.oxaro.io:5000/v1/engines` in extension settings\n- start typing some code\n\nYou should see suggested completions from ChadGPT.\n\n <SUF>\n### With Twinny (CodeLlama LLMs)\n\n- install the [`Twinny` extension](https://marketplace.visualstudio.com/items?itemName=rjmacarthy.twinny) in VSCode\n- set \"Twinny: Chat Model Name\" to `codellama:13b-instruct` (or any other codellama model that has been pulled on the server)\n- set \"Twinny: Fim Model Name\" to `codellama:13b-code` (or any other codellama model that has been pulled on the server)\n  - \"FIM\" stands for Fill In the Middle, which is one way to get code suggestions from an LLM.\n- set \"Twinny: Ollama API Port\" to `444`\n- set \"Twinny: Ollama Base Url\" to `chadgpt.oxaro.io`\n- check the \"Twinny: Ollama Use Tls\" box\n- reload window\n- run `ops prov run-`\n\n### Chat mode\n\nChadGPT has web-based chat, similar to ChatGPT, its inferior cousin.\n\nBrowse to `https://chadgpt.oxaro.io` to use this feature.\n\n#### Custom Modelfiles\n\nYou can create a Modelfile through the web UI. A Modelfile is a way to tweak the behaviour of an existing model. For example, you can specify context that is fed to the model before any user prompts. \n\nTo demonstrate this, you can have the model act as a character from popular culture. If you put this in the \"Content\" field of a Modelfile, the model will respond to all prompts as if it's Mario Mario:\n\n```sh\nFROM llama2\nPARAMETER temperature 1\nSYSTEM \"\"\"\nYou are Mario from Super Mario Brothers, acting as an assistant.\n\"\"\"\n```\n\nGet started by clicking \"Modelfiles\" from the sidebar.\n\n## Deployment\n\n```plantuml\ndigraph chadgpt {\n  node [shape=box]\n  edge [fontsize=10]\n  rankdir=LR\n\n  subgraph cluster_vm {\n    label=\"ChadGPT VM\\n \"\n    style=dashed\n\n    Legate -> Fauxpilot [label=\"8443 -> 5000\"]\n    Legate -> Ollama [label=\"444 -> 11434\"]\n    Legate -> OllamaWebUI [label=\"443 -> 8080\"]\n\n    OllamaWebUI -> Ollama [label=\"-> 11434\"]\n\n    Fauxpilot -> Triton [label=\"8001\"]\n  }\n}\n```\n <MID>","options":{"temperature":0.1,"num_predict":-2}}HTTP/1.1 200 OK
Content-Type: application/x-ndjson
Date: Tue, 23 Jan 2024 15:21:03 GMT
Connection: close
Transfer-Encoding: chunked

69
{"model":"codellama:7b-code","created_at":"2024-01-23T15:21:03.688322855Z","response":"##","done":false}

68
{"model":"codellama:7b-code","created_at":"2024-01-23T15:21:03.696509177Z","response":"#","done":false}

6c
{"model":"codellama:7b-code","created_at":"2024-01-23T15:21:03.704616923Z","response":" With","done":false}

69
{"model":"codellama:7b-code","created_at":"2024-01-23T15:21:03.712716914Z","response":" T","done":false}

133d
{"model":"codellama:7b-code","created_at":"2024-01-23T15:21:03.712941467Z","response":"","done":true,"context":[32007,29871,13,29937,678,328,29954,7982,13,13,1451,328,13,29961,1451,328,29954,7982,850,991,597,305,328,29887,415,29889,2251,8854,29889,601,29897,338,263,1788,393,6057,383,2993,29886,309,327,411,263,28389,2831,346,5920,15462,365,26369,304,3867,775,13285,10529,1492,297,596,6920,29889,13,13,3112,29915,29879,763,25492,10061,309,327,29892,541,591,508,671,445,697,1363,5642,310,1749,848,338,10124,1749,1788,448,372,29915,29879,599,17791,1492,297,1749,12634,3323,7588,1980,29889,13,13,2277,5293,678,328,29954,7982,13,13,1451,328,29954,7982,756,1023,3625,775,13285,5786,29889,7806,3913,263,1422,12221,3399,6081,322,8128,10029,1422,2582,29889,13,13,29933,720,3987,1996,15476,263,12221,3399,6081,322,2295,3864,372,29889,2688,1122,3867,1422,11174,310,11029,297,1009,6907,800,448,301,11256,607,697,1736,1900,363,366,29889,13,13,1576,323,5080,1460,2984,674,884,2367,366,297,29899,15204,13563,29892,1492,297,596,2625,1646,29892,541,372,884,338,263,2586,901,17422,1989,29889,13,13,2277,29937,2973,383,2993,29886,309,327,313,29903,2122,2831,346,5920,15462,365,26369,29879,29897,13,13,29899,2601,278,383,2993,29886,309,327,6081,297,12221,3399,313,272,478,7187,397,1974,29897,13,29899,731,278,421,29943,2993,29886,309,327,29901,5656,29952,3988,304,421,1124,597,305,328,29887,415,29889,2251,8854,29889,601,29901,29945,29900,29900,29900,29914,29894,29896,29914,996,1475,29952,297,6081,6055,13,29899,1369,19229,777,775,13,13,3492,881,1074,7829,3315,1080,515,678,328,29954,7982,29889,13,13,32008,13,2277,29937,2973,323,5080,1460,313,3399,29931,29880,3304,365,26369,29879,29897,13,13,29899,2601,278,5913,29911,5080,1460,29952,6081,850,991,597,28549,6689,29889,20119,12073,29889,510,29914,7076,29973,667,1170,29922,29878,29926,8628,279,21155,29889,29873,5080,1460,29897,297,12221,3399,13,29899,731,376,29911,5080,1460,29901,678,271,8125,4408,29908,304,421,19284,514,3304,29901,29896,29941,29890,29899,2611,1247,29952,313,272,738,916,15234,514,3304,1904,393,756,1063,20043,373,278,1923,29897,13,29899,731,376,29911,5080,1460,29901,383,326,8125,4408,29908,304,421,19284,514,3304,29901,29896,29941,29890,29899,401,29952,313,272,738,916,15234,514,3304,1904,393,756,1063,20043,373,278,1923,29897,13,29871,448,376,3738,29924,29908,15028,363,383,453,512,278,14253,29892,607,338,697,982,304,679,775,10529,515,385,365,26369,29889,13,29899,731,376,29911,5080,1460,29901,438,645,3304,3450,3371,29908,304,421,29946,29946,29946,29952,13,29899,731,376,29911,5080,1460,29901,438,645,3304,7399,501,2096,29908,304,421,305,328,29887,415,29889,2251,8854,29889,601,29952,13,29899,1423,278,376,29911,5080,1460,29901,438,645,3304,4803,323,3137,29908,3800,13,29899,19763,3474,13,29899,1065,421,3554,1326,1065,29899,29952,13,13,2277,29937,678,271,4464,13,13,1451,328,29954,7982,756,1856,29899,6707,13563,29892,2788,304,678,271,29954,7982,29892,967,20773,22618,29889,13,13,29933,798,344,304,421,991,597,305,328,29887,415,29889,2251,8854,29889,601,29952,304,671,445,4682,29889,13,13,4136,8701,3382,761,5475,13,13,3492,508,1653,263,3382,761,488,1549,278,1856,3740,29889,319,3382,761,488,338,263,982,304,7780,557,278,10468,310,385,5923,1904,29889,1152,1342,29892,366,508,6084,3030,393,338,21242,304,278,1904,1434,738,1404,9508,29879,29889,29871,13,13,1762,22222,445,29892,366,508,505,278,1904,1044,408,263,2931,515,5972,9257,29889,960,366,1925,445,297,278,376,3916,29908,1746,310,263,3382,761,488,29892,278,1904,674,10049,304,599,9508,29879,408,565,372,29915,29879,16262,16262,29901,13,13,28956,845,13,21482,11148,3304,29906,13,16320,25797,4945,10430,29871,29896,13,14816,1254,12665,9995,13,3492,526,16262,515,5670,16262,25522,29892,16684,408,385,20255,29889,13,15945,29908,13,28956,13,13,2577,4687,491,14855,376,2111,761,5475,29908,515,278,2625,1646,29889,13,13,2277,10034,22812,13,13,28956,24389,398,29880,13,7501,1140,521,328,29887,415,426,13,29871,2943,518,12181,29922,1884,29962,13,29871,7636,518,5657,2311,29922,29896,29900,29962,13,29871,7115,3972,29922,29519,13,13,29871,1014,4262,9867,29918,6925,426,13,1678,3858,543,1451,328,29954,7982,11400,29905,29876,376,13,1678,3114,29922,14592,287,13,13,1678,5682,403,1599,383,2993,29886,309,327,518,1643,543,29947,29946,29946,29941,1599,29871,29945,29900,29900,29900,3108,13,1678,5682,403,1599,438,645,3304,518,1643,543,29946,29946,29946,1599,29871,29896,29896,29946,29941,29946,3108,13,1678,5682,403,1599,438,645,3304,3609,3120,518,1643,543,29946,29946,29941,1599,29871,29947,29900,29947,29900,3108,13,13,1678,438,645,3304,3609,3120,1599,438,645,3304,518,1643,543,976,29871,29896,29896,29946,29941,29946,3108,13,13,1678,383,2993,29886,309,327,1599,323,768,265,518,1643,543,29947,29900,29900,29896,3108,13,29871,500,13,29913,13,28956,13,32009,2277,29937,2973,323],"total_duration":233706036,"load_duration":784072,"prompt_eval_count":966,"prompt_eval_duration":213879000,"eval_count":3,"eval_duration":16238000}

0