diff --git a/developer_manual/client_apis/OCS/index.rst b/developer_manual/client_apis/OCS/index.rst index 5c1e6b88e69..565bb1993c1 100644 --- a/developer_manual/client_apis/OCS/index.rst +++ b/developer_manual/client_apis/OCS/index.rst @@ -12,6 +12,7 @@ The old documentation is still kept as it provides some additional documentation ocs-openapi ocs-api-overview + ocs-assistant-api ocs-share-api ocs-sharee-api ocs-status-api diff --git a/developer_manual/client_apis/OCS/ocs-assistant-api.rst b/developer_manual/client_apis/OCS/ocs-assistant-api.rst new file mode 100644 index 00000000000..1e2a1e02709 --- /dev/null +++ b/developer_manual/client_apis/OCS/ocs-assistant-api.rst @@ -0,0 +1,26 @@ +.. _ocs-assistant-api: + +================= +Assistant OCS API +================= + +The assistant OCS API lets you schedule and manage AI tasks, retrieve task history, and query +task results from any client or application. + +The API is documented using OpenAPI. To browse the full specification: + +1. Install the `OCS API viewer app `_. +2. Install the *assistant* app. +3. Open the **OCS API viewer** from the app menu. +4. Select **assistant** in the left sidebar. + +.. note:: + The assistant API currently has separate endpoints for Text Processing, Speech to Text, and + Image Generation. These will be unified in a future release, which may introduce breaking + changes. + +Related documentation +--------------------- + +- Server-side integration: :doc:`../../digging_deeper/task_processing` +- Frontend integration: :doc:`../../digging_deeper/assistant_integration` diff --git a/developer_manual/digging_deeper/ai.rst b/developer_manual/digging_deeper/ai.rst index 8667fe43bfd..e2a6fd37be4 100644 --- a/developer_manual/digging_deeper/ai.rst +++ b/developer_manual/digging_deeper/ai.rst @@ -7,6 +7,7 @@ AI & Machine Learning context_chat task_processing + assistant_integration machinetranslation speech-to-text text_processing diff --git a/developer_manual/digging_deeper/assistant_integration.rst b/developer_manual/digging_deeper/assistant_integration.rst new file mode 100644 index 00000000000..f1a08cd9932 --- /dev/null +++ b/developer_manual/digging_deeper/assistant_integration.rst @@ -0,0 +1,145 @@ +.. _assistant-integration: + +========================= +Integrating the assistant +========================= + +This section covers integrating the Nextcloud assistant into the web frontend of other Nextcloud +applications. For backend integration using the Task Processing OCP API, see +:doc:`task_processing`. For the OCS API, see +:doc:`../client_apis/OCS/ocs-assistant-api`. + +Displaying a task result +------------------------ + +There are two ways to display a task result using the assistant. + +Open the assistant modal +^^^^^^^^^^^^^^^^^^^^^^^^ + +If you have a task object retrieved from the +``/ocs/v2.php/apps/assistant/api/v1/task/TASK_ID`` or +``/ocs/v2.php/apps/assistant/api/v1/tasks`` OCS endpoint, pass it to the helper function +``OCA.Assistant.openAssistantTask``. This opens the assistant modal with the task type, input, +and output pre-loaded. + +Browse the task result page +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A standalone page is available at ``/apps/assistant/task/view/TASK_ID``. It renders the same +content as the assistant modal. + +Running a task +-------------- + +Use ``OCA.Assistant.openAssistantForm`` to open the assistant modal from your application. The +function accepts a single configuration object with the following keys: + +.. list-table:: ``openAssistantForm`` options + :header-rows: 1 + :widths: 20 10 70 + + * - Key + - Required + - Description + * - ``appId`` + - Yes + - App ID of the calling application. + * - ``customId`` + - No + - Custom identifier for the task; useful when correlating the ``task finished`` backend + event with the originating call. Defaults to ``''``. + * - ``taskType`` + - No + - Initially selected task type (e.g. ``core:text2text``, ``speech-to-text``, + ``OCP\TextToImage\Task``). Defaults to the last used task type. + * - ``input`` + - No + - Object containing initial input values, specific to each task type. Defaults to ``{}``. + * - ``isInsideViewer`` + - No + - Set to ``true`` if the function is called while the Viewer is open. Defaults to ``false``. + * - ``closeOnResult`` + - No + - If ``true``, the modal closes after a synchronous task completes and results are + available. Defaults to ``false``. + * - ``actionButtons`` + - No + - List of extra buttons to show in the result form. Only used when ``closeOnResult`` is + ``false``. Defaults to an empty list. + +The function returns a Promise that resolves when the modal is closed — either because a task +was scheduled, or a synchronous task ran and produced results. The promise resolves with a task +object: + +.. code-block:: javascript + + { + appId: 'text', + id: 310, + customId: 'my custom identifier', + input: { input: 'give me a short summary of a simple settings section about GitHub' }, + ocpTaskId: 152, + output: { output: 'blabla' }, + status: 'STATUS_SUCCESSFUL', + type: 'core:text2text', + lastUpdated: 1711545305, + scheduledAt: 1711545301, + startedAt: 1711545302, + endedAt: 1711545303, + userId: 'janedoe', + } + +Possible ``status`` values are: ``STATUS_UNKNOWN`` (0), ``STATUS_SCHEDULED`` (1), +``STATUS_RUNNING`` (2), ``STATUS_SUCCESSFUL`` (3), ``STATUS_FAILED`` (4). + +Complete example: + +.. code-block:: javascript + + OCA.Assistant.openAssistantForm({ + appId: 'my_app_id', + customId: 'my custom identifier', + taskType: 'core:text2text', + inputs: { input: 'count to 3' }, + actionButtons: [ + { + label: 'Label 1', + title: 'Title 1', + variant: 'warning', + iconSvg: cogSvg, + onClick: (output) => { console.debug('first button clicked', output) }, + }, + { + label: 'Label 2', + title: 'Title 2', + onClick: (output) => { console.debug('second button clicked', output) }, + }, + ], + }).then(task => { + console.debug('assistant promise success', task) + }).catch(error => { + console.debug('assistant promise failure', error) + }) + +Populating input from a file +----------------------------- + +You can pre-fill a task input field with the content of a file by passing a ``fileId`` or +``filePath`` instead of a plain string value: + +.. code-block:: javascript + + OCA.Assistant.openAssistantForm({ + appId: 'my_app_id', + customId: 'my custom identifier', + taskType: 'core:text2text', + inputs: { input: { fileId: 123 } }, + }) + + OCA.Assistant.openAssistantForm({ + appId: 'my_app_id', + customId: 'my custom identifier', + taskType: 'core:text2text', + inputs: { input: { filePath: '/path/to/file.txt' } }, + })